Sensory Contributions to Impaired Emotion Processing in Schizophrenia

Abstract

Both emotion and visual processing deficits are documented in schizophrenia, and preferential magnocellular visual pathway dysfunction has been reported in several studies. This study examined the contribution to emotion-processing deficits of magnocellular and parvocellular visual pathway function, based on stimulus properties and shape of contrast response functions. Experiment 1 examined the relationship between contrast sensitivity to magnocellular- and parvocellular-biased stimuli and emotion recognition using the Penn Emotion Recognition (ER-40) and Emotion Differentiation (EMODIFF) tests. Experiment 2 altered the contrast levels of the faces themselves to determine whether emotion detection curves would show a pattern characteristic of magnocellular neurons and whether patients would show a deficit in performance related to early sensory processing stages. Results for experiment 1 showed that patients had impaired emotion processing and a preferential magnocellular deficit on the contrast sensitivity task. Greater deficits in ER-40 and EMODIFF performance correlated with impaired contrast sensitivity to the magnocellular-biased condition, which remained significant for the EMODIFF task even when nonspecific correlations due to group were considered in a step-wise regression. Experiment 2 showed contrast response functions indicative of magnocellular processing for both groups, with patients showing impaired performance. Impaired emotion identification on this task was also correlated with magnocellular-biased visual sensory processing dysfunction. These results provide evidence for a contribution of impaired early-stage visual processing in emotion recognition deficits in schizophrenia and suggest that a bottom-up approach to remediation may be effective.

Introduction

Disturbances in affect have long been considered to be a hallmark of schizophrenia.¹ There has been increased interest in emotion processing in schizophrenia and its interface with social cognition.^2,3 Deficits in ability to recognize facial emotion are among the best documented in schizophrenia^4–8 and contribute significantly to impaired social competence and ward behavior and poor functional outcome.^9–13 Understanding the basis for facial emotion recognition deficits in schizophrenia, therefore, is key to designing more effective interventions and remediation strategies.

Alternative explanations can be considered to account for impairments in emotion processing in schizophrenia. First, patients may have impaired function of specific emotion-processing regions of the brain, such as the limbic system or a specific region such as amygdala. Theories regarding deficits in specific emotion-processing brain regions are supported by literature showing flat or inappropriate affect in response to emotionally evocative stimuli such as film clips, foods, and social interactions^14–20 along with more recent neuroimaging data showing altered limbic function.^21–23 In addition, some studies have shown relative specificity for facial emotion recognition over other types of face processing²⁴ (for example, see reference cited), supporting a deficit in emotion processing per se.

Other studies, however, support a more generalized visually based deficit in facial emotion recognition.^25–27 For instance, studies find similar levels of impairment in nonemotional facial processing tasks such as age and gender identification as in facial emotion recognition.^5,8,28–31 Studies also show that people with schizophrenia have similar and in some cases greater “in the moment” emotional experience compared with controls,^2,15,32–34 suggesting that internal representation of emotional concepts remains relatively intact. Deficits in facial emotion recognition, therefore, may reflect impaired ability to process physical features of the face that give rise to emotion percepts (eg, configuration of eyes and mouth) rather than specific deficits in emotional circuits. Indeed, numerous studies have shown deficits in early-stage visual processing in schizophrenia.^35–44 In order to process faces, the visual features of faces need to be correctly processed.

The human visual system contains two major subsystems—the magnocellular and parvocellular—which interact at cortical levels to produce a holistic representation of the visual environment. Over recent years, deficits in visual sensory processing have been documented in schizophrenia, particularly involving the magnocellular system.^35–44 Both magnocellular and parvocellular systems originate in the retina and project through the dorsal lateral geniculate nucleus to primary visual cortex. Magnocellular neurons are specialized for rapid conduction and project preferentially to the dorsal visual stream (the “where” pathway).^45,46 Additional projections through superior colliculus and pulvinar to the amygdala may also play a specific role in fear processing.⁴⁷ In contrast, parvocellular neurons are smaller but more numerous and project with slower conduction preferentially to the ventral visual stream (the “what” pathway).⁴⁵ The interplay between magnocellular and parvocellular processing is necessary for complex visual processes such as perceptual closure^48–50 and object recognition.^51–53

The magnocellular and parvocellular systems show differential psychophysical properties. First, neurons in the magnocellular stream are preferentially activated by large (low spatial frequency; LSF) stimuli (<∼3 cycles/degree), whereas neurons in the parvocellular stream are preferentially activated by small (high spatial frequency; HSF) stimuli (>∼4 cycles/degree). However, ranges overlap based upon factors such as stimulus duration, intensity, and contrast.^54–58 LSFs are particularly important for face emotion recognition,^47,59,60 indicating a potential preferential role for the magnocellular system.

Furthermore, magnocellular and parvocellular systems differ with respect to stimulus contrast. Magnocellular neurons function in a nonlinear mode with compressive gain control^61,62 and therefore show a sharp increase in response with increases in low levels of luminance contrast but reach a saturation-level response after luminance contrast reaches approximately 16%. Compressive gain control refers to the decrease or compression in the slope of the curve at higher contrasts compared with lower contrasts.⁶³ Parvocellular neurons do not start responding until stimuli reach approximately 10% contrast and show a linear (nonsaturating) increase in response amplitude across a wide range of luminance contrasts.^61,62,64 The shape of the contrast response function can therefore be used to distinguish contributions from the 2 pathways. Profiles that depend primarily upon magnocellular input are expected to show a nonlinear increase with saturating response to contrast similar to magnocellular neurons themselves, whereas profiles that depend upon parvocellular input are expected to show a linear response with increasing contrast.³⁵

Such analysis has been applied previously to responses elicited by simple stimuli^63,65–67 but has not been applied to more complex stimuli, such as faces. In the present study, 2 complementary techniques were used to assess relative contributions of early-stage visual processing impairments to emotion discrimination. Thus, the rationale for the study follows from findings of visual sensory deficits in schizophrenia, particularly in the magnocellular pathway,^35–44 findings that deficits are not only specific to emotion but also occur to faces^5,8,28–31 and that patients appear to have intact internal representations of emotion,^2,15,32–34 suggesting that impaired visual sensory input may contribute to emotion processing deficits in schizophrenia. First, we evaluated deficits in facial emotion processing relative to deficits in sensitivity to simple LSF and HSF visual stimuli (experiment 1). Second, we manipulated contrast levels of faces themselves and evaluated the contrast response pattern of patients and controls (experiment 2).

We tested the following predictions of the hypothesis that facial emotion identification deficits in schizophrenia reflect disturbances in early stages of sensory processing: (1) facial emotion identification will be impaired in schizophrenia across emotions in experiment 1, (2) impairments in emotion identification will correlate preferentially with impaired LSF visual performance in experiment 1, (3) emotion detection curves in both control and schizophrenia participants will show a contrast gain pattern characteristic of magnocellular neurons in experiment 2, but (4) overall performance levels will be lower in patients and will correlate with impaired early sensory processing performance in experiment 2.

Methods

Participants

Participants were 20 patients (18 male) meeting Diagnostic and Statistical Manual of Mental Disorder (Fourth Edition) (DSM-IV) criteria for schizophrenia (n = 17) or schizoaffective (n = 3) disorder and 17 healthy volunteers (12 male). Patients were recruited from inpatient (n = 14) and outpatient facilities (n = 6) associated with the Nathan Kline Institute for Psychiatric Research. Diagnoses were obtained using the Structured Clinical Interview for DSM-IV (SCID)⁶⁸ and all available clinical information. Controls were recruited through the Volunteer Recruitment Pool at the Nathan Kline Institute. Healthy volunteers with a history of SCID-defined Axis I psychiatric disorder were excluded. Patients and controls were excluded if they had any neurological or ophthalmologic disorders that might affect performance or met criteria for alcohol or substance dependence within the last 6 months or abuse within the last month. All participants provided informed consent according to the Declaration of Helsinki. This study was approved by the Nathan Kline Institute for Psychiatric Research/Rockland Psychiatric Center and Rockland County Department of Mental Health Institutional Review Boards.

Clinical and demographic information are included in table 1. The patient and control groups did not differ significantly in age (t = 0.05, df = 35, P = .9), gender (Fisher’s exact test, P = .14), or parental socioeconomic status (t₂₉ = 0.5, P = .6). All patients were receiving antipsychotic medication at the time of testing. Chlorpromazine equivalents were calculated using conversion factors described previously.^72–75 All participants had 20/32 or better corrected visual acuity on the Logarithmic Visual Acuity Chart (Precision Vision, LaSalle, IL).

Table 1.

Demographic and Clinical Characteristics of Healthy Controls and Patients With Schizophrenia

Characteristic	Controls (n = 17)	Patients (n = 20)
Age	36.5 ± 2.3	36.4 ± 2.2
Gender (M/F)	12/5	18/2
Chlorpromazine daily equivalent, mg		1195.3 ± 133.2
Antipsychotics
Atypical		17
Typical		0
Both		3
Parental socioeconomic status	43.1 ± 2.9 (n = 17)	39.9 ± 5.6 (n = 14)
BPRS total score		45.5 ± 3.0
SANS total score (including global scores)		43.5 ± 3.1
Duration of illness (years)		16.1 ± 2.0

Note: Values are mean±SEM. Numbers of subjects per group are noted when there is missing data. Socioeconomic status was measured by the 4-factor Hollingshead Scale⁶⁹. M, male; F, Female; BPRS, Brief Psychiatric Rating Scale⁷⁰; SANS, Schedule for Assessment of Negative Symptoms⁷¹.

Contrast Sensitivity

Spatial contrast sensitivity functions were obtained as previously described.⁷⁶ Horizontal sine-wave gratings were presented for 32 milliseconds at spatial frequencies of 0.5, 7, or 21 cycles/degree. Spatial frequency (SF) refers to the number of pairs or cycles of light and dark bars in 1 degree of visual angle, expressed as cycles/degree. A spatial 2-alternative forced-choice procedure was used. Gratings were presented randomly on one-half (either the right or left side) of a visual display, with the other side having a uniform field of equal space average luminance to the pattern field. The grating and uniform fields were presented simultaneously. The viewing distance was 160 cm, and the grating and uniform field together subtended 5.7° × 5.7° of visual angle. Participants stated which side of the display contained the grating and the experimenter pressed the response button. An up-and-down transformed response rule was used to determine contrast sensitivity (the reciprocal of threshold) associated with 79.4% correct responses for each SF. For each SF, contrast was changed in 3 dB steps for each correct or incorrect response until 2 errors were made. Then, the up-and-down transformed response rule began and contrast was changed in 1.5 dB steps. The mean of 8 reversals was used to estimate a threshold.

Penn Emotion Recognition Task

This computerized task assessed facial emotion recognition ability.^77,78 It included 40 color photographs of faces expressing 4 basic emotions—happiness, sadness, anger, or fear—and neutral expressions, with 8 photographs for each expression, presented in random order. Stimuli were balanced for poser’s gender, age, and ethnicity. For each emotion category, 4 high-intensity and 4 low-intensity expressions were included. Participants were instructed to identify the expressed emotion from the 5 possible choices and the experimenter pressed the response button. The task began with a practice trial in which feedback was provided. The task and scoring programs are available at https://penncnp.med.upenn.edu/.

Penn Emotion Differentiation Task

This task assessed ability to differentiate intensity of emotions shown in 2 faces side by side from the same individual.⁷⁹ Participants were asked to point to the face that was the happier of the 2 (in the case of happy) or sadder of the 2 (in the case of sad) or to point to a box indicating that the 2 faces showed equal intensity. There were 20 trials for each emotion. Outcome is percent of happy or sad faces correctly discriminated or the total percent of faces correctly discriminated.

Emotion Identification at Different Contrast Levels

Stimuli were from Ekman and Friesen⁸⁰. The task included happy, sad, and neutral emotions from 11 different individuals, for a total of 11 different stimuli per emotion and 33 total stimuli (figure 1). The original faces were used so that the hair and white rectangular background were shown. Contrast of the 33 faces was altered in Adobe Photoshop. Michelson contrast ([I_max − I_min]/[I_max + I_min]) was then calculated based on luminance of gray levels displayed on the monitor. A lookup table was used to convert gray levels to luminance (candela per meter square). I_max was the luminance of the pixel at the highest gray level, and I_min was the luminance of the pixel at the lowest gray level for each image. Only the gray levels in an oval aperture that contained the face image without the surrounding hair were considered in computing contrast. Contrasts were 1%, 2%, 4%, 8%, 24%, and 100%. Root mean square contrast is frequently reported for complex figures such as faces⁸¹ and was <1%, 1%, 2%, 3%, 8%, and 57% for each of the 6 levels.

Fig. 1.

Examples of faces used in experiment 2. Faces were presented at different contrasts, with several of the contrasts shown here.

The task began with a practice trial of posers not used in the test paradigm. For the test paradigm, each of the 33 stimuli was presented 3 times at each contrast level for a total of 99 stimuli per contrast level. Emotion type and contrast level were randomly intermixed. Stimuli were shown for 500 milliseconds followed by a screen asking the participant to choose happy, sad, or neutral as a response. The response screen remained present until the experimenter pressed the key corresponding to the participant’s verbal response.

Steady-State Visual Evoked Potentials to Magnocellular- and Parvocellular-Biased Stimuli

Steady-state visual evoked potentials (ssVEPs) were obtained as described previously.^63,76 ssVEPs were elicited by sinusoidal modulation of an array of isolated checks (16 × 16 checks, each 15 min of arc of visual angle) on a steady background. The luminance of the checks was modulated below that of the static background, producing negative contrast (dark checks). The field size was 8° × 8° and viewing distance was 114 cm.

ssVEPs were biased toward magnocellular or parvocellular systems in separate runs through the use of different contrast pedestals.^63,76 In both conditions, luminance of the checks was sinusoidally modulated around the standing contrast level (pedestal) in 7 steps of depth of modulation (DOM) (0%, 1%, 2%, 4%, 8%, 16%, and 32%). For the magnocellular condition, the pedestal equaled the DOM so that stimuli appeared and disappeared. For the parvocellular condition, the pedestal equaled 48% so that contrast never went below 16% and thus remained above the approximate contrast level where the magnocellular system saturates.

Each of the 7 steps of DOM was presented for approximately 1 s so that each run lasted approximately 7 s. Stimuli were presented at a temporal frequency of 12 Hz. Ten magnocellular-biased runs were followed by 10 parvocellular-biased runs.

ssVEPs were recorded from an occipital midline site (Oz) relative to the vertex (Cz) using gold-cup electrodes, with a ground at a parietal site (Pz). Ten runs for each participant were averaged, and signal-to-noise ratios (SNRs) were obtained for each DOM.^63,82,83,84 SNR values greater than 1 indicate a significant ssVEP response at an α level of .05.

Statistical Analysis

One patient did not receive the Emotion Differentiation (EMODIFF) test. Between group differences were assessed with mixed-design analyses of variance (ANOVAs). Relationships between measures were assessed using linear regression with variables entered step-wise. Pearson correlation coefficients (r) and F change at each step of the regression are reported for correlation and linear regression analyses, respectively. For emotion identification at different contrast levels, signal detection theory was used and the nonparametric index A′ was calculated to assess sensitivity and B″ was calculated to assess response bias and nonspecific factors such as expectancy, motivation, or fatigue.⁸⁵ A′ is a nonparametric measure and was used instead of d′ because we cannot assume a normal distribution (ie, only 1 point on the curve was used because we did not vary eg, probability that a given emotion would occur).⁸⁶ Percent correct for results at different contrast levels was also reported. Curve fitting was accomplished using Graphpad PRISM 4.0 (GraphPad Software, Inc., La Jolla, CA). Data were alternatively fit to a nonlinear model using the Michaelis-Menten equation and to a linear model, as previously described,⁷⁶ and goodness of fit measures (R²) were compared across models.

Results

Experiment 1: Correlational Assessment of Sensory/Emotion Processing Measures

For correlational analyses, 2 sets of measures were obtained for each participant. First, contrast sensitivity was obtained to stimuli biased toward magnocellular and parvocellular systems by manipulation of SF to assess basic visual perceptual ability. Second, emotion-processing ability was assessed using the Penn Emotion Recognition (ER-40) and EMODIFF tasks. The relationship between visual perceptual ability and emotion processing was assessed using linear regression to test the hypothesis that deficits in LSF (magnocellular-biased) contrast sensitivity predict impairments in emotion discrimination and differentiation.

Contrast Sensitivity

Contrast sensitivity was analyzed using a 2 group (patients and controls) × 3 Spatial Frequency (0.5, 7, and 21 cycles/degree) mixed-design ANOVA. There were significant main effects of Group (F_1,35 = 7.0, P = .01), SF (F_2,34 = 515.5, P < .001), and a Group × SF interaction (F_2,34 = 4.1, P = .03). Post hoc analyses of the Group × SF interaction showed that patients had impaired contrast sensitivity compared with controls for the magnocellular-biased 0.5 (t = 3.0, df = 35, P = .005) cycle/degree grating but not the 7 (t = 0.6, df = 35, P = .6) or 21 (t = 1.0, df = 35, P = .3) cycle/degree gratings (figure 2).

Fig. 2.

Psychophysical contrast sensitivity functions for patients with schizophrenia and healthy controls. Values are plotted as contrast sensitivity, defined as the reciprocal of contrast threshold percentage. Results at the lower spatial frequencies are indicative of performance of the magnocellular system, and results at the higher spatial frequencies are indicative of performance of the parvocellular system. **P = .005.

Emotion Processing

Patients showed significant impairment in both the ER-40 (F_1,35 = 25.4, P < .001) and the EMODIFF (F_1,34 = 41.6, P < .001) tasks across all emotions. For ER-40, there was a significant Group × Emotion interaction (F_4,32 = 6.04, P = .001), with patients showing identification deficits for all emotions except happy (figure 3A). The interaction is likely due to a ceiling effect for the happy emotion on the identification task. For EMODIFF, there was also a significant Group × Emotion interaction (F_1,34 = 13.5, P = .001), with patients showing greater deficits for happy than for sad faces (figure 3B). Across tasks, patients showed highly reliable deficits (F_1,34 = 59.7, P < .001), with somewhat greater differences in intensity differentiation than identification (Group × Task: F_1,34 = 8.67, P = .006).

Fig. 3.

Percent correct on the Penn Emotion Recognition (ER-40) (A) and Penn Emotion Differentiation (EMODIFF) (B) tasks. Relationships between contrast sensitivity for the magnocellular-biased low spatial frequency (0.5 cycles/degree) grating and performance on the ER-40 (C) or EMODIFF (D) tasks. Triangles correspond to patients and circles to controls. *P < .05; **P < .001.

Correlational Analysis

Across groups, significant correlations were observed with contrast sensitivity at 0.5 cycles/degree and ER-40 (r = .41, n = 37, P = .01; figure 3C) and EMODIFF (r = .54, n = 36, P = .001; figure 3D). No significant correlations were observed for contrast sensitivity at either 7 (ER-40: r = .09, n = 37, P = .6; EMODIFF: r = .15, n = 36, P = .4) or 21 (ER-40: r = .13, n = 37, P = .5; EMODIFF: r = .25, n = 36, P = .15) cycles/degree.

To account for differences due to group, a step-wise regression was used with cohort entered in the first step and contrast sensitivity at 0.5 cycles/degree entered in the second step. Emotion performance was the dependent variable. For EMODIFF, F change for group was significant (F change_1,34 = 41.3, P < .001). F change was also significant after contrast sensitivity was added in the second step (F change_1,33 = 4.5, P = .04), demonstrating that contrast sensitivity remains a significant predictor of EMODIFF performance even after accounting for group. In contrast, for ER-40, F change for group entered in the first step was significant (F change_1,35 = 25.4, P < .001) but F change was not significant when contrast sensitivity was entered in the second step (F change_1,34 = 1.3, P = .27).

Experiment 2: Parametric Assessment of Sensory/Emotion-Processing Function

In order to evaluate sensory responses across a range of contrasts, ssVEPs were obtained to simple isolated-check stimuli biased toward the magnocellular (figure 4A) or parvocellular (figure 4B) system through use of different standing levels of contrast (pedestals). Similarly, facial emotion recognition was assessed across a range of contrasts using modified Ekman faces (figure 5). Shapes of response curves for both sensory and emotion processing measures were assessed using nonlinear vs linear curve fits. Profiles that depend primarily upon magnocellular input are expected to show a nonlinear increase with saturating response to contrast similar to magnocellular neurons themselves, whereas profiles that depend upon parvocellular input are expected to show a linear response with increasing contrast. Relationships among processes were assessed using step-wise linear regression.

Fig. 4.

Steady-state visual evoked potential (ssVEP) signal-to-noise ratios for patients with schizophrenia and healthy controls in test conditions using depth of modulation (% luminance modulation) to emphasize the magnocellular (A) or parvocellular (B) visual pathways. Patients needed 16% contrast to obtain the same response as controls at 8% contrast (C). *P ≤ .05.

Fig. 5.

Sensitivity (A′) for detection of happy (A), sad (B), or neutral (C) faces at different contrasts. Patients needed 8% contrast to obtain the same sensitivity as controls at 4% contrast (D). *P < .01; **P < .001.

Steady-State Visual Evoked Potential

For magnocellular-biased ssVEP, significant main effects of Group (F_1,35 = 5.0, P = .03) and DOM (F_5,31 = 23.9, P < .001) were found but the Group × DOM interaction was not significant (F_5,31 = 1.1, P = .36) (figure 4A). As expected, controls and patients showed a sharp rise in response (SNR) as contrast increased from 1% to 16%, with approximate plateau at 16% contrast. The response of patients at plateau was significantly lower than that of the controls (t = 2.2, df = 35, P = .03). In addition, patients needed approximately twice as much contrast to obtain the same response as controls (figure 4C).

For the parvocellular-biased ssVEP condition, there was no significant main effect of Group (F_1,35 = 3.5, P = .07) and no significant Group × DOM interaction (F_5,31 = 0.2, P = .9) but there was a significant main effect of DOM (F_5,31 = 18.2, P < .001). As expected, curves for controls and patients showed a linear increase with contrast and did not plateau (figure 4B).

When both magnocellular- and parvocellular-biased responses were analyzed with repeated measures of condition (magnocellular vs parvocellular), and DOM (1%–32%), and between group measure of Group (patients vs controls), there was a significant main effect of Group (F_1,35 = 5.0, P = .03) but no Group × Magnocellular/Parvocellular condition interaction (F_1,35 = 1.8, P = .19).

The shape of the curves was assessed. In the magnocellular-biased condition, nonlinear Michaelis-Menten fits were superior to linear functions for both groups (R²: control, .95 vs .80; schizophrenia patients, .89 vs .85). By contrast, in the parvocellular condition, linear fits were superior to nonlinear fits (R²: control, .98 vs .82; schizophrenia patients, .98 vs .75).

Emotion Recognition

As hypothesized, emotion detection curves (figures 5A–C) in both controls and schizophrenia patients showed a pattern characteristic of magnocellular neurons—ie, sharp rise in response as contrast is increased from 1% to 8%—but plateau thereafter even if ceiling level performance was not reached, with plateau performance of patients significantly reduced vs controls (t = 4.2, df = 35, P < .001) using A′.

A 2 Group (Patients and Controls) × 3 Emotions (Happy, Sad, and Neutral) × 6 Contrasts (1%, 2%, 4%, 8%, 24%, and 100%) mixed-design ANOVA with A′ as the dependent variable showed significant main effects of Group (F_1,35 = 12.3, P = .001), Contrast (F_5,31 = 80.7, P < .001), and Emotion (F_2,34 = 123.0, P < .001). Emotion × Contrast (F_10,26 = 9.3, P < .001) and Emotion × Contrast × Group (F_10,26 = 2.5, P = .03) interactions were also significant, reflecting a different response pattern across emotions. For happy, the main effect of Group (F_1,35 = 11.1, P = .002) and Group × Contrast interactions (F_5,31 = 2.93, P = .03) were significant, reflecting greater degree of deficit at 4% and 8% than 100% contrast as determined by simple contrasts (4%: F_1,35 = 4.4, P = .04; 8%: F_1,35 = 14.4, P = .001) (figure 5A). However, statistics may have been affected by an apparent ceiling effect within controls on this task. For both sad (figure 5B) and neutral (figure 5C), the main effect of Group was significant (Sad: F_1,35 = 9.7, P= .004; Neutral: F_1,35 = 10.1, P = .003) but no significant Group × Contrast interactions were found (Sad: F_5,31 = 1.13, P = .4; Neutral: F_5,31 = 1.44, P = .2). Across emotions, no significant Group × Contrast (F_5,31 = 0.9, P = .5) or Group × Emotion (F_2,34 = 0.06, P = .9) interactions were seen.

As with magnocellular-biased ssVEP curves, patients required approximately twice as much contrast to perform as effectively as controls (figure 5D) and the response of patients at plateau was significantly lower than that of the controls. Patients plateaued at approximately the level associated with 4% contrast performance in controls (figure 5A–C) and neither group showed substantial benefit as contrast was increased above 8%.

The pattern of A′ by contrast across all emotions fit better to a nonlinear Michaelis-Menten function than to a linear function for both groups (R²: control, .92 vs .30; schizophrenia patients, .80 vs .37), corresponding to a magnocellular pattern of response.

For response bias (B″), there was no significant main effect of Group (F_1,35 = 0.3, P = .6) and no significant Group × Contrast (F_5,31 = 1.6, P = .2) or Group × Emotion × Contrast (F_10,26 = 0.4, P = .9) interactions.

Analysis with percent correct across all emotions (figure 6A) showed similar results to those using A′, with significant main effects of Group (F_1,35 = 17.0, P < .001) but no significant Group × Contrast interaction (F_5,31 = 2.1, P = .09). As with magnocellular-biased ssVEP curves, patients required approximately twice as much contrast to perform as effectively as controls (figure 6B). Similar to results with A′, the pattern of percent correct by contrast across all emotions fit better to a nonlinear Michaelis-Menten function than to a linear function for both groups (R²: control, .92 vs .38; schizophrenia patients, .92 vs .48), corresponding to a magnocellular pattern of response.

Fig. 6.

Percent correct across emotions for faces at different contrasts (A). Patients needed 8% contrast to obtain the same response as controls for 4% contrast (B). *P < .05; **P < .001.

Correlational Analyses

To examine relationships between sensory and emotional processing, a mean plateau A′ value was calculated for each subject by averaging across all emotions and the 24% and 100% contrast levels. A similar calculation was made using percent correct responses. The mean plateau for magnocellular-biased ssVEP was calculated for each subject by averaging SNRs for the 16% and 32% DOMs. The average of SNRs at the 16% and 32% DOM was also used for the parvocellular-biased ssVEP condition. Correlations between emotion processing and contrast sensitivity for each spatial frequency were also assessed using the contrast sensitivities obtained in experiment 1. Across groups, correlations were observed between emotion recognition ability and contrast sensitivity at 0.5 cycles/degree (r = .51, n = 37, P = .001) but not 7 (r = .29, n = 37, P = .09) or 21 (r = .25, n = 37, P = .13) cycles/degree. Similarly, a significant correlation was observed between emotion recognition ability and SNR of magnocellular-biased ssVEP response (r = .38, n = 37, P = .02) but not parvocellular-biased (r = .24, n = 37, P = .16) ssVEP response.

In order to exclude nonspecific correlations due to group effects, a step-wise linear regression was used with Group entered in the first step, contrast sensitivity at 0.5 cycles/degree entered in the second step, and magnocellular-biased ssVEP SNR entered in the third step, relative to emotion recognition. F change for Group was significant (F change_1,35 = 18.9, P < .001) and F change was also significant after adding contrast sensitivity (F change_1,34 = 4.7, P = .037) and magnocellular-biased ssVEP SNR (F change_1,33 = 5.9, P = .02), demonstrating that contrast sensitivity at 0.5 cycles/degree and magnocellular-biased ssVEP remained significant predictors of emotion recognition even after accounting for group.

Analyses using percent correct across all emotions showed similar results to those using A′. Across groups, significant correlations were observed between emotion recognition ability and contrast sensitivity at 0.5 cycles/degree (r = .51, n = 37, P = .001) but not 7 (r = .24, n = 37, P = .15) or 21 (r = .23, n = 37, P = .18) cycles/degree. Similarly, a significant correlation was observed between emotion recognition ability and SNR of magnocellular-biased (r = .41, n = 37, P = .01) but not parvocellular-biased (r = .28, n = 37, P = .09) ssVEP response. In step-wise regression, F change was significant after addition of Group (F change_1,35 = 21.6, P < .001), after adding contrast sensitivity at 0.5 cycles/degree in the second step (F change_1,34 = 4.4, P = .04), and magnocellular-biased SNR in the third step (F change_1,33 = 6.9, P = .01).

Medication

No significant relationship was observed between chlorpromazine (CPZ) equivalents and emotion processing measures (ER-40, EMODIFF, percent correct, or A′ scores for all emotions combined at each contrast or for each individual emotion across contrast levels), contrast sensitivity, or ssVEP (all P > .1).

Discussion

Face emotion processing is a complex sensory task requiring extraction of configural information from a distributed network of features (eg, eyes, mouth, cheeks, corner of nose) in order to reach a categorical decision regarding another person’s internal state. Over recent years, deficits in early sensory processing have become increasingly well documented,^{37,39,44,76,87–90} particularly involving the magnocellular visual system. The present study demonstrates specific contributions of impaired early sensory processing to face emotion recognition deficits in schizophrenia.

Two experiments were performed. In experiment 1, subjects performed 2 face emotion processing tasks, the ER-40 which includes 4 emotions plus neutral faces and EMODIFF in which participants distinguish between levels of emotional expression for happy and sad faces,^24,79,91 along with contrast sensitivity to low-, medium-, and high-SF stimuli. In the ER-40, patients showed deficits across all emotions except for happy, where ceiling level performance was observed for both groups. In the EMODIFF, deficits were observed in discrimination of both happy and sad emotions, with greater deficits in discriminating intensities of happy. Thus, while patients are not as impaired in identifying happy faces as for other emotions, they have greater deficits in differentiating intensities of happy.

Consistent with several prior reports, patients also showed impaired contrast sensitivity to LSF, but not HSF, stimuli,^76,89 although other studies have also found HSF deficits.^92,93 The current study supports the concept of a magnocellular deficit with less of a deficit in parvocellular function in schizophrenia. Furthermore, deficits in magnocellular-biased LSF contrast sensitivity correlated significantly with performance on the ER-40 and EMODIFF tasks. Even when nonspecific correlations due to group effects were considered, contrast sensitivity at 0.5 cycles/degree remained a significant predictor of emotion processing on the EMODIFF task.

Similar findings were obtained by Norton et al,²⁶ who found that contrast sensitivity at an SF of 2 cycles/degree was related to fear recognition. The present study extends these findings across a range of emotions and demonstrates as well relative specificity for LSF vs HSF gratings, suggesting preferential magnocellular involvement.

Experiment 2 examined the pattern of emotional performance across contrast levels in schizophrenia. Although SF can be used to bias processing toward the magnocellular vs parvocellular pathways, considerable overlap may also be observed across pathways.⁵⁵ A potentially more effective method for distinguishing visual systems is through manipulation of contrast.

In general, magnocellular neurons are sensitive to much lower levels of contrast than parvocellular neurons, which do not begin to respond until contrast levels reach approximately 10%.⁶⁴ In addition, responses of magnocellular neurons saturate above approximately 16% contrast, whereas parvocellular neurons show continued increase in response as contrasts increase from approximately 16% to 100%. In ssVEP studies, therefore, the 2 systems can be studied independently by examining contrast response functions in which stimuli appear and disappear, which yields a typical nonlinear magnocellular response curve (figure 4A), or by modulating contrasts around a high standing level of contrast, which yields a more linear nonsaturating curve typical of parvocellular neurons (figure 4B).^55,61,63

As in prior ssVEP studies, patients showed significant reduction in plateau level of the magnocellular, with relatively preserved parvocellular, response.^76,84 The nonlinear response pattern in the magnocellular-biased condition reflects both amplification of the responses at low contrast by nonlinear circuit elements, such as NMDA-type glutamate receptors, and divisive gain at higher contrast (ie, neurons are inhibited due to local GABAergic feedback within primary visual regions).³⁵ As with other neurophysiological deficits in schizophrenia, impaired ssVEP generation, therefore, may reflect either decreased excitatory drive through NMDA receptors or altered GABAergic feedback.^87,94 Preferential magnocellular dysfunction in schizophrenia may reflect impaired nonlinear gain, with parvocellular dysfunction seen when the parvocellular system is driven into a more nonlinear gain process.⁷⁶

The present study is the first to evaluate patterns of emotion-processing deficits across a range of contrasts. Both groups showed high sensitivity even at relatively low contrasts with saturation in performance above 8% contrast, suggesting that, at least for emotions tested here, the information required for emotion discrimination can be carried primarily by the magnocellular system, with little additional benefit obtained from the more highly detailed information provided by parvocellular input. In addition, impairments in face processing correlated significantly with impaired generation of magnocellular-based, but not parvocellular-based, ssVEP responses, and with reduced contrast sensitivity to LSF, but not HSF, stimuli, supporting preferential magnocellular involvement in face emotion processing.

In the present study, deficits in emotion recognition were also observed across a range of emotions, supporting generalized sensory deficits rather than deficits in specific emotion circuits. For example, with amygdala damage, such as with the well-described subject SM,⁹⁵ patients have selective deficits in detection of fear based upon impaired scanning of eyes. The absence of a selective deficit in the present study argues against preferential amygdala involvement, although contributions of impaired amygdala function to the overall pattern of deficit cannot be excluded.

Conscious awareness of stimuli is mediated primarily based upon activity within ventral stream structures, such as lateral occipital complex,⁹⁶ which receive predominant input from the parvocellular system, whereas activity in dorsal stream does not typically reach consciousness. Consequently, schizophrenia patients would not necessarily be consciously aware of magnocellular dysfunction nor would they show impairments on routine assessments of visual function such as eye charts, which test primarily ventral stream object recognition-type functions. Nevertheless, the magnocellular system provides a crucial organizing input that permits the brain to make rapid assessments of potential object identity⁵³ and to control eye movements to efficiently explore the visual space.^97,98 Deficits in magnocellular function thus contribute to impairments in a range of complex visual tasks such as fragmented object recognition,⁴⁸ motion detection,⁹⁹ and reading.¹⁰⁰ Seen from this perspective, face emotion recognition is merely a complex visual task in which disparate information must be extracted from a complex visual stimulus using an efficient search approach.¹⁰¹

In this study, all patients were on antipsychotic medication raising the issue of a potential medication effect. However, emotion processing as well as visual processing deficits are also found in unmedicated first-degree family members^77,102 and in the present study no significant correlations were found with CPZ equivalents. The majority of the patients tested were inpatients. Future studies will address effects of sensory processing deficits on emotion dysfunction in outpatients as well as at different stages of illness.

Face emotion recognition, along with auditory emotion recognition, are hallmarks of the construct of social cognition. We have over the past several years documented significant contributions of impaired early auditory processing, such as inability to detect pitch modulations, to voice emotion recognition.^103,104 The present study thus suggests that the severe impairments in social cognition in schizophrenia may result from a “perfect storm” in which signals needed for emotion detection are deficient in both the auditory and the visual modalities, leaving subjects diminished opportunity to compensate for the combined deficit.

Determining the neural basis for emotion-processing deficits helps clarify their mechanism and contribution to impaired outcome in schizophrenia. Demonstrating the relationship with sensory processing deficits is critical for several reasons. First, it underscores that patients are not inherently nonemotional. As shown in various evocative studies, patients experience similar amounts of pleasant and unpleasant emotions as controls.^{2,14,15,17,32–34} Second, even if sensory deficits are generally not treated at present, understanding their contribution to impaired social cognition may assist patients, families, and caregivers in defining compensatory strategies. Perhaps most importantly, present results highlight the need for remediation of basic sensory deficits in schizophrenia (see Adcock et al¹⁰⁵, this volume) and suggest that such approaches may be critical for restoration of social function.

Funding

National Institutes of Health (RO1 MH66374 and RO1 MH084848 to P.D.B., RO1 MH060722 and RO1 MH084856 to R.C.G., and R37 MH49334 to D.C.J.); Lieber Young Investigator Award from the National Alliance for Research on Schizophrenia and Depression to P.D.B.