The dysfunction of processing emotional faces in schizophrenia revealed by expression-related visual mismatch negativity

Abstract

To investigate the emotional face processing in patients with schizophrenia, the preattentive automatic processing of emotional faces in individuals with schizophrenia was compared with that of age-matched healthy control group as indexed by the expressional mismatch negativity (EMMN) elicited by facial expressions. Compared with neutral faces as standard stimuli, deviant emotional faces elicited posterior EMMN between 150 and 500 ms after stimuli onset, with larger amplitudes for sad than happy deviant faces. Both early and late EMMNs significantly decreased in the schizophrenia group, regardless of sad or happy EMMN, in comparison with the healthy control group. These data suggest the dysfunction of automatic processing of expressional information in patients with schizophrenia.

Introduction

Schizophrenia is associated with various clinical symptoms such as auditory hallucinations, paranoid delusional thoughts, disorganized thinking, and disturbances of self [Diagnostic and Statistical Manual of Mental Disorders, 4th ed. (DSM-IV)]. Interestingly, it has been shown that patients with schizophrenia often exhibit impairments in facial emotion recognition, which contributes to their poor social functioning 1–5.

Facial expressions are fundamental emotional stimuli as they convey important information such as a person’s mental state and disposition to social interaction. Therefore, considerable efforts have been made to investigate the processing of facial expressions changes in patients with schizophrenia. Previous studies focusing on recognition and memory of facial expressions revealed that impairments in facial emotion recognition were stable across different stages of the disorder in schizophrenia. One recent electrophysiological report found that the recognition deficit in patients with schizophrenia was accompanied by the absence of the mid-frontal FN400 and parietal old/new event-related potential (ERP) effects in the mismatched-expression condition 6. Importantly, it should be noted that processing of emotional faces is fast, nonconscious, mandatory, and capacity free 7. Although the patients with schizophrenia frequently showed the abnormal processing of emotional expressions, the automatic change detection of facial expressions affected by schizophrenia is not clear, which would be investigated in the present study by recording and analyzing the visual mismatch negativity elicited by facial expression, that is, the expression-related mismatch negativity (EMMN).

It has been widely accepted that mismatch negativity (MMN), the difference between the ERPs elicited by deviant and standard stimuli, reflects automatic change detection processing at the preattentive stage 8. Although the MMN has been well defined in the auditory modality, convincing evidence has been provided for visual MMN (vMMN) such as colors, motion, and spatial frequency 9–11. In addition to vMMN being relevant to low-level simple visual features, the possibility that facial expressions might elicit vMMN has been confirmed 12–16. For example, using a modified cross-modal delayed response paradigm, Zhao and Li 12 first found a right-posterior expression-related vMMN (called expression MMN, EMMN), which is larger for sad than that for happy expressions, with the generator mainly related to the occipital, temporal and frontal brain regions 13–16. Furthermore, there was evidence that temporal and occipital visual areas as well as frontal generators automatically represent regularities in unattended emotional faces and store them as predictive memory representations 15,16. To date, it has been accepted that the EMMN reflects the violation of abstract sequential regularities of facial expressions in a predictive memory representation 12–17.

There are few studies investigating visual MMN in patients with schizophrenia 18. For example, it has been found that deviant motion direction elicited a reduction of vMMN signals in individuals with schizophrenia, indicating the impairment of early processing of visual information 19. Particularly relevant to the present study, there was evidence that compared with healthy people, neither happy nor fear faces elicited any mismatch responses in individuals with schizophrenia, indicating insufficient automatic processing of facial expressions 1.

To further explore the emotion-related visual mismatch responses in schizophrenia, in the present study, we recruited patients with only paranoid and undifferentiated schizophrenia. In addition, it has been shown that schematic faces are useful and reliable for studying brain responses to emotional faces owing to their simplicity 20,21. In addition, there was evidence that the task-irrelevant schematic facial expressions can elicit EMMN 22,23. Therefore, in the current study, schematic emotional faces were used to minimize the variance associated with genuine facial photographs. If there is dysfunction in processing facial expressions under nonattentional condition, the decreased EMMN should be expected in individuals with schizophrenia in comparison with healthy participants.

Patients and methods

Participants

Twenty-five patients with schizophrenia (13 females; mean age: 31.3±12.2 years) and 25 age-matched healthy control participants (13 females; mean age: 30.9±13.2 years) participated in this study. Each patient was diagnosed with schizophrenia according to Diagnostic and Statistical Manual of Mental Disorders, 4th ed. (DSM-IV). None of the included patients had a history of severe medical disorder or severe neurological disorder. A trained psychiatrist or psychologist evaluated psychiatric symptoms on the Positive and Negative Syndrome Scale 24. It was found that patients showed higher score than did controls (positive syndromes: 14.1 and 7.0 for patients and controls, respectively, P<0.001; negative syndromes: 12.4 and 7.6 for patients and controls, respectively, P<0.001; and general syndromes: 28.8 and 16.8 for patients and controls, respectively, P<0.001). We also used the Personal and Social Performance scale to assess people’ social functioning and found lower score for patients (61.1) than controls (90.2; P<0.001) 25.

The healthy volunteers had no history of any major psychiatric disorders or major physical illnesses and were not taking any medication that affects the central nervous system. This study was approved by the Institutional Review Board of Taiyuan Brain Hospital. All participants received payments for their participation and gave their informed consents before the experiment.

Stimuli and procedure

To avoid low-level processing of facial features, 54 schematic faces with happy, sad, and neutral expressions were presented (Fig. 1). Each type of expressions included 18 different schematic facial models modulated by changing the distance among the facial features as well as the shape of the facial features 23. The stimuli were presented for 100 ms on two sides of the fixation, with an interstimulus interval of 500 ms and a visual angle of 3.68°×3.42°.

Fig. 1.

Schematic illustration of emotional stimuli used in the present experiment.

Figure 2 showed schematic illustrations of the sequence, with the neutral faces as standard stimuli and happy and sad faces as deviant stimuli. To establish sensory memory pattern, 10 standard stimuli (neutral faces) were presented at the head of the stimulus sequence, and there were no less than two standards between consecutive deviants. The target stimuli, that is, the fixation crosses with changes in size, always were presented without facial stimuli to prevent contamination from motor-response-related artifacts and the active task. Four sequences were conducted with 250 trials for each (standard: 200 neutral faces; deviant: 25 happy and 25 sad faces). The participants were asked to detect the unpredictable change in size of the fixation cross (‘+’) by pressing the button of ‘/’ in the key board as quickly and correctly as possible, ignoring the face stimuli.

Fig. 2.

The corresponding event-related potential (ERP) waveforms in the two groups. EMMN, expressional mismatch negativity; MMN, mismatch negativity.

Electroencephalogram recording and analysis

Electroencephalogram (EEG) signals were continuously recorded with NeuroLab digital amplifier system, using NeuCap with 64-channel Ag/AgCl electrodes, according to the extended international 10-20 system (Yiran Sunny Technology Co. Ltd, Beijing, China, http://www.yiransunny.com.cn). The reference electrode was placed on the nose tip. Vertical and horizontal electrooculography (EOG) signals were recorded with two electrodes placed above and below the right eye and with two electrodes at the right and left outer canthi of the eyes, respectively. The impedances of the electrodes were kept below 5 kΩ. EEG and EOG signals were amplified with a band pass of 0.1–100 Hz at a sampling rate of 500 Hz.

EEGLab software (https://sccn.ucsd.edu/eeglab/index.php) was used to analyze EEG data. After EOG artifact correction 26, the EEG was segmented into the epoch from 100 ms prestimulus to 500 ms poststimulus. The trials contaminated with artifacts greater than ±100 μV as well as with targets when participants’ responses occurred were rejected before averaging. The EEG segments were averaged separately for standard (553.6 trials and 563.3 trials for control and patient groups, respectively; P>0.1) and deviant stimuli (happy: 73.5 and 75.6 trials for control and patient groups, respectively; P>0.1; sad: 71.2 and 72.9 trials for control and patient groups, respectively; P>0.1). EMMN was obtained by subtracting ERPs to standard stimuli from ERPs to either happy or sad face stimuli, respectively.

According to the grand averaged EMMNs elicited by sad and happy faces, correspondingly (Fig. 3), the mean amplitudes of EMMN were measured for early EMMN (150–250 ms time windows after stimulus onset) and late EMMN (250–350 and 350–500 ms time windows after stimulus onset, respectively), correspondingly. We conducted one-tailed t-tests to determine whether the EMMN mean amplitudes were significantly different from zero.

Fig. 3.

The topographic distributions of the expressional mismatch negativity (EMMN) mean amplitudes (150–250 and 300–450 ms for the early and late EMMNs, respectively), correspondingly.

Four-way analysis of variance with repeated measures was conducted with deviant type (happy and sad), hemisphere (left and right), and site (P7/8, PO7/8, O1/2, and M1/2) as within-subject factors and group (patients and normal participants) as the between-subject factor. Greenhouse–Geisser corrections were made when appropriate.

Results

Responses to detect cross changes were scored as hit if the correct button was pressed within 150–1000 ms after targets onset. Response time and the mean accuracy rate were 375±75 ms and 96.2±4.2%, respectively, for patients and 368±80 ms and 96.6±3.5%, respectively, for controls, and there were no significant group differences (Fs<1).

Across groups, as shown in Fig. 2, all face stimuli elicited P1, N170, and P2 components at the temporal-occipital electrode sites. In healthy normal controls, compared with standard neutral faces, deviant facial expressions elicited a more negative shift during the N170 and P2 time range. In healthy controls, the difference waveforms (EMMN and expression-related MMN) between ERPs in response to deviants and that to standard stimuli actually comprised two subcomponents: early EMMN between 150 and 250 ms and late EMMN between 250 and 500 ms. However, the EMMN was not evident in the patient group.

The mean amplitudes of EMMN component were analyzed by analyses of variance mentioned in the method session. For early EMMN, although facial expressions did not modulate its amplitude (F<1), there was a significant main effect of group [F(1, 48)=5.46, P=0.024; partial η²=0.102], indicating larger early EMMN in the normal controls (−0.70 μV) than that in the patient group (−0.01μV). We did not find other significant main effects and interactions (Ps>0.1).

Similar to the analysis of the early EMMN, overall, the late EMMN within the time window of 250–350 ms was larger for normal controls (−0.38 μV) than that for patients [0.36 μV; F(1, 48)=4.57, P=0.038; partial η²=0.187]. This effect interacted with deviant type [F(1, 48)=5.30, P=0.028; partial η²=0.227], reflecting that the group difference was evident only in the sad expression condition (−0.62 and 0.45 μV for patients and normal participants, respectively; P=0.015) and not in the happy expression condition (−0.13 and 0.24 μV for patients and normal participants, respectively; P=0.298) and that the deviant type effect, that is, larger EMMN for sad than happy faces, was found in normal controls (P=0.042) but not in patients (P=0.392). No other effects reached the significant level (Ps>0.1).

For the late EMMN within the time window of 350–500 ms, we only found a significant main effect of group [−0.49 and 0.40 μV for normal controls and patients, respectively; F(1, 48)=6.55, P=0.014, partial η²=0.120]. No other main effects and interactions reached the significant level (Ps>0.1).

To further confirm the presence of EMMN, the comparison between EMMN amplitude and zero was conducted for each group. For the early EMMN, in the healthy group, the mean amplitude of EMMN was significantly different from zero for each channel (Ps<0.02), regardless of happy or sad condition. However, in the patient group, the comparison was not significant (Ps>0.1), indicating the absence of early EMMN, regardless of happy or sad condition. For the late EMMN, similar to the analysis of the early EMMN, the mean amplitudes of late EMMN were significantly different from zero (Ps<0.05), regardless of happy or sad condition in the healthy group. In patients, interestingly, the mean amplitudes of difference waveforms between ERPs elicited by sad faces and neutral faces were positively larger than zero (P=0.041), that is, the mismatch positivity response, and the mean amplitudes of late EMMN for happy faces were not significantly different from zero (P=0.44), indicating the absence of happy EMMN.

Discussion

In the present study, we compared visual mismatch responses elicited by unattended facial expressions between healthy controls and patients with schizophrenia. We found that although the responses for detecting cross changes did not differ between the two groups of participants, the EMMN was not evident in the patient group, regardless of sad or happy EMMN.

In line with the present findings, previous studies have shown that the EMMN between ERPs in response to deviants and that to standard stimuli comprised two subcomponents, that is, early EMMN and late EMMN 1,12,23, and reflected preattentive change detection of facial emotion, with larger amplitudes for sad than happy EMMN 15,23,27,28. Importantly, the EMMNs significantly decreased in patients with schizophrenia compared with healthy controls, indicating the dysfunction of processing task-irrelevant facial expressions, regardless of sad or happy expression. These data are in agreement with the recent findings that the motion-direction-related visual MMN decreased in patients with schizophrenia 19. Particularly relevant to the present study, Csukly et al. 1 also found that mismatch responses to both fear and happy emotional faces were significantly attenuated in patients compared with healthy controls. Although the conclusion is similar, there are some methodological differences between the study by Csukly and colleagues and the present study. Different from the study by Csukly and colleagues, in the present study, we recruited only paranoid and undifferentiated schizophrenia and used schematic emotional faces as test stimuli to eliminate the influence of irrelevant information from faces such as the low-level features of faces and the possible interaction between participants and sexes of faces. The findings further indicated that the schematic faces may be useful for clinical study and application owing to the simplicity versus human faces 21.

It should be noted that we did not test the emotional recognition in the present study. Csukly et al. 1 proposed that emotion recognition deficits might mediate the association between automatic preattentive information processing deficits and impairments of everyday life functioning in schizophrenia. However, using the same emotional stimuli with the present study, one of our ongoing studies investigated the positive classification advantage (i.e. principal component analysis, categorizing the positive facial expression more quickly than the negative facial expression) in individuals with schizophrenia and found a similar response pattern of principal component analysis between patients and controls, though the overall performance decreased in patients. Although the present schematic emotional face stimuli have been used in previous studies and similar EMMN results were found with real faces 23, it is necessary to use real faces to further investigate this issue.

Conclusion

To investigate whether there is a dysfunction of automatically processing of nonattentional emotional faces in patients with schizophrenia, the EMMN elicited by emotional faces was recorded. The EMMN decreased significantly in the patient group, regardless of sad or happy EMMN, indicating a dysfunction of processing nonattentional emotional faces in patients with schizophrenia.