Visual Attention to Emotional Face in Schizophrenia: An Eye Tracking Study

Abstract

Objective:

Deficits in the processing of facial emotions have been reported extensively in patients with schizophrenia. To explore whether restricted attention is the cause of impaired emotion processing in these patients, we examined visual attention through tracking eye movements in response to emotional and neutral face stimuli in a group of patients with schizophrenia and healthy individuals. We also examined the correlation between visual attention allocation and symptoms severity in our patient group.

Method:

Thirty adult patients with schizophrenia and 30 matched healthy controls participated in this study. Visual attention data were recorded while participants passively viewed emotional-neutral face pairs for 500 ms. The relationship between the visual attention and symptoms severity were assessed by the Positive and Negative Syndrome Scale (PANSS) in the schizophrenia group. Repeated Measures ANOVAs were used to compare the groups.

Results:

Comparing the number of fixations made during face-pairs presentation, we found that patients with schizophrenia made fewer fixations on faces, regardless of the expression of the face. Analysis of the number of fixations on negative-neutral pairs also revealed that the patients made fewer fixations on both neutral and negative faces. Analysis of number of fixations on positive-neutral pairs only showed more fixations on positive relative to neutral expressions in both groups. We found no correlations between visual attention pattern to faces and symptom severity in schizophrenic patients.

Conclusion:

The results of this study suggest that the facial recognition deficit in schizophrenia is related to decreased attention to face stimuli. Finding of no difference in visual attention for positive-neutral face pairs between the groups is in line with studies that have shown increased ability to positive emotional perception in these patients.

Patients with schizophrenia have disturbed social functioning that remains constant over time and across the acute and partially remitted phases of the illness (1). Recognizing the social-emotional cues like face and its expressions is essential for social communication (2).

A general deficit in the processing of faces has been shown in schizophrenic patients, which is associated with worsening of their symptoms and reduced social functioning (3). Some studies have also found that people with schizophrenia show reduced facial expressiveness in response to emotional stimuli (4). An initial question is whether this deficit pertains to facial emotion recognition or to a more generalized facial information processing deficit. Some studies have shown that these patients have impairments on numerous and varied tasks involving the analysis of faces, including recognizing familiar and unfamiliar faces and tasks concerning with identity matching (3, 5). Individuals with schizophrenia also have difficulties in emotion recognition tasks even in positive emotions such as happiness (6). Results of some studies support deficits in recognizing negative emotions in these patients (7), while others emphasize the impaired recognition of specific emotional faces such as fearful (8), disgusted (9, 8), angry and/or sad (10). Several researches have shown that face recognition and facial emotion processing are independent and have parallel processes (11). This view comes from both neuropsychological studies indicating dissociated face recognition and facial emotion processing abilities (12), and human fMRI studies that have shown different parts of the brain process of these two kinds of information (13).

It is unclear whether deficit in recognition and identification of facial emotional expressions in schizophrenia is a result of abnormal emotion processing or an impairment in processing complex visual stimuli such as face (14). In a recent ERP study, it has been suggested that the impaired facial expression perception in schizophrenia is related to delayed responses in face perception (15). Another potential cause of this deficit might be restricted attention in these patients. Attention has a critical role for almost all cognitive functions and recognizing face feature. It seems that reduced attention to face impairs the face processing ability (16). Patients with schizophrenia typically use a “restricted” strategy in their visual attention allocation, by making fewer and longer fixations (17). This restricted strategy is more obvious when looking at faces compared to other complex stimuli such as geometric figures, indicating the face-specific nature of this disturbance (18). Given the increased processing load associated with emotion processing, this abnormality would even become more obvious when looking at facial emotions (19). Commonly used behavioral tasks have not been able to answer the fundamental questions about the nature and scope of these impairments (5). Assessing visual attention would provide a direct way to test the hypothesis that impaired emotion recognition pertains to basic perceptual processing in schizophrenia. We tracked eye-movements of patients with schizophrenia while viewing facial pictures with different emotional and neutral contents passively. Initial orientation of visual attention of these patients for neutral, negative and positive facial expressions was assessed to examine the possible mechanism that may underlie face and emotion perception deficits in these patients. Furthermore, we also explored any potential correlation between symptom severity and pattern of visual attention for different facial expressions in our patient group. The aim of this study was to extend previous findings of facial processing deficits of schizophrenia in relation to visual attention using eye tracking technique, avoiding language or memory demands which may require an additional cognitive load. We hypothesized that individuals with schizophrenia will show impairment on visual attention to emotional faces compared to healthy individuals.

Material and Methods

Participants

The study sample consisted of thirty patients (15 males and 15 females) with schizophrenia-spectrum recruited from residential psychiatric centers in Karaj during their stable chronic phase of schizophrenia and thirty healthy normal volunteers (15 males and 15 females) recruited from Azad University, Karaj branch. Across groups, the age varied between 23–57years. According to DSM-IV-TR (Statistical Manual of Mental Disorders, Fourth edition) criteria, a psychiatrist diagnosed schizophrenia in the patient group. The diagnosis was then confirmed by a trained physician using the Persian version of SADS interview (Schedule for Affective Disorders and Schizophrenia) in which the inter-rater reliability has been reported to be 0.91 for psychotic disorders (20, 21). There were 10 paranoid and 20 undifferentiated schizophrenia cases in the patient group.

All participants were initially screened for exclusion criteria. According to self-reports, case-file reviews and medical examination which were done by a physician, none of the participants had a history of current or past neurological illnesses. Participants with IQ less than 70, left-handedness (based on checking the hand used while writing), and abnormal or not corrected visual acuity were excluded from the study. All participants were matched by age, sex, and educational level. SADS interview for healthy participants (done by the same trained physician) confirmed that they were free from any Axis I or Axis II disorders and history of schizophrenia. Additionally, healthy participants had no personal or family history of schizophrenia. Via SADS interview, patients were also confirmed not to have other psychiatric comorbidities based on DSMIV-TR criteria. All patients were receiving second-generation antipsychotics which the most common medications was Clozapine and Rispridone at the time of the evaluation. All participants were speaking Persian as their first language. Participants had ≥ 9 years of formal education.

Stimuli

Three female and three male Caucasian faces expressing full-blown negative (anger, sadness), positive (happy, surprised) and neutral expressions were collected from Cohn Kanade AU-coded Facial Expressions Database. The database has more than 2000 image sequences of subjects performing 23 facial displays coded according to FACS (facial action coding system) describing expressions in terms of action unit (AU) (22). The photos were in black and white and resized to 397×425 pixels (14×5.5 cm), using the Microsoft Paint while the contrast, resolution, or illumination remained constant.

We used the Experiment Builder 1.6.121 (SR Research Ltd, Mississauga, Ontario, Canada) to design the task. We paired each emotion with the same person’s neutral expression (emotional-neutral pairs), neutral expression with the neutral expression (neutral-neutral pairs), and emotion with other emotions (emotional-emotional pairs). The last section (emotional-emotional pairs) is not reported in this paper.

Each trial consisted of two adjacent pictures in two opposing sides (right and left). The innermost edges of images were 3cm in distance. The location of face pairs was counterbalanced during the task.

Two blocks which the second was the repetition of first were used. A total number of 96 emotional-neutral. (6persons×4emotions×2counterbalance×2blocks) and 12 neutral-neutral (6persons×1neutral expression×2blocks) pairs was compiled. Since these data are part of a more comprehensive study, we have presented only 108 of the total 252 trials (126 in each block) in this paper.

Instruments

The experimental setup was identical to that used in our previous study. For more details of the system setup, see (23). In short, the stimuli were presented on a 19” AOC monitor connected to a 2.60 GHz Pentium Dual core CPU computer (Subject PC). Movements of the participants’ right eye were recorded via Eye Link II eye tracker (SR Research Ltd, Mississauga, Ontario, Canada) with the sampling rate of 250Hz (recording every 4 ms), connected to a synchronized computer (Operator PC) to calculate and save eye movements data.

Procedure

All participants filled in a written contest form. All assessments were done in the Neuropsychology Lab of Institute for Cognitive Science Studies (ICSS) in same conditions.

The Persian version of Positive and Negative Syndrome Scale (PANSS) was applied to the patient group. This questionare has been used in Iranian research extensively (24, 25) with inter-rater reliability of 0.71 (26)

After completing phenomenological (SADS and PANSS) assessments, participants were seated in a quiet dark room in front of a monitor with a 55 cm distance, placing their chins in a chin rest to avoid unwanted head movements. After a 9-point calibration, the participants were instructed both orally and by a written instruction on the monitor to fixate on a cross at the beginning of the task and then look at the faces in whatever style they chose. The task then began with a white fixation appearing at the center of a black screen (location: 511×383) which was constant during the task even when the face pairs were presented to ensure that all participants started their gaze from the same location. A drift correction was done before every trial. After a short break of about 10 minutes, recalibration was done and the second block was being presented in the same manner. It took about 30 minutes to complete the whole task. Each face pair was presented for 3000 milliseconds (ms) on a black background. The current paper contains the data of the first 500 ms of the trials only.

Data Analysis

After collecting data from Data viewer 1.8.1 software, we defined a fixation as an eye position remaining within a 50 pixel area for more than 100 ms (27). Fixations occurring before 100ms of presentation of face-pairs (28) or lasting more than 3050 ms of presentation were omitted. Moreover, after defining a rectangular interest area (IA) surrounding the two facial stimuli (397×425 pixels each), any fixation out of the IAs was deleted. Any sampling rate less than 75% (watching less than 189/252 trials according to the exclusion criteria for fixations) was also an exclusion criterion.

Three separate 2 × (2), repeated measure ANOVAs were applied to compare the number of fixations on emotional-neutral, negative-neutral and positive neutral stimuli between the two groups during the first 500 ms of stimulus presentation.

Linear regressions were also applied to examine any correlation between the above mentioned variables and the PANSS results in both groups.

The P-value less than 0.05 were considered as significant for all statistical tests.

Analysis of number of fixations on the neutral-neutral face pairs showed no preference for right or left positions in any group (all t (29) ≤ |1.17|, ns), ruling out any orientation dominancy or spatial hemi neglect.

We hypothesized that patients with schizophrenia would show fewer fixations in the first 500 ms on emotional faces in general and negative faces in particular.

Results

Demographic Data

Independent samples t-tests showed no differences between the groups in terms of age and level of education (Table 1)

Table 1:

Group Comparison Based on Age and Education Level using Independent Samples t-tests

Variable	Control (n = 30) M (SD)	Schizophrenia (n = 30) M (SD)	t	df	Sig (2-tailed)
Age	37.8 (8.58)	39.23 (10.65)	−0.57	58	0.56
Education level	13.67 (2.2)	12.64 (2.37)	1.7	56	0.09

Eye Tracking Results

Comparing the number of fixations made during the first 500 ms of emotional-neutral pairs, we found that people with schizophrenia made less fixations on faces, regardless of their emotions. Moreover, the main effect of expression was significant, pointing to more fixations being made on emotional faces compared to neutral faces in both groups. There were no significant differences between the two groups in terms of facial expression (no significant expression × group interaction) (Table 2).

Table 2:

Group Comparison Based On the Number of Fixations on Emotional-Neutral Face Pairs during the First 500 ms of Stimuli Presentation

Variable	Group	Normal (n=30) M (SD)		Schizophrenia (n=30) M (SD)		F(1,58)	p-value
	Main Effect	Emotional	Neutral	Emotional	Neutral
Number of Fixations in first 500 ms	Expression Group Expression × Group	46.06 (12.005)	42.23 (10.002)	39.66 (11.64)	37.46 (9.42)	7.002	0.01
						4.78	0.03
						0.51	0.47

^*Significant Expression × group Interaction

Analysis of number of fixations on negative-neutral pairs only showed a significant group main effect, indicating that the patients made fewer fixations on both neutral and negative faces (Table 3)

Table 3:

Group Comparison Based On the Number of Fixations on Negative-Neutral Face Pairs during the First 500 ms of Stimuli Presentation

Variable	Group	Normal (n = 30) M (SD)		Schizophrenia (n = 30) M (SD)		F(1,58)	p-value
	Main Effect	Negative	Neutral	Negative	Neutral
Number of Fixations in first 500 ms	Expression Group Expression × Group	21.77 (6.45)	22.93 (5.83)	19.03 (6.84)	18.8 (5.37)	0.32	0.57
						6.39	0.01
						0.72	0.39

Number of fixations on positive-neutral faces was not significantly different between the two groups as well. The main effect of expression was seen significantly, indicating that both groups made more fixations on positive relative to neutral expressions (Table 4).

Table 4:

Group Comparison Based On the Number of Fixations on Positive-Neutral Face Pairs during the First 500 ms of Stimuli Presentation

Variable	Group	Normal (n=30) M (SD)		Schizophrenia (n=30) M (SD)		F(1,58)	p-value
	Main Effect	Positive	Neutral	Positive	Neutral
Number of Fixations in first 500 ms	Expression Group Expression × Group	24.3 (7.01)	19.3 (5.73)	20.63 (5.67)	18.67 (5.36)	14.78	<0.001
						2.95	0.09
						2.8	0.1

Linear regressions did not show any significant correlation between the positive or negative PANSS scores and the number of fixations during 500 ms in group with schizophrenia.

Discussion

Patients with schizophrenia seems to have impairments in recognizing socially relevant stimuli such as faces that leads to disturbed social functioning (29).

In this study, we examined visual attention through tracking eye movements in response to emotional and neutral face stimuli in patients with schizophrenia and healthy controls. The hypothesis was that schizophrenia is associated with restricted attention to emotional faces as an underlying cause of social isolation in these patients. Moreover, we examined the correlation between visual attention allocation and symptoms severity in our patient group.

Our analyses revealed that patients with schizophrenia made less fixations on both emotional and neutral faces compared to healthy participants during the first 500 ms of stimuli presentation. This finding supports previous studies which indicate restricted attention to visual stimuli and especially human faces in patients with schizophrenia. Li H et al. reported that individuals with schizophrenia fixate on facial features significantly less than controls (30). Vuilleumier et al. reported that reduced number of fixations in schizophrenia was affected by an ongoing difficulty in visual attention disengagement and suggested that impairment in allocation of attention might be independent of the oculomotor control circuitry (31). Lee et al. in a study using event related potentials (ERP) recording, proposed that schizophrenic patients showed reduced bilateral N170 amplitudes in response to faces indicating that these patients have difficulties in face processing (32).

Analyzing the number of fixations on negative-neutral pairs, we found that the patients made fewer fixations on both neutral and negative faces compared to healthy groups, but there was no bias for either of the expressions between the two groups. Some studies have shown that performance accuracy to negative emotions is more severely damaged compared to other emotional valences. Loughland et al. reported that patients with delusional schizophrenia have an attentional bias away from negative expressions (sad and anger), and attributed this bias to the acute nature of the participants’ disorder (33). Our finding did not support this view, but it is in line with findings of other studies which emphasized the face processing impairment in schizophrenia regardless of the emotional theme (34). For example, some recent ERP studies suggest that patients with schizophrenia show longer latencies and reduced N170 peaks in response to all happy, fearful and neutral face stimuli (15) and hence it seems that the type of facial expression has no effect on the pattern seen in these studies.

We did not find any significant differences in number of fixations on positive emotions compared to neutral faces between the two groups. The expression main effect which was seen, show that the both groups made more fixations on positive relative to neutral expressions. This finding is in line with that of previous studies which found that recognition of happy expressions is more accurate, rapid and efficient compared to recognition of sad expressions in both healthy participants and patients with schizophrenia (35). The finding of difference between the patients and controls in attending to positive emotions may indicate that distinct and rapid neural substrates account for the faster processing of happy expressions (19).

The reduced visual attention seen in our schizophrenia group seems to be related to a generalized face processing deficit rather than a specific emotion processing difficulty (35). Nonetheless, another interpretation is that the reduced number of fixations may be related to elevated thresholds for distinguishing emotions in schizophrenic patients so the 500 ms period was not long enough for them to make the discrimination and make a bias toward or away from a specific emotion.

Finally, the correlation between the positive and negative symptoms and the number of fixations was not significant. Findings of previous studies on this correlation are also inconsistent. Walther et al. proposed that deficiency in recognizing non-affective faces in schizophrenia related to positive psychotic symptoms (36). In contrast, another study showed the association of facial emotion perception deficiency in patients with schizophrenia with negative symptoms (37). A probable reason of our finding is that the fact that our patients were all suffering from chronic schizophrenia. It seems that emotion perception impairments are more evident in first-episode patients. It seems that chronically ill patients show such impairments during the periods of clinical remission (38). However, because of the lack of data on association between number of fixations and symptom severity, our findings should be interpreted cautiously. For further clarification of face processing deficits in schizophrenia, we suggest investigating visual attention over longer durations and in different types of acute and chronic phases of the disorder.

This study had some limitations that should be noted. All participants in the patient group were on medication and there may be an effect of medication on visual attention.

In conclusion, the present study revealed that patients with schizophrenia exhibit fewer fixations to facial stimuli regardless of their emotional expression. The findings support the hypothesis that face processing deficit in patients with schizophrenia is not restricted to emotional face, rather it is generalized to all face stimuli and relates to their reduced visual attention to faces.