Visual and associated affective processing of face information in schizophrenia: A selective review

Abstract

Perception of facial features is crucial in social life. In past decades, extensive research showed that the ability to perceive facial emotion expression was compromised in schizophrenia patients. Given that face perception involves visual/cognitive and affective processing, the roles of these two processing domains in the compromised face perception in schizophrenia were studied and discussed, but not clearly defined. One particular issue was whether face-specific processing is implicated in this psychiatric disorder. Recent investigations have probed into the components of face perception processes such as visual detection, identity recognition, emotion expression discrimination and working memory conveyed from faces. Recent investigations have further assessed the associations between face processing and basic visual processing and between face processing and social cognitive processing such as Theory of Mind. In this selective review, we discuss the investigative findings relevant to the issues of cognitive and affective association and face-specific processing. We highlight the implications of multiple processing domains and face-specific processes as potential mechanisms underlying compromised face perception in schizophrenia. These findings suggest a need for a domain-specific therapeutic approach to the improvement of face perception in schizophrenia.

INTRODUCTION

Face perception represents a behavioral capacity to perceive various types of physical and social features embedded in faces. These features include aspects of face as a special visual object, as a personal identity, as a member of a race group, as an indicator of age, as an indicator of gender and as an expression of emotion. The information carried by these facial features is of the paramount importance for social life. As social beings, humans fittingly possess a robust perceptual capacity to process these and other facial features. Compared to perception of non-face visual objects, perception of faces is a highly efficient and sophisticated process (1, 2). A specialized brain network has evolved to support the processing of face information (3–5).

According to Bruce and Young’s functional model of face recognition (6), the perception of various facial features is hierarchically organized, including, but not limited to, functional components of discrimination of individuality (“identity-specific semantic codes”) and analysis of emotion expression (“expression codes”). Other components important for face perception are visual detection, which identifies a visual object as face before probing into more detailed facial features, and working memory which maintains face representations for immediate uses. These components are presumably implemented in the face processing system either serially or in parallel, as illustrated by a hierarchical framework (Figure 1).

Figure 1.

A simplified version of a hierarchical framework of facial processing. Adapted from the face recognition model by Bruce and Young, the framework is consisted of a number of face processing components organized in serial and parallel fashions, with emphasis on visual detection and working memory components.

In schizophrenia, the impairments in processing various facial features, especially those in the emotion domains, have been previously shown (7–14). The question about the mechanisms underlying these perceptual impairments however remained incompletely answered.

Visual and cognitive information contributes to face perception through a bottom-up and a top-down process, respectively (15). With respect to schizophrenia, the bottom-up and top-down processes may be implicated in various behavioral responses (16–20). These findings help to define the question about the mechanisms underlying face processing impairment into a hypothesis of contemplating two distinct but complementary notions, i.e. whether impaired face perception represents a manifestation of cognitive dysfunction as well as affective dysfunction, stems from basic visual processing problems in patients, or even if both types of mechanism are implicated. Recently, a series of investigations has addressed this hypothesis through first evaluating basic visual and cognitive processes as well affective cognitive processes, and then evaluating a potential link between these information processing domains (21–24).

Another question pertaining to this issue is whether face-specific processing, a key component for face perception, is altered in this psychiatric disorder. The issue here is whether or not impairment specific to face processing in schizophrenia exists in addition to deficient visual and cognitive processing. This question, only sporadically addressed in early studies, has recently generated new interest in patient research (25–27), thanks to the advances in knowledge of face-specific processing in the normal perceptual system (4). The answer to these questions may further clarify the nature of face perception problems in this psychotic disorder.

Literature on face processing in schizophrenia has increased rapidly and extensively. Since 1995, several reviews have been published on the affective aspect of face perception (11, 28–30) as well as the non-affective aspect (13, 31, 32) in schizophrenia. While generally comprehensive, these reviews did not specifically target the questions outlined above – the association between visual and cognitive processing and affective processing of face information and integrity of face-specific processing. This review emphasizes the more basic visual aspect of face processing and its relationship with the affective aspect of face processing. Face perception in the context of basic visual and cognitive processing and face-specific processing are highlighted here. This integrated and focused approach to face perception provides a new way to address face-related perceptual functioning in schizophrenia patients.

VISUAL AND COGNITIVE PROCESSING OF FACE STIMULI

Although impaired facial affect recognition is known to be present in schizophrenia (7, 8), an understanding of this impairment’s underlying mechanisms remains incomplete. The processing of affective face signals is clearly impaired (29, 33). Given the hierarchical framework for face perception (outlined above), it is reasonable to ask whether non-affective aspects of visual and cognitive processing such as basic visual detection and discrimination of personal identity also contribute to face-related behavioral functions in schizophrenia patients (13, 14).

Early non-affective face recognition research

Early investigations of face perception in this psychiatric disorder employed faces with neutral expressions to compare with the perception of faces with emotive expressions. One finding was that patients had abnormal performance not only in the presence of emotive face images but also images with neutral expression (34, 35). From a perspective of emotion perception, misattribution of neutral expressions to an emotional category could suggest an affective processing problem. However, from a perspective of non-emotion perception (such as face identity discrimination), poor perception of faces with neutral expressions may instead stem from a non-affective processing problem. This second interpretation implies that the impairment of face perception is not limited to affective domains. The early studies also were limited in their approach to other components of face processing such as visual detection. Their study designs failed to establish grounds for distinguishing basic visual processing components from other face processing components. Nor did the previous investigations systematically differentiate between face perception and the perception of non-face visual objects. The latter limitation is critical because the comparison of face perception and the perception of non-face visual objects would speak directly to the question of whether or not face-specific processing dysfunction is present in this psychiatric disorder.

Research on face-specific processing incorporating visual detection and inversion effect

Recent studies have addressed some of these limitations through studying more isolated facial processing components in schizophrenia. Two of the adopted approaches have been 1) to examine face detection (discerning the presence of a face in visual fields), which is considered putatively the first stage of face perception, and 2) to evaluate face inversion effect (a disproportionately greater performance advantage when recognizing upright vs. upside-down faces relative to recognizing upright vs. upside-down non-face visual objects), which is considered a hallmark of face-specific processing. When using specially processed face images that contain only information about facial configuration but not information about other facial aspects such as emotion expression or identity, one study demonstrated reduced stimulus inversion effect during face detection but not during tree detection in schizophrenia patients, suggesting an abnormality specific to face processing (26). Another study showed deficient performance in perceiving faces of neutral expression, but did not show a reduced face inversion effect in schizophrenia patients (25). These two studies had a substantial methodological difference in that brief stimulus presentation times (100 msec or shorter) were used in the former and relatively long stimulus presentation times (300 msec or longer) were used in the latter. It is possible that under challenging task conditions (e.g. short stimulus presentation times), schizophrenia patients exhibited face detection impairment to a greater extent, which would be consistent with the finding that schizophrenia patients require a longer time delay between target and mask for conscious perception (36). Additional investigations are warranted to clarify whether the same patients would exhibit different extents of face detection impairment at varying stimulus presentation times. Along these same lines, several other studies also found compromised face-inversion effects in patients (37–39) whereas others found no such effects (40, 41). It has also been reported that the detection of human face images, as compared to animal face images, were specifically impaired in schizophrenia patients (42). These investigations suggest that the processing of facial configuration information, relative to the processing of non-face information, is compromised in schizophrenia and future investigations explore the conditions under which face-specific processing might be implicated.

Research on face-specific processing incorporating personal identity discrimination

Personal identity discrimination is another component of face perception into which recent studies have gained new insights. This component mediates the identification of different individuals based upon information conveyed by faces. A few earlier investigations reported poor face identification performance in patients along with their impaired recognition of facial emotion expression (43–45). One study found poor performances when schizophrenia patients were required to judge whether two face images were from one person or had one type of emotional expression (46). These early results had a limitation – they could not determine whether the degraded performances in patients were restricted to the personal identity discrimination domain alone, as other factors such as the capacity to perceive general visual objects might have played a role. One strategy to overcome this issue is to manipulate face-identity-specific signal strengths. By morphing the differences between two facial identities, one study used a series of face images representing a gradual spectrum of change from one individual person to another, and determined perceptual thresholds for discriminating two independent facial identities. Although a moderate performance decrement was found in patients (27), their degraded face identity discrimination and car identity discrimination were not correlated, suggesting a problem in face-specific processing. When one’s own face was included as a stimulus for personal identity discrimination, one study showed that patients were quicker in identifying their own faces than the faces of famous people (a normal performance) (47) yet another study showed impaired recognition of one’s own face in patients if face stimuli were displayed on the right visual field (48). Using the Benton Test of Face Recognition which requires matching one face image with another (49), some studies found impaired face identity discrimination in schizophrenia patients (43, 50, 51) whereas others found no such impairments (33, 52). The diversity of findings may reflect a possibility that a moderate face identity discrimination problem exists in schizophrenia which could only be shown when measurements were administered optimally and not otherwise.

Research on face-specific processing incorporating working memory

Impaired face working memory in schizophrenia appears to be a consistent finding according to several investigations (27, 53–55). Although the results of these investigations agree that face working memory is impaired in this psychiatric disorder, the nature of this impairment is still not well understood. Working memory impairment is a general cognitive problem associated with schizophrenia patients (56–58), which could result in poor working memory that involves faces. The issue of whether or not there exists a working memory impairment specific to the face processing domain has not been systematically evaluated.

Like other perceptual and cognitive impairments, impaired visual detection, personal identity discrimination and working memory of faces may be related to the level of psychiatric symptoms in patients. Impaired facial processing in schizophrenia has been speculated as a factor linked to delusional misidentification (9). A few studies found that behavioral performances in face perception and psychotic statuses were associated in patients (26, 27, 44, 46). Patients’ performances on face identity discrimination, for example, were correlated with positive and negative psychotic symptoms (27). Yet others did not find any such association (43, 45, 55). The relationship between face processing and psychiatric symptoms was equivocal. A recent study made efforts to address this issue by dividing the patient group based on psychosis/mood disorder classification and then comparing face perception performances between the subgroups. With this strategy, this study found only a modest group difference in face detection between schizophrenia patients and patients with schizoaffective disorder (59). This finding suggests that the impairment in this initial stage of face perception is not specific to a subtype of schizophrenia spectrum disorders. The same study, however, found that the performance on face identity discrimination was significantly worse in schizophrenia patients than in patients with schizoaffective disorder. This latter finding suggests that the impaired face identity discrimination and the mood problems in patients with schizoaffective disorder are not associated.

CORTICAL PROCESSING OF VISUAL AND COGNITIVE INFORMATION FROM FACES

Among the core cortical regions subserving face perception (15), the FG primarily mediates the processing of face-specific information (3). Reduced FG volume has been found in both first-episode and chronic schizophrenia patients (60–62). The brain structure change may potentially serve as a neural substrate of the functional change in face perception of schizophrenia patients.

Functionally, brain responses to faces appear to be abnormal in patients. A few electrophysiological investigations found that when responding to faces, patients had reduced EEG amplitudes (63), especially in the components pertinent to the encoding of face features (64). Using fMRI, a few investigations showed a range of brain activations to face stimuli in patients. One fMRI study reported that FG BOLD signals were similar between patients and controls (65). Another fMRI study however reported decreased FG BOLD signals in the patient group (66). In terms of brain hemisphere, one study reported that patients had increased BOLD signals in the left FG (67) whereas another reported patients had decreased BOLD signals in the right FG (68). Given the differences in face perception tasks or in face images used in these studies and heterogeneous brain responses in patients, one may question whether the amplitude of brain response offers an effective index for the characterization of face processing in schizophrenia. Using relative response to face vs. non-face objects to index face processing may better reflect the nature of underlying problems (63).

A key property in the FG and other face-sensitive brain regions is preferential response to face versus non-face visual objects, or face selectivity. Functional integrity of face selectivity has not been systematically evaluated in schizophrenic brains. Behavioral studies in patients have found face-specific perceptual problems as evidenced by altered face-inversion effects (26, 37, 38) (but also see (25, 41)). One ERP study reported that unlike controls, schizophrenia patients did not show a face inversion effect in N170 response (69), suggesting deficient processing of face-specific information. Note that because face-specific processing is supported by face selectivity in the brain, it is critical to examine this key property of face processing in the FG and other face-sensitive brain regions such as occipital face area and superior temporal sulcus. When explored in patients, this knowledge would help to identify and clarify the neurophysiological mechanisms responsible for impaired face perception. Another consideration regarding research of cortical processing of face information is that available behavioral and neuroimaging findings were typically obtained using different paradigms that would tap into different face perception processes. Using similar paradigms in future studies would allow an integration of behavioral and cortical response to faces in patients.

ASSOCIATION OF VISUAL AND AFFECTIVE FACE PROCESSING

In schizophrenia, impaired processing of affective face information has been extensively studied and summarized in the existing literature (11, 29, 30, 64, 70–74). The main result of this literature indicates 1) impaired behavioral performance involving facial emotion recognition, 2) altered responses in the brain regions mediating facial emotion recognition tasks (75) and 3) involvement of different brain regions during performance of such facial emotion recognition tasks (76). On the other hand, the literature on whether or not affective and cognitive face perceptions are associated in schizophrenia has been largely lacking, until recently.

Several recent studies have accumulated evidence for an association between affective and visual face perceptions in this psychiatric disorder. One EEG study found that an early response component which putatively mediates basic visual processing was reduced for face images with emotive expressions (64). The suggestion from this result is that basic sensory processing deficit is a factor for the abnormal processing of facial emotion signals in patients. Another psychophysics study examined contrast detection (an index of visual processing) and facial emotion discrimination, and showed that these two types of behavioral responses are significantly correlated in the same patients (22). The same study also examined personal identity discrimination using faces with neutral expressions and showed that the performances on face emotion discrimination and on face identity discrimination were significantly correlated in patients. A separate study found similar results through examining the relationship between contrast detection and facial emotion perception in patients (24). Taken together, these recent findings suggest that the impairments in processing of affective and visual face information are associated in this psychiatric disorder.

While demonstrating a relationship between visual perception and affective perception of faces in schizophrenia, the aforementioned studies used mostly correlation measures. Such a correlation-based association did not specify the directionality of this relationship: it remained unclear whether impaired visual processing was a causal factor leading to impaired affective processing (21, 77), or vice versa (78, 79). Two recent investigations addressed this causality issue first by manipulating visual features (spatial frequency) of face images and then by examining effects of the manipulation on affective perception. Using a Bubble technique (viewing through randomly located Gaussian apertures or ‘bubbles’ at different spatial frequency scales), one study showed an “atypical strategy” adopted by patients to utilize visual information while perceiving facial emotions (23). Patients, for example, used less bilateral eye regions at high-spatial frequency during identification of fear expressions. Another study found that facial emotion perception in patients was altered to a greater extent by the modulation of spatial frequency contents of face images (80). The findings of both visual manipulation studies thus provide direct evidence for the notion that basic visual processing plays a causal role in impaired facial emotion perception in schizophrenia.

CONCLUSIONS AND FUTURE DIRECTIONS

In schizophrenia, visual and cognitive aspects of face perception are compromised. An important question is whether or not the face perception impairment stems from a face-specific processing problem, or reflects a general visual and cognitive problem. This question has been examined behaviorally in previous and recent studies. Evidence for and against the notion of implicating face-specific processing mechanisms in schizophrenia has emerged. Additional mechanism-driven studies of face-sensitive brain responses are required before a convincing answer is rendered.

Examining the effects of manipulating visual and cognitive contents contained within face images, as adapted in recent investigations, presents a powerful approach to specifying the relationship between different face processing components. Further, the examination of face processing should be extended to the relatives of schizophrenia patients who share the predisposition to schizophrenia, but not the generalized deficits associated with psychotic symptoms (81, 82).

Although a disassociation of the face-specific impairment from generalized visual and cognitive impairments (81, 82) is important, a viable possibility is that both types of impairments exist concurrently and affect jointly face perception in patients. It was reported that face emotion perception deficit in schizophrenia was not specific to a particular emotion category when compared to healthy controls’ results obtained with degraded face images (83). To understand specific visual and cognitive mechanisms underlying face perception problems in schizophrenia, innovative and sophisticated study designs are needed.

The finding of an association among visual, cognitive and affective aspects of face perception points to a new approach to further understand and remedy impaired facial emotion perception in patients. That is to highlight the roles of basic visual processing and face-specific processing in future research, not only to more fully understand mechanisms underlying face processing impairment but also to develop more targeted therapeutic approaches to improve face-related perceptual and social functioning of patients.