The Core Brain Region for Face Processing in Schizophrenia Lacks Face Selectivity

Abstract

Background:

Face perception impairment in schizophrenia has long been recognized. However, brain mechanisms underlying this socially important perceptual deficit are not well understood. Previous magnetic resonance imaging (MRI) studies have shown that patients have altered structure in brain regions responsible for processing face information, but functional properties of these brain regions are not clearly determined. A key functional property of the face-processing system—face selectivity—has yet to be evaluated in schizophrenia.

Methods:

We used functional MRI (fMRI) to examine face selectivity of 3 core face-processing regions—fusiform face area (FFA), occipital face area (OFA), and superior temporal sulcus (STS)—in schizophrenia patients (n = 24) and healthy controls (n = 23). To disassociate face-specific processing from general perceptual processing, we compared cortical activations during performance of perceptually equated face and tree detection tasks.

Results:

Activation levels of the 3 putative face-processing regions during face detection did not differ between patients and controls, being similar for FFA and OFA and absent for STS. However, face selectivity, indexed by the difference in cortical activation between face and tree detection, was significantly reduced in patients for FFA, especially for low-contrast stimuli. FFA activation and perceptual performance during face detection were associated in patients.

Conclusions:

These results show a lack of face-specific processing in the schizophrenic brain region presumably subserving face perception. This finding suggests boosting visual salience of face images as a potential therapeutic venue for improving face perception in this psychiatric disorder.

Introduction

One cause of the social functioning problems in schizophrenia may be impairment in processing face information.^1–3 Face processing begins with the detection of the presence of faces in the visual world. Face detection allows a face to be properly distinguished from nonface visual objects and is presumably required before the more detailed face-specific analyses (such as recognition of emotional expression and identity) can occur.⁴ Patients have shown face detection impairment,^5,6 yet the brain mechanisms underlying this perceptual deficit are not understood.

Faces are a privileged class of visual object that is processed in specialized cortical regions, including the fusiform face area (FFA),⁷ occipital face area (OFA),⁸ and superior temporal sulcus (STS).^9,10 The FFA has been the most consistently observed structure in participating in several levels of face processing including face detection.^11,12 These regions emphasize various functional aspects of face recognition—FFA for face configuration information, OFA for facial parts such as eyes, and STS for facial social contents such as eye gaze. These 3 core face-processing regions show face selectivity or selective activations during perception of faces as opposed to nonface objects.^7,13 This face-selectivity property allows a greater sensitivity to faces, an important neural processing property especially when detecting weakened face signals.

In schizophrenia, one potential mechanism for face detection deficits is dysfunction in the core face-processing regions. Structurally, reduced fusiform gyrus volume has been found in patients.^14,15 However, functional magnetic resonance imaging (fMRI) work on FFA activation has yielded mixed results, with some showing spared blood oxygen level dependent (BOLD) responses¹⁶ when patients passively viewed face images, others showing reduced responses^17,18 when patients recognized affective and nonaffective face images, and still others showing increased responses when patients performed a gender discrimination task.¹⁹

A couple factors may contribute to the discrepant FFA activations in patients. First, face stimuli salience differed in these previous studies. Visual sensitivities are generally reduced in patients, which may alter their cortical responses (especially those in an early processing stage) to face stimuli (especially to those with low visual salience). Second, face perception involves face-specific and general perceptual processing. These 2 types of face-related processing were not always separated in previous studies which examined mostly FFA activations just to face stimuli (but Yoon et al¹⁶).

Cortical activations in OFA have not been specifically evaluated in schizophrenia. While having been evaluated in this psychiatric disorder, STS activations have not been measured in the context of face-specific processing.

Therefore, at least 2 critical issues remain unresolved in regard to the face-processing system in schizophrenia. First, while previous studies assessed patients’ face-related cortical activities, none of them have assessed functional activations of FFA or other face-processing regions during face detection, the initial stage of face-processing. Second, previous studies used diverse face stimuli but did not take into account the impaired and heterogeneous capacity of basic visual processing in patients.^20–23 It has been reported that visual sensitivity to contrast, a major stimulus variable that influences the detection of visual targets including faces, is altered in schizophrenia.^24,25

In this study, we acquired patients’ fMRI responses during both performance of face detection and tree detection. Given patients’ behavioral impairments in face-specific tasks^5,26 (but also Butler et al⁶), we hypothesize reduced face selectivity of the core face-processing system (FFA, OFA, and STS) in schizophrenia. We customized fMRI protocols for individual patients according to their predetermined perceptual thresholds to ensure similar visual inputs during acquisitions of cortical activation. Considering the putative link between face perception and the cortical systems subserving face-processing, we hypothesize that patients’ performance in face detection is associated with activations of the core face-processing regions.

Methods

Participants

Twenty-four schizophrenia patients participated. All met the DSM-IV criteria for schizophrenia or schizoaffective disorder. Diagnoses were made independently by experienced clinicians, based on a review of a structured clinical interview for DSM-IV conducted by trained interviewers²⁷ and by evaluating available medical records. Ten patients had schizophrenia and the rest had schizoaffective disorder. All patients were chronically ill (duration of illness: mean = 23.9 y, SD = 14.8). Twenty-one patients were taking antipsychotic medication (daily chlorpromazine equivalent: mean = 579.4mg, SD = 579.6).²⁸

Twenty-three controls participated. None met DSM-IV criteria for any psychotic condition (lifetime) and for any schizotypal, schizoid, or paranoid personality disorder, based on the Structured Interview.²⁹ None had a family history of psychosis.

No participants had any diagnosed organic brain disease or any history of substance abuse or dependence during the past 2 years, as reported during the screening interviews. The groups did not differ in terms of age or gender, but controls had a higher verbal IQ (administered with Wechsler Adult Intelligence Scale [WAIS]³⁰) (t(37) = 2.42, P = .021) (table 1). Written informed consent was obtained from all participants prior to testing.

Table 1.

Demographic Characteristics for all Subjects and Clinical Rating for Patients

Group	Sex	Age (y)	Verbal IQa	PANSS+	PANSS−	PANSSgen
Control	n = 23 (f = 11)	39.13 (15.38)	114.82 (13.75)
Patient	n = 24 (f = 10)	44.83 (13.11)	103.84 (16.1)	17.14 (4.97)	13.68 (4.1)	30.36 (7.44)

SD in parentheses.

^a< .05

Procedure and Equating Perceptual Stimuli for fMRI

Prior to fMRI, psychophysical testing was conducted to determine perceptual performance levels of all participants. The task was to detect the presence of a target (face or tree). The face detection task involves a holistic process of visual information that requires detection of a face as a whole (or a pop-out). The holistic processing depends on global visual information and occurs in an early stage of the perceptual system.^31,32 Nevertheless, this seemingly simple task relies on complex cortical networks for detecting the presence of a face. Perceptual threshold, defined as the lowest contrast level at which a participant can reliably (ie, 80% correct) detect a stimulus, was first measured (supplementary material 1) and then used to equate visual stimuli for fMRI.

For the fMRI, face and tree stimuli were set for each participant at 4 contrast levels: 0% contrast (a control condition), perceptual threshold (Th, personalized), 2 times perceptual threshold (Th2, personalized), and 100% contrast. Figure 1 shows the stimuli at perceptual threshold and 100% contrast levels.[AU: Pl]

Fig. 1.

Illustration of face and tree stimuli at average perceptual threshold and 100% contrast.

Participants lay in a supine position and viewed visual stimuli that were back-projected on a screen and reflected in a mirror attached to the MRI head coil. The fMRI protocol used an event-related design over 4 runs, 2 for face detection and 2 for tree detection (presented in a counterbalanced order across participants). The stimulus presentation sequences were determined by optseq2³³ such that overlapping hemodynamic responses for each condition could be efficiently deconvolved. Each of the 4 runs had 192 trials (4 contrast levels, 2 target locations, and 24 repeats). In each trial, a stimulus was displayed for 300ms. The average interstimulus interval was 2056ms. Participants’ perceptual judgments (a target on the right or the left) were given using an MR compatible button box.

MRI Acquisition

All MRI images were collected at the McLean Imaging Center (Belmont, MA) on a 3.0 Tesla Siemens TIM Trio scanner (Siemens AG) using a 32-channel head coil. High-resolution structural images (Repetition Time [TR] = 2.1 s, Echo Time [TE] = 3.3ms, slices = 128, matrix = 256×256, flip angle = 7°, resolution = 1.0×1.0×1.33mm) were used to register subjects’ imaging data to a standard space. Structural imaging data were read and interpreted by a clinical neuroradiologist to rule out neurological abnormalities.

To increase temporal resolution while ensuring sufficient spatial resolution, fMRI were acquired using a state-of-the-art multiband imaging technique. Each functional run had 32 axial slices (3.5 mm each) with TR/TE = 400ms/30ms, flip angle = 30°, matrix = 64×64 on a 220×220mm field of view, in plane resolution = 3.44×3.44mm, and multiband factor = 8.^34,35

Analysis of fMRI Data

Analysis of fMRI data was performed using the fMRI Expert Analysis Tool (FEAT) version 6 (FMRIB’s Software Library, http://www.FMRIb.ox.ac.uk/fsl).³⁶ After preprocessing (supplementary material 2), data were analyzed with a 3-level hierarchical general linear modeling strategy. For the first 2 levels, a standard fixed-effects analysis was performed. The first-level analysis was for each individual run, whereas the second-level analysis combined runs for the same task (eg, the 2 face runs). The third-level analysis used ANOVA on second-level parameter estimates averaged over regions-of-interest (FFA, OFA, STS) across subjects for each condition and group. An exploratory third-level whole brain analysis was also performed using a GLM to combine subjects in a mixed-effects analysis (Flame 1) by group.

Dependent variables were mean percent signal change values for each region-of-interest (FFA, OFA, and STS) extracted from spatial maps of regression coefficients using Featquery in FSL for each condition and group. Coordinates for region of interests (ROIs) (depicted in figure 2) were based on previous work probing face-selective processing³⁷ (Supplementary material 2). They were as follows in Montreal Neurological Institute (MNI) space: left FFA (x = −40, y = −59, z = −23); right FFA (x = 42, y = −51, z = −24); left OFA (x = −41, y = −84, z = −17); right OFA (x = 44, y = −78, z = −17); left STS (x = −55, y = −50, z = 3); right STS (x = 55, y = −50, z = 3).

Fig. 2.

The blood oxygen level dependent signal changes (Z-maps) for face detection and tree detection in MNI space (x = 42, y = −50, z = −18) for controls and patients. Each column represents a different stimulus contrast level (Th, Th2, and 100%) separately by task.

Results

Behavioral Responses

For pre-fMRI testing, patients showed higher perceptual thresholds (poor performance) than controls (F[1,45] = 5.39, P < .025) (figure 3a), and thresholds were higher for faces than trees (F[1,45] = 9.17, P = .004) over both groups. There was no interaction of task and group.

Fig. 3.

(a) Pre-functional magnetic resonance imaging perceptual thresholds for face detection and tree detection. The group of individual participants is denoted “c” for control and “p” for patient. (b) Average percent correct (y axis) during the scanning is plotted as a function of stimulus contrast (x axis). Error bars represent SE.

For performance accuracy (% of correct trials) during fMRI, no effects of group or interactions with group were found (Ps > .05, figure 3b). An effect of contrast was found (F[1,45] = 28.04, P < .001) where performance improved as contrast increased. Face and tree detection performances for each contrast including the threshold are above the chance level (binomial tests, Ps < .05). Analysis of reaction time is provided in supplementary material 4.

ROI BOLD Responses

Three ANOVAs, one each for FFA, OFA, and STS, were performed on BOLD signal change with factors of group by task (face vs tree) by contrast level by hemisphere. The values used for this analysis were extracted from parameter estimate contrasts of the 0% condition versus each of the other 3 contrasts (ie, ANOVA contrasts were: Th vs 0%, Th2 vs 0%, 100% vs 0%) for each task.

These ANOVAs yielded a significant interaction of task by contrast by group for FFA (F[2,90] = 3.81, P = .026; figure 4). This interaction was nonsignificant for OFA (F[2,90] = 2.76, P = .072; figure 4). No effect was found for STS.

Fig. 4.

Means of BOLD signal changes at fusiform face area (top) and occipital face area (bottom) for controls (blue) and patients (red). Task (face and tree) and contrast level (Th vs 0%, Th2 vs 0%, and 100% vs 0%) are depicted on the x axis. Error bars denote SE. BOLD, blood oxygen level dependent; FFA, fusiform face; OFA, occipital face area.

Separate ANOVAs for each group were performed to follow up the 3-way interaction. These analyses revealed a significant interaction of task by contrast for controls (F[2,44] = 10.87, P < .001), but not for patients. This interaction indicates controls’ FFA activation changed with task (face vs tree) and contrast, whereas patients’ FFA activation did not. Further breakdown for this interaction (t tests to understand the interaction found in controls) revealed that at the 2 low contrasts (Th and Th2), controls had significantly greater FFA activations during the face than during the tree detection tasks (t(22) = 4.53, P < .001 at Th; t(22) = 3.24, P < .01 at Th2). At the 100% contrast, the activation did not differ between the 2 tasks. In patients, however, the activations were similar during the 2 tasks at all 3 contrasts.

These analyses, along with the analysis of BOLD timecourse (supplementary material 3), indicate that controls had greater FFA activations for low-contrast face stimuli than for low-contrast tree stimuli, whereas patients lacked such a face-selective response, regardless of stimulus contrast.

Relationship Between Perceptual Performances and Cortical Activations

In patients, FFA and OFA activations during face detection were significantly correlated with perceptual threshold (right FFA, r = .521, P < .01 (figure 5); left FFA, r = .495, P < .05; right OFA, r = .527, P < .01; left OFA, r = .574, P < .01). This result suggests a relationship between perceptual sensitivity to face and cortical activation in patients. Patients also showed a relationship between OFA activation and perceptual threshold during tree detection (right, r = .482, P < .05; left, r = .469, P < .05). In controls, only the left OFA activation during face detection showed a significant relationship with perceptual threshold (r = .465, P < .05).

Fig. 5.

Scatter plot for correlation between BOLD response in right fusiform face area and pre-functional magnetic resonance imaging determined perceptual thresholds for face detection. Each data point represents a patient. BOLD, blood oxygen level dependent; FFA, fusiform face.

Relationships Between Cortical Activations and Clinical Symptoms

While only one individual correlation between Positive and Negative Syndrome Scale (PANSS) scores and BOLD activations reached statistical significance (table 2), a pattern emerged during face detection but not during tree detection (the top 2 sections of table 2). To assess this pattern statistically, correlations across all ROIs and PANSS scores were compared between the 2 tasks (12 pairs each), using a Wilcoxon-Signed Ranks Test. The correlation comparison test revealed significantly higher correlations for face detection than for tree detection (Z = 3.06, P = .002). When face-selectivity index (differences between BOLD responses to face and to tree) was compared with PANSS, a similar result was found (the bottom section of table 2). The result that the entire set of correlations between FFA activation and PANSS showed a clear trend whereas individual correlations did not (except for one) may be due to the limited sample size for singular comparisons in this type of neuroimaging study.

Table 2.

Correlation Coefficients (P Value in Parentheses) Between BOLD Responses and Psychotic Symptomology

Note: The BOLD responses are averaged across stimulus contrasts. The symptoms are measured by PANSS.⁵¹ The color scale indicates the strength of the correlation coefficient. The top 2 sections are for BOLD responses to faces and trees, respectively. The bottom section is for face-selectivity index (differences in BOLD response to faces and to trees). BOLD, blood oxygen level dependent; ROI, region of interest; PANSS, Positive and Negative Syndrome Scale; FFA, fusiform face; OFA, occipital face area.

No correlation was found between chlorpromazine and FFA or OFA activations under any conditions (Ps > .05).

Discussion

This study demonstrates a lack of FFA face selectivity in schizophrenia. This is the first work, to our knowledge, investigating cortical correlates of face detection in schizophrenia. With equated visual inputs across participants and between visual tasks, the results of FFA activation in patients indicate a dysfunction in face-specific processing.

Face Selectivity

Face selectivity is a key property of the face-processing system that subserves highly efficient and sophisticated face-related cognitive functioning. This key property of cortical face processing has not been investigated systematically in schizophrenia.³⁸ By comparing activations between face and tree detection, this study found a lack of face selectivity in FFA of schizophrenic brains for nonsalient stimuli (perceptual threshold and 2 times perceptual threshold). This finding, consistent with several behavioral studies, provides the first evidence of dysfunction of face-specific cortical processing in schizophrenia and offers a mechanism for understanding a series of face perception problems displayed in patients.

First, face perception impairment in patients may not be interpreted by reduced activations in the putative face-processing regions. In agreement with previous work,¹⁶ this study did not find overall group differences in FFA response to faces. Overall spared activation in schizophrenic brains is reported in other perceptual and cognitive domains such as memory retrieval³⁹ and visual processing.^40,41 Therefore, relevant behavioral problems such as face perception impairment must be mediated through more sophisticated cortical mechanisms beyond levels of activation.

Second, face perception problems in patients may not be interpreted as a result of deficits in basic visual processing. Face detection relies on visual signals. It would be reasonable to ask if basic visual processing deficits^23,42,43 lead to the face detection problems. This study resolved this issue by experimentally equating visual inputs (contrast level) among patients and controls with diverse visual capacities (figure 3). The results obtained with such procedures are not confounded with individual differences in basic visual sensitivity and should be attributed to face-specific processing.

Third, the absence of FFA face-selective response in patients highlights a lack of face selectivity in this psychiatric disorder. The significant group difference in face selectivity for low-, but not for high-, contrast stimuli (figures 4) indicates diminished sensitivity to low-salience faces and nondifferential responses to face and tree in this putative face-processing region. The lack of face-selective responses to low-salience faces disallows efficient and sophisticated processing during face detection. Although many faces we encountered have high salience, the perceptual processes involved may not necessarily be optimal if face configuration is masked by other conditions (such as lighting and viewing angle). In this sense, the results of diminished sensitivity for low-contrast stimuli found in the present study may be applicable when patients encounter faces in nonoptimal situations in which face configuration is not easily discerned. A mechanism for simply detecting the presence of faces in the environment should be especially well equipped for situations where viewing conditions are not optimal. For highly salient faces, the face-selectivity mechanism may not be essential, as was the case for the 100% contrast condition.

Fourth, controls’ OFA and FFA responses to faces did not change with contrast. Conversely, their responses to trees were modulated by contrast. This highlights functional specialization of the face-processing regions. Unlike controls, patients’ FFA and OFA responses to faces and trees were unaffected by contrast, evincing a lack of functional specialization. That is, in schizophrenia, face is encoded similarly to nonface object, making face detection possible but inefficient.

Relationships Between Brain Responses, Performance, and Clinical Symptoms

In patients, consistent correlations were found between perceptual thresholds and FFA and OFA responses during face detection (figure 5). This relationship was also found for trees, but in OFA only. This relationship suggests an association between face detection deficit and the putative cortical substrates. Noteworthy is the direction of the correlations—the greater the cortical activation for face detection, the higher the perceptual threshold (or the lower performance level). This suggests that the face-processing system operates as efficiency machinery—high-level perceptual performance is supported by low-level cortical activations. Patients’ reduced face detection performance is associated to their increased activations in the face-processing regions. The low face-processing efficiency may stem from a low reserve of available brain anatomy (such as reduced volume of fusiform gyrus¹⁴) for this presumably sophisticated neural capacity. Given the promising efficacy of perceptual and cognitive training in schizophrenia,^44,45 applying an intervention approach to face-specific tasks should help patients improve face perception.

Consistent positive relationships were shown between patients’ symptoms and cortical activations for faces, but not for trees (table 2). This suggests that responses to faces in these face-specialized regions are specially associated to the severity of psychotic symptoms. The lack of face selectivity may reduce perceptual sensitivity for detecting faces and hinder social interactions. Severely and long-term poor interactions with external world might in turn lead to a recalibration of one’s internal system and instantiate positive and negative symptoms.

The Performance and FFA Activation During MRI

In previous studies (eg, ^5,6), a group difference in face detection performance was found for stimuli with a uniform high contrast (100%). The lack of such a group difference during fMRI here (figure 3b) may reflect a greater reduction of controls’ performance level, due to less ideal stimulus presentation, supine position of subjects, and loud noise associated with MRI environment. When individualized contrasts were used, patients typically got a higher contrast stimulus which may be a reason for relatively better performance. Thus, group differences in face detection performance is expected to diminish under the individualized contrast conditions, as intended and occurred in this study.

A question may arise as to whether the performance and FFA activation of patients were affected by a generalized deficit or fatigue factor. This may not be the case, for following reasons. The low- and high-contrast trials were mixed sequentially during fMRI, which makes it difficult to attribute the group difference for the low-contrast stimuli to a fatigue factor. Also the task requirements were the same for low- and high-contrast stimuli, which makes it difficult to attribute the effect to a high-level cognitive factor. Generalized deficit in schizophrenia⁴⁶ is conceptualized and evaluated as cognitive performance decrement resulting from accumulative stages of information processing. On the other hand, the group difference in this study reflects cortical activations of a particular region (FFA). This result, together with the lacking of an association between FFA activation and verbal IQ in patients (supplementary material 2), suggests that the yielded group difference is genuine to face-specific processing.

Limitations and Future Directions

Several existing limitations spur future investigation. First, as only face detection was evaluated here, it is yet to be determined whether face-selectivity dysfunction in schizophrenia exists for other stages of face processing, such as identity or emotion recognition. Second, the result from only one type of nonface stimuli, tree, should be replicated with other types of nonface visual objects.^7,47 Third, for both controls and patients, STS remained virtually inactive during face detection. This is not entirely unexpected as STS mostly engages in the processing of dynamic face signals such as eye and mouth movements,⁹ whereas the face detection task involves only static face images. A different face perception task (other than face detection) should be used to explore face processing of STS. Fourth, impairment in facial emotion recognition has been associated with basic face perception deficits in schizophrenia.^48–50 Previous reports on misattribution of fear/threat to neutral expressions typically employed photo images that include various types of face information. Our stimuli have an advantage in this regard because only configural face information is included. As such, it makes it less likely for such a misattribution to occur. Eventually, this issue should be addressed in future studies in which an explicit emotion perception assessment is obtained even for neutral faces. Follow-up studies should also evaluate the relationship of perceptual and affective processing at cortical response levels. This relationship would further define target domains for the development of therapeutic interventions.

Concluding Remarks

By contrasting cortical responses to face versus nonface signals and controlling visual inputs, this study found that the face-specific ventral stream structure FFA in schizophrenia lacks a key property—face selectivity—when stimulus salience is low. The altered FFA responses are associated with impaired perceptual performances and clinical symptoms. This work suggests that remediation of face-related social functioning problems in schizophrenia should steer toward the face-specific processing domain, rather than general perceptual processing domains.

Funding

This study was funded by grant R01 MH 096793 from the National Institutes of Health.