Temporal Lobe Structures and Facial Emotion Recognition in Schizophrenia Patients and Nonpsychotic Relatives

Abstract

Temporal lobe abnormalities and emotion recognition deficits are prominent features of schizophrenia and appear related to the diathesis of the disorder. This study investigated whether temporal lobe structural abnormalities were associated with facial emotion recognition deficits in schizophrenia and related to genetic liability for the disorder. Twenty-seven schizophrenia patients, 23 biological family members, and 36 controls participated. Several temporal lobe regions (fusiform, superior temporal, middle temporal, amygdala, and hippocampus) previously associated with face recognition in normative samples and found to be abnormal in schizophrenia were evaluated using volumetric analyses. Participants completed a facial emotion recognition task and an age recognition control task under time-limited and self-paced conditions. Temporal lobe volumes were tested for associations with task performance. Group status explained 23% of the variance in temporal lobe volume. Left fusiform gray matter volume was decreased by 11% in patients and 7% in relatives compared with controls. Schizophrenia patients additionally exhibited smaller hippocampal and middle temporal volumes. Patients were unable to improve facial emotion recognition performance with unlimited time to make a judgment but were able to improve age recognition performance. Patients additionally showed a relationship between reduced temporal lobe gray matter and poor facial emotion recognition. For the middle temporal lobe region, the relationship between greater volume and better task performance was specific to facial emotion recognition and not age recognition. Because schizophrenia patients exhibited a specific deficit in emotion recognition not attributable to a generalized impairment in face perception, impaired emotion recognition may serve as a target for interventions.

Introduction

Temporal lobe dysfunction has long been thought to be a core element of the schizophrenia diathesis.^1,2 Important functions of the temporal lobe include face and emotion perception, which are impaired in schizophrenia. The goal of the present study was to investigate temporal lobe structural abnormalities and directly examine their relevance to facial emotion recognition deficits in schizophrenia patients and nonpsychotic first-degree biological relatives. Investigating both patients and family members allowed a better examination of genetic (familial) liability, as well as disease-related processes.

The temporal lobe is involved with the processing of sound, speech, and lexicon, as well as perception of faces and facial affect. A body of imaging, psychophysiology, and lesion studies demonstrate that the fusiform or occipitotemporal gyrus has an important role in facial perception.³ Emotion-specific perception appears to further increase fusiform activation.^4–6 Neuroanatomical studies have detailed intricate connections of the fusiform gyrus with the hippocampus and amygdala, which appear to underlie a network that processes emotional stimuli, including faces.⁷ Also, activity of select neurons in the medial temporal lobe have been found to discriminate faces from inanimate objects suggesting a specific role of cells in this region in aspects of face perception.⁸ Medial temporal neurons have additionally been shown to respond to certain emotional expressions.⁸ Another temporal lobe region, the superior temporal sulcus, is activated by both static and dynamic motion of faces and bodies and thereby functions as another node of the facial emotion processing system.⁹ Lastly, a meta-analysis of emotion viewing tasks in healthy individuals revealed the bilateral fusiform, amygdala, and middle temporal gyrus as reliably activated, indicating that these regions are associated with the processing of emotions.¹⁰

Investigations have typically found schizophrenia patients to have reduced temporal lobe volume compared with control groups (for review, see Lawrie and Abukmeil¹¹ and Shenton et al¹²). Medial temporal, superior temporal, middle temporal, inferior temporal, and fusiform regions all appear to be structurally smaller in patients than controls.^13,14 In addition, medial and lateral temporal lobe volumes exhibit progressive reductions as high-risk individuals precede toward the onset of schizophrenia.¹⁵ Prominent abnormalities in temporal lobe gray matter also appear to be present in nonpsychotic adult relatives¹⁶ and have been identified in medial temporal,^17–19 superior temporal,¹⁷ and middle temporal¹⁷ regions. Thus, abnormalities in temporal lobe volume may be an expression of genetic liability for schizophrenia.

In addition to structural abnormalities of the temporal lobe, schizophrenia patients have difficulty in recognizing basic facial emotions such as happiness, sadness, anger, and fear.²⁰ Emotion recognition deficits could underlie the misinterpretation of others’ intentions and subsequent social isolation in patients.^21,22 Furthermore, impaired emotion recognition in schizophrenia may be related to poor work function and limited independence in living.^23–26 Kee and colleagues²³ found facial emotion recognition was related to both work functioning and independent living at both baseline and after 1 year of psychosocial rehabilitation in a sample of schizophrenia patients. Further causal statistical modeling suggested that difficulties in facial emotion perception may impede work functioning and independent living in schizophrenia patients over the course of 1 year.

First-episode schizophrenia patients have shown difficulties in correctly identifying emotions; therefore, these abnormalities are likely independent of medication effects.²⁷ Prodromal patients are also impaired in correctly identifying emotions compared with controls, suggesting that these abnormalities may predate acute symptoms of the disorder.²⁷ Additionally, high-risk individuals have been found to misattribute negative emotions (mostly sad) to neutral faces, which may reflect biased affect perception associated with genetic vulnerability for schizophrenia.²⁸ A number of studies have found emotion recognition deficits in nonpsychotic adult relatives of patients with schizophrenia, further supporting the idea that the abnormality may mark genetic liability for the disorder.^29,30 Finally, there is some evidence that emotion recognition deficits are a specific deficit and not a result of a generalized cognitive dysfunction in the disorder^20,31,32; however, this is not an entirely consistent finding.^33,34

Two studies provide evidence for a relationship of fusiform gyrus volume with perception and memory of faces. Onitsuka and colleagues³⁵ demonstrated schizophrenia patients had smaller fusiform gray matter volumes and lower immediate and delayed facial memory accuracy compared with controls. Importantly, delayed recall for faces was correlated with fusiform volumes only in patients suggesting a structural abnormality selectively associated with a behavioral impairment in the disorder. In an additional study, Onitsuka and colleagues³⁶ found schizophrenia patients had reduced bilateral N170 amplitudes in response to faces, as well as smaller fusiform gray matter volumes compared with controls. Smaller right fusiform volumes were associated with greater reductions in N170 amplitude measured at the right posterior temporal site in patients. Thus, these 2 studies suggest a link of fusiform volume and function with facial perception deficits in schizophrenia; however, the link between temporal lobe regions and facial emotion recognition in the disorder has still to be elucidated.

The goal of the present study was to further examine the significance of temporal lobe abnormalities in schizophrenia in several ways. First, to more fully understand the structural correlates of face perception and facial emotion recognition, we measured several temporal lobe regions (the fusiform, superior temporal, middle temporal, amygdala, and hippocampus) that are associated with face processing in normative samples as well as abnormal in schizophrenia. Second, to determine whether slowed processing accounted for facial emotion recognition impairments in schizophrenia, we modified an established facial recognition procedure²⁰ to examine whether observed deficits were dependent on the amount of time participants were allowed to view faces. Lastly, we investigated temporal lobe structures and facial emotion recognition in first-degree nonpsychotic biological relatives of schizophrenia patients in addition to patients and controls to reveal both familial (genetic) and disease effects on the abnormalities.

Methods

Participants

Schizophrenia and schizoaffective probands were recruited from the Minneapolis VA Medical Center outpatient clinics and community support programs for the mentally ill. Research staff identified first-degree biological relatives by completing a pedigree with the proband. Controls were recruited through posting announcements in the community.

Twenty-seven schizophrenia and schizoaffective patients (3 bipolar subtype, 2 depressed subtype; hereafter referred to as schizophrenia patients), 23 nonpsychotic first-degree biological relatives of schizophrenia patients, and 36 community control subjects participated in the neuroimaging protocol. Participants were excluded if English was their second language, had mental retardation, current alcohol abuse, current drug abuse/dependence, a current or past central nervous system condition, history of head injury with skull fracture or substantial loss of consciousness, a history of electroconvulsive therapy, and an age less than 18 or greater than 60 years. Relatives and controls were additionally excluded for a current major depressive episode, current or previous use of antipsychotic medications, or a personal history of psychosis or bipolar affective disorder. Relatives and controls were also excluded for a Cluster A personality disorder to reduce the possibility that subthreshold symptoms may affect brain structure and task performance. Lastly, controls were excluded for a family history of psychosis or bipolar affective disorder. Participants were recruited from a larger family study conducted through the Minneapolis VA Medical Center because they fulfilled the outlined criteria. Therefore, patients and relatives were not necessarily biologically related to one another in the sample who underwent the neuroimaging. The Minneapolis VA Medical Center and University of Minnesota Institution Review Boards approved the protocol.

Diagnosis and Assessment

To obtain diagnostic information, the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders and the Psychosis Module of the Diagnostic Interview for Genetic Studies³⁷ were completed with each participant. Axis II Cluster A traits in relatives and controls were assessed with the Structured Interview for Schizotypy³⁸ with the addition of supplemental questions. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision diagnoses were determined after review of clinical interview materials and medical records by a doctoral-level clinical psychologist. Schizophrenia patients’ current symptomatology was assessed using the Scale for the Assessment of Negative Symptoms (SANS)³⁹ and the Scale for the Assessment of Positive Symptoms (SAPS).⁴⁰ All participants had their psychiatric functioning assessed using the Brief Psychiatric Rating Scale.⁴¹

Structural Neuroimaging

Structural images were acquired using a standard MP-RAGE sequence (160 slices) on a 3T Siemens scanner at the Center for Magnetic Resonance Research, University of Minnesota. Images were collected with an 8-channel head coil system prior to a scanner upgrade after which images were collected with a 12-channel head coil system. Parameters of the magnetization prepared rapid acquisition gradient echo (MP-RAGE) sequence prior to scanner upgrade were as follows: repetition time (TR) = 1600 ms, echo time (TE) = 4.38, flip angle = 15°, field of view (FOV) = 256 mm, thickness = 1.0 mm. Parameters of the MP-RAGE sequence postscanner upgrade were as follows: TR = 2300 ms, TE = 2.98, flip angle = 9°, FOV = 256 mm, thickness = 1.2 mm. There was no difference in group number distributions pre–post scanner upgrade (X²(2) = 0.34, P = .84). To assess the potential impact of scanner upgrade and head coil change on the structural neuroimaging data, a MANOVA on the temporal lobe brain regions was conducted with pre–post scanner as a factor in the model. There was an overall effect of head coil version on the structural neuroimaging data (F(10,71) = 8.39, P < .0001), with greater gray matter volumes postscanner upgrade. Importantly, there was no interaction between scanner version and group (F(20,144) = 0.59, P = .91). Furthermore, there was no interaction between scanner version and group on any of the individual regions. Given these findings, it is unlikely the scanner version and head coil change will affect the group differences.

Volumetric segmentation, cortical reconstruction, and cortical and subcortical parcellations were performed with the Freesurfer image analysis suite, version 4.0.3 (http://surfer.nmr.mgh.harvard.edu/). Briefly, Freesurfer processing included motion correction, removal of nonbrain tissue, automated Talairach transformation, watershed/surface deformation procedure, segmentation of the subcortical gray matter volumetric structures, intensity normalization, tessellation of the gray matter white matter boundary, and surface deformation.^42,43 Further data processing and analysis included surface inflation, registration to a spherical atlas which utilized individual cortical folding patterns, and parcellation of the cerebral cortex.^44,45 Freesurfer morphometric procedures demonstrated good test–retest reliability across scanner manufacturers and across field strengths.⁴⁶ This study focused on a priori defined fusiform, superior temporal, middle temporal, amygdala, and hippocampal regions.

Facial Emotion and Age Recognition Tasks

The facial emotion recognition task and age recognition experimental control task were designed after the task of Schneider and colleagues.²⁰ The task and stimuli were acquired from Dr Schneider. Participants responded target or nontarget to the emotion requiring discrimination in that block (eg, within the happy block, participants would view a face and determine whether the emotion depicted was happy or not and respond with a button press accordingly). This study investigated participant’s ability to evaluate 4 facial emotions, anger, fear, happy, and sad. Each emotion was presented in blocks of 60 faces (16 target emotion expression trials, 16 nontarget emotion expression trials distributed among the 3 other emotion categories, and 28 nontarget neutral expression trials).

The age recognition task required participants to respond whether or not the face presented was above or below the age of 30 years. Similar to the emotion recognition task, this task was presented with 60 faces (32 emotion expression trials, 28 neutral expression trials). There were 24 faces under 30 years of age and 36 faces over 30 years of age. This task was used as an experimental comparison task to control for aspects of facial perception, as well as task engagement, attention, and response selection.

Half of all the depicted faces for each emotion and age recognition block represented each gender. Four races were represented in the images. The distribution of races across emotion and age blocks was similar: Caucasian (58%–67% of faces per block), African American (20%–28% of faces), Asian (3%–7% of faces), and Hispanic (7%–12% of faces).

Each block had 2 presentation types. The first presentation type was time-limited, modeled after Schneider and colleagues.²⁰ In this condition, each face was presented for 3 s after which it would automatically advance to the next face. Participants were aware of the need to respond within the 3 s window. The second presentation type was time-unlimited (ie, self-paced). Each face was presented until the participant responded. In both presentation types, faces were presented one after another without an interstimulus interval. Presentation type, emotion block type, and face within block were randomized at each presentation. Participants responded with separate button presses for target and nontarget trials. Buttons for target and nontarget responses were counterbalanced across participants. For each facial emotion and age recognition block, the target (eg, happy) and nontarget (eg, not happy) message were kept up on the left or right side of the screen to facilitate responding with the corresponding button press. The emotion blocks were presented before the age recognition task within a presentation type.

Fifty-three percent of participants completed the neuroimaging scan and behavioral task on the same day. For those who were not able to complete the scan and the behavioral tasks on the same day, the mean interval was 11.3 weeks (SD = 15.8). There was no difference between groups in mean interval between neuroimaging scan and task completion (F(1,77) = 1.21, P = .30).

Statistical Analyses

First, the normality of the data was tested using the one-sample Kolmogorov–Smirnov analysis. All the structural neuroimaging data were normally distributed (P's > .32). The behavioral task data were largely normally distributed. As is often the case, the accuracy data were skewed for the time-unlimited facial emotion recognition condition (P < .001) and marginally for the time-limited emotion (P = .057) and age recognition (P = .052) conditions. All the reaction time data were normally distributed.

Multivariate analyses of covariance (MANCOVA) were used to assess the effect of group on subregions of the temporal lobe, fusiform, superior temporal, middle temporal, amygdala, and hippocampus, and to control for multiple comparisons. A priori contrasts were planned for schizophrenia patients vs controls on the temporal lobe regions for comparison with previous and future research. Age, gender, and intracranial volume were included as covariates in the structural analyses. Analyses were also conducted excluding the covariates.

Mixed-model ANCOVAs were used to analyze the effect of presentation type, recognition task, and group on the behavioral task accuracy and reaction time. All the facial emotion stimuli were combined to contrast the emotion recognition performance with the age recognition control task performance to assess the specificity of facial emotion processing deficits. Additionally, ANCOVAs analyzed the accuracy and d-prime sensitivity data for individual emotion blocks. D-prime scores were used as a signal detection measure to assess a participants’ ability to distinguish between targets and distracters. Participant age was used as a covariate in all the task analyses. Analyses were also conducted excluding the participant age covariate.

Pillai’s trace test statistic was reported for the MANCOVAs. Greenhouse–Geisser correction was reported for ANCOVAs. Only significant findings on a MANCOVA or mixed-model ANCOVA were followed up with individual ANCOVAs to assess which groups differed and in which condition. Generally, testing across all 3 groups was used to guard against multiple comparisons, with the exception that a priori contrasts were planned to compare schizophrenia patients and controls on the temporal lobe regions. Partial eta-squared effect sizes (η²) were provided.

All correlations involving behavioral task accuracy were assessed using the Spearman’s ρ, due to it being more robust to violations of parametric assumptions. The remaining correlations were conducted using Pearson’s r. For the temporal lobe regions that showed group differences, a priori one-tail correlations were conducted to test the hypothesis of whether less gray matter volume was associated lower accuracy on the facial emotion compared with age recognition tasks. Fisher’s r-to-z transformation (one-sample) was used to test whether the correlations differed between diagnostic groups and whether the correlations between temporal lobe structures and emotion recognition performance were significantly greater than the correlations between temporal lobe structures and age recognition performance. Lastly, correlations assessed the relationship between temporal lobe structures and behavioral performance with symptoms (SAPS and SANS total scores) and intelligence quotient (IQ).

Results

Participants

Participant characteristics are presented in table 1. The 3 groups differed in age (F(2,83) = 4.43, P = .02), with relatives being older than patients (P = .03) and controls (P = .005). The 3 groups also differed in gender distribution (X²(2) = 9.42, P = .009), with patients having fewer females than in the relative group (X²(1) = 9.43, P = .002) and marginally fewer females than in the control group (X²(1) = 3.35, P = .07). Groups did not differ for high school completion for participants (X²(2) = 2.12, P = .35) or for their mothers (X²(2) = 0.66, P = .72). There was a trend toward a difference between groups for years of education completed (F(2,73) = 2.50, P = .09), with post hocs demonstrating schizophrenia patients having fewer years of education than controls (P = .04). Groups also differed on estimated intelligence (F(2,78) = 3.61, P = .03), with schizophrenia patients having lower IQ scores than controls (P = .03) and relatives (P = .02). Groups did not differ for handedness distributions (X²(2) = 1.93, P = .38). Relatives and controls were comparable for lifetime history of any Axis I disorder (X²(1) = 1.01, P = .32). Twenty-four of the schizophrenia patients were on antipsychotic medication, with 1 patient solely being on typical medication, 20 patients solely being on atypical medication, and 3 patients being on both types of medication. Data were missing for 1 patient, 2 relatives, and 2 controls on the face recognition tasks.

Table 1.

Participant Characteristics

	Schizophrenia	Relative	Control
N	27	23	36
Age	43.4 (10.1)a	49.8 (6.5)	42.1 (11.4)a
Gender (% female)	22a	65	44
Education (years completed)	14.3 (2.5)b	14.5 (2)	15.5 (1.9)
Education (% HS completed)	93	100	97
Maternal education (% HS completed)	82	87	89
Intelligence quotient	99.5 (12.6)a,b	109.9 (17.4)	108.2 (13.2)
Handedness (% right handed)	96	87	86
BPRS total: range	46 (13): 27–74	29 (4): 24–38c	28 (4): 24–45c
SANS total: range	26 (15): 2–47	—	—
SAPS total: range	18 (16): 1–55	—	—
Years ill: range	21 (11): 1–37	—	—
CPZ equivalence	523 (558)	—	—
Axis I (% with any lifetime diagnosis)	—	52d	39e
Relative status—Parent:sibling	—	3:20	—

Note: Data was missing from 1 patient, 2 relatives, and 2 controls for the face recognition tasks. HS, high school; estimated intelligence quotient was measured using Wechsler Adult Intelligence Scale-Third Edition Vocabulary and Block Design subtests.⁵⁵ BPRS, Brief Psychiatric Rating Scale: 24 items, scores can range from 24 to 168; SANS, Scale for Assessment of Negative Symptoms: 20 items, scores can range from 0 to 100; SAPS, Scale for Assessment of Positive Symptoms: 30 items, scores can range from 0 to 150. CPZ, chlorpromazine equivalence for antipsychotic medications.

^aLess than relatives.

^bLess than controls.

^cLess than patients.

^dRelatives had a lifetime history of the following Axis I disorders (some relatives has more than one disorder): depressive disorder (5 participants), posttraumatic stress disorder (1 participant), cocaine dependence (1 participant), alcohol abuse (8 participants), cannabis abuse (1 participant), and stimulant abuse (1 participant).

^eControls had a lifetime history of the following Axis I (some controls has more than one disorder): depressive disorder (5 participants), anxiety disorder (6 participants), bulimia (1 participant), attention-deficit hyperactivity disorder (1 participant), cocaine dependence (1 participant), alcohol dependence (6 participants), alcohol abuse (5 participants), cannabis abuse (2 participants), and stimulant abuse (1 participant).

Analysis of Temporal Lobe Structures

Main Effects of Age and Gender.

Collapsing across groups, in many of the temporal lobe regions, there was a relationship between greater participant age and less gray matter (r’s = −.54 to −.08, P’s < .001–.45). There was an overall effect of gender across groups in the temporal lobe regions (F(10,71) = 2.69, P = .007), with males having larger volumes. There was no interaction between gender and group. Given these effects, participant age and gender were entered as covariates in subsequent analyses. An additional covariate of intracranial volume was also entered. To test the effects of the covariates on the results, analyses were also conducted excluding covariates. As the results were largely similar across the 2 approaches, the results from the model including covariates are reported. Mean and SDs for both the raw and adjusted values are given in table 2. for reference.

Table 2.

Data Values for Temporal Lobe Grey Matter Regions

Regions	Schizophrenia, N = 27		Relative, N = 23		Control, N = 36
	Adjusted	Raw	Adjusted	Raw	Adjusted	Raw
Left fusiform	8587 (1313)	8765 (1448)	9028 (1362)	8827 (1397)	9668 (1284)	9663 (1238)
Right fusiform	8809 (1229)	8939 (1363)	8899 (1275)	8730 (1245)	9017 (1202)	9027 (1210)
Left middle temporal	10 745 (1505)	11 005 (1761)	11 827 (1561)	11 311 (1552)	11 692 (1472)	11 826 (2170)
Right middle temporal	12 375 (1547)	12 652 (2451)	13 129 (1605)	12 582 (1797)	12 905 (1513)	13 047 (1917)
Left superior temporal	12 195 (1274)	12 458 (1631)	12 766 (1321)	12 153 (1602)	12 897 (1245)	13 091 (1844)
Right superior temporal	11 638 (1146)	11 802 (1753)	11 973 (1189)	11 482 (1707)	12 008 (1121)	12 199 (1679)
Left amygdala	1619 (236)	1645 (249)	1677 (244)	1642 (184)	1643 (230)	1646 (273)
Right amygdala	1608 (238)	1634 (249)	1673 (247)	1633 (221)	1658 (233)	1664 (248)
Left hippocampus	3874 (328)	3931 (385)	4306 (341)	4191 (401)	4218 (321)	4249 (491)
Right hippocampus	4113 (366)	4170 (554)	4369 (379)	4273 (413)	4225 (358)	4243 (417)

Note: Adjusted: Means and SDs for gray matter cortical and subcortical volumes are adjusted for age, gender, and intracranial volume. Raw: Original mean and SDs for gray matter cortical and subcortical volumes.

Effects of Group.

The MANCOVA of the 10 brain structures (5 regions, 2 hemispheres) demonstrated an effect of group (F(20,144) = 2.17, P = .005, η² = 0.23) and a temporal lobe region by hemisphere by group interaction (F(8,156) = 2.12, P = .04, η² = 0.10). Given these significant effects, 5 follow-up mixed-model ANCOVAs (2 hemispheres × 3 groups) were conducted for the temporal lobe regions. The statistics for these models are reported in table 3.

Table 3.

ANCOVA Results for the Individual Temporal Lobe Regions

	F Value	df	P Value	Effect Size
Fusiform region
Overall 3 group ANCOVA
Main effect of group	2.86	2, 80	.06	0.07
Group × hemisphere interaction	3.75	2, 80	.03	0.09
Schizophrenia vs control ANCOVA
Main effect of group	4.83	1, 58	.03	0.08
Group × hemisphere interaction	6.62	1, 58	.01	0.10
Relative vs control ANCOVA
Main effect of group	4.49	1, 54	.04	0.08
Group × hemisphere interaction	0.95	1, 54	.33	0.02
Schizophrenia vs relative ANCOVA
Main effect of group	2.10	1, 45	.15	0.05
Group × hemisphere interaction	0.05	1, 45	.82	0.001
Middle temporal lobe
Overall 3 group ANCOVA
Main effect of group	3.46	2, 80	.04	0.08
Group × hemisphere interaction	0.58	2, 80	.56	0.01
Schizophrenia vs control ANCOVA
Main effect of group	4.42	1, 58	.04	0.07
Group × hemisphere interaction	1.20	1, 58	.28	0.02
Relative vs. control ANCOVA
Main effect of group	<0.001	1, 54	.99	<0.001
Group × hemisphere interaction	0.05	1, 54	.82	0.001
Schizophrenia vs relative ANCOVA
Main effect of group	11.02	1, 45	.002	0.20
Group × hemisphere interaction	0.002	1, 45	.96	<0.001
Hippocampus
Overall 3 group ANCOVA
Main effect of group	7.21	2, 80	.001	0.15
Group × hemisphere interaction	5.27	2, 80	.007	0.12
Schizophrenia vs control ANCOVA
Main effect of group	6.43	1, 58	.01	0.10
Group × hemisphere interaction	8.87	1, 58	.004	0.13
Relative vs control ANCOVA
Main effect of group	0.57	1, 54	.45	0.01
Group × hemisphere interaction	0.15	1, 54	.70	0.003
Schizophrenia vs relative ANCOVA
Main effect of group	18.94	1, 45	<.001	0.30
Group × hemisphere interaction	1.42	1, 45	.24	0.03
Superior temporal lobe
Overall 3 group ANCOVA
Main effect of group	2.03	2, 80	.14	0.05
Group × hemisphere interaction	0.76	2, 80	.41	0.02
Schizophrenia vs control ANCOVA
Main effect of group	3.17	1, 58	.08	0.05
Group × hemisphere interaction	1.75	1, 58	.19	0.03
Amygdala
Overall 3 group ANCOVA
Main effect of group	0.45	2, 80	.64	0.01
Group × hemisphere interaction	0.24	2, 80	.79	0.006
Schizophrenia vs control ANCOVA
Main effect of group	0.38	1, 58	.54	0.007
Group × hemisphere interaction	0.30	1, 58	.58	0.005

Note: ANCOVA of temporal lobe regions, including age, gender, and intracranial volume as covariates. Effect size reported is partial eta-squared (η²).

The overall 3 group × 2 hemisphere ANCOVA for the bilateral fusiform region revealed a trend for a main effect of group and an overall group by hemisphere interaction with the left hemisphere showing a stronger effect than the right hemisphere (see figure 1). Specific group comparison of schizophrenia patients and controls revealed a main effect of group, with schizophrenia patients demonstrating less bilateral fusiform gray matter than controls (see table 3). In addition, in patients compared with controls, a hemisphere by group interaction was found, with the left hemisphere being more affected than the right hemisphere in schizophrenia patients. Specific comparison of relatives compared with controls demonstrated a main effect of group with relatives showing less bilateral fusiform gray matter compared with controls. This finding suggests an association between less fusiform gray matter volume and the genetic liability for the disorder. Furthermore, there were no differences in fusiform gray matter between patients and relatives.

Fig. 1.

Left and Right Fusiform Grey Matter Volumes Adjusted for Age, Gender, and Intracranial Volume.

The overall 3 group × 2 hemisphere ANCOVA for the bilateral middle temporal region revealed a main effect of group. Specific group comparisons revealed a main effect of group, with schizophrenia patients having less bilateral middle temporal gray matter than controls and relatives. There were no differences in the middle temporal region between relatives and controls.

The overall 3 group × 2 hemisphere ANCOVA revealed a main effect of group for the bilateral hippocampal volume and a hemisphere by group interaction, with the left hemisphere showing a stronger effect than the right hemisphere. Specific group comparisons revealed a main effect of group, with schizophrenia patients demonstrating smaller bilateral hippocampal volumes compared with controls. In addition, in patients compared with controls, a hemisphere by group interaction was found, with the left hemisphere being more affected than the right hemisphere in schizophrenia patients. Comparison of schizophrenia patient and relatives demonstrated a main effect of group with schizophrenia patients demonstrating smaller bilateral hippocampal volumes than relatives. There were no differences in hippocampal volumes between relatives and controls.

No differences were found in the overall ANCOVA comparing the 3 groups on the superior temporal lobe and amygdala gray matter volumes. A priori comparisons of schizophrenia patients and controls demonstrated no significant main effects of group or group by hemisphere interactions.

In summary, genetic liability effects were found in the fusiform region, particularly the left hemisphere, with both patients and relatives showing significantly less gray matter volume than controls. Patients additionally demonstrated smaller bilateral hippocampal and middle temporal regions than controls and relatives. No effects of group or interactions with group were found for either the superior temporal or amygdala volumes.

Performance on the Facial Emotion Recognition and Age Recognition Tasks

Main Effects of Age and Gender.

Across groups, there were relationships between greater participant age and lower accuracy for judging facial emotions presented at a time-unlimited rate (Spearman’s ρ = −.30, P = .006), whereas the association was less prominent for the facial emotions presented at a time-limited rate (Spearman’s ρ = −.19, P = .09) or the time-limited and time-unlimited age recognition conditions (Spearman’s ρ < −.06, P’s > .57). Across groups and recognition conditions, there was no effect of gender (F(10,66) = 0.73, P = .70) and no interaction of gender with group. More specifically, no individual emotion or age recognition condition across presentation types demonstrated a significant relationship with gender. Given these effects, participant age was entered as a covariate. The main analyses were also conducted excluding participant age as a covariate. As the results were largely similar, only the results including the participant age covariate are reported. Mean and SDs for both the raw and adjusted values are given in table 4 for reference.

Table 4.

Accuracy Values (Percentage Correct) for Facial Emotion Recognition and Age Recognition Tasks

	Schizophrenia, N = 26		Relative, N = 21		Control, N = 34
	Adjusted	Raw	Adjusted	Raw	Adjusted	Raw
Time-limited
Anger	79 (10)	79 (12)	88 (10)	87 (8)	86 (9)	86 (8)
Fear	76 (10)	77 (13)	87 (10)	85 (9)	84 (10)	84 (8)
Happy	85 (10)	85 (14)	92 (10)	91 (10)	95 (10)	95 (5)
Sad	75 (14)	76 (17)	88 (14)	85 (14)	85 (14)	86 (12)
Age	76 (9)	77 (14)	85 (9)	84 (5)	83 (9)	84 (6)
Time-unlimited
Anger	78 (12)	78 (18)	90 (12)	89 (8)	88 (12)	88 (7)
Fear	78 (11)	79 (17)	91 (11)	90 (5)	86 (11)	87 (8)
Happy	88 (9)	88 (13)	95 (9)	94 (7)	93 (9)	94 (5)
Sad	78 (11)	78 (15)	91 (11)	90 (8)	87 (11)	88 (9)
Age	82 (6)	82 (7)	84 (6)	83 (6)	84 (6)	84 (5)

Note: Adjusted: Means and SDs are adjusted for age. Raw: Original mean and SDs.

Specific Deficit in Facial Emotion Recognition.

A presentation type (time-limited, time-unlimited) by recognition tasks (emotion recognition, age recognition) by group (schizophrenia patients, relatives, and controls) mixed-model ANCOVA was conducted to determine whether certain groups demonstrated a specific deficit in facial emotion recognition (collapsing across emotion blocks) and how that interacted with presentation type. Follow-up analyses were conducted for significant effects as necessary to determine group differences and the nature of the impairment.

There was a main effect of recognition task (F(1,77) = 4.86, P = .03, η² = 0.06), with participants being more accurate on emotion recognition than age recognition. There was also a main effect of group (F(2,77) = 14.27, P < .001, η² = 0.27) and a trend toward an interaction between group and recognition condition (F(2,77) = 2.49, P = .09, η² = 0.06). There was no main effect of presentation type or interaction between presentation type and group.

Most importantly, there was a 3-way interaction between presentation type, recognition task, and group (F(2,77) = 3.31, P = .04, η² = 0.08; see figure 2). Table 5 presents the statistics for the follow-up ANCOVAs comparing the groups on the different task conditions. In the time-limited presentation condition, schizophrenia patients were less accurate than controls and relatives on both the facial emotion recognition task and age recognition task. In contrast, on the time-unlimited presentation condition, schizophrenia patients were less accurate than controls and relatives on the emotion recognition task but not the more difficult age recognition task. There was no difference between controls and relatives in either timing condition, though relatives tended to have a higher accuracy in the time-unlimited emotion condition than controls.

Fig. 2.

Specific Deficit in Facial Emotion Recognition Compared with Age Recognition in Schizophrenia Patients. Accuracy values are adjusted for participant age.

Table 5.

ANCOVA Results for the Facial Emotion Recognition and Age Recognition Task Accuracy

	F Value	df	P Value	Effect Size
Time-limited condition
Emotion recognition
Overall 3 group ANCOVA	10.57	2, 77	<.001	0.22
Schizophrenia vs control ANCOVA	15.31	1, 57	<.001	0.21
Relative vs control ANCOVA	0.02	1, 52	.90	<0.001
Schizophrenia vs relative ANCOVA	14.09	1, 44	.001	0.24
Age recognition
Overall 3 group ANCOVA	6.12	2, 77	.003	0.14
Schizophrenia vs control ANCOVA	7.34	1, 57	.009	0.11
Relative vs control ANCOVA	0.06	1, 52	.81	0.001
Schizophrenia vs relative ANCOVA	6.96	1, 44	.01	0.14
Time-unlimited condition
Emotion recognition
Overall 3 group ANCOVA	10.77	2, 77	<.001	0.22
Schizophrenia vs control ANCOVA	10.41	1, 57	.002	0.15
Relative vs control ANCOVA	3.04	1, 52	.09	0.06
Schizophrenia vs relative ANCOVA	12.77	1, 44	.001	0.23
Age recognition
Overall 3 group ANCOVA	1.11	2, 77	.34	0.03
Schizophrenia vs control ANCOVA	2.15	1, 57	.15	0.04
Relative vs control ANCOVA	0.10	1, 52	.76	0.002
Schizophrenia vs relative ANCOVA	0.47	1, 44	.50	0.01

Note: ANCOVA of facial emotion recognition and age recognition accuracy, including participant age as a covariate. Effect size reported is partial eta-squared (η²).

Three sets of analyses were performed to scrutinize this finding. Follow-up 2 presentation type × 2 group ANCOVAs compared the specific groups on the 2 presentation types for facial emotion recognition and age recognition accuracy. Schizophrenia patients showed a tendency toward greater improvement in age recognition from time-limited to unlimited viewing periods compared with controls (F(1,57) = 3.23, P = .08, η² = 0.05) and relatives (F(1,44) = 5.51, P = .02, η² = 0.11). Unlimited viewing time failed to benefit the schizophrenia patients more than controls or relatives during emotion recognition, but relatives improved their emotion recognition with unlimited time compared with controls (F(1,52) = 4.22, P = .05, η² = 0.08). Analyses were also conducted excluding the 5 schizoaffective patients as affective symptoms may influence facial emotion recognition. All statistics were similar, and interpretations were consistent with the analyses including patients with schizoaffective disorder. Finally, to further determine whether reaction time differences were responsible for the reduced accuracy (ie, speed-accuracy trade-offs) in schizophrenia patients in the time-limited condition vs time-unlimited condition, ANCOVAs were conducted. There was no differential effect of presentation type on facial emotion recognition (η² = 0.02) or on age recognition (η² = 0.01) reaction time across groups.

In summary, findings in schizophrenia patients were suggestive of a specific deficit in emotion recognition because patients were able to improve their age recognition performance with unlimited viewing time but not their emotion recognition performance compared with control and relatives. The specificity of the emotion recognition impairment in schizophrenia patients was most apparent when there was an indefinite amount of time to appraise facial stimuli. With time constraints for responding to facial stimuli, schizophrenia patients exhibited deficits in both emotion recognition and age recognition suggesting a more general impairment face perception under this condition. Relatives generally demonstrated intact performance on emotion recognition and age recognition tasks.

Differential Effect of Emotion Block.

A 2 presentation type × 4 emotion block (anger, fear, happy, and sad) × 3 group mixed-model ANCOVA investigated whether a selective impairment for a specific facial emotion was present in the schizophrenia patients or relatives (see table 4). There was no main effect of presentation type, emotion block, or interaction with group or interaction between presentation type, emotion block, and group (P’s > .33, η²’s < 0.03) on accuracy. As captured in previous analyses, there was a main effect of group (F(2,77) = 11.85, P < .001, η² = 0.24). Schizophrenia patients had lower accuracy for all emotion blocks at both presentation types compared with controls (P’s < .02, η² = 0.09–0.22) and relatives (P’s < .03, η² = 0.10–0.21). The effects were similar when patients with schizoaffective disorder were excluded. Comparison between relatives and controls revealed a more nuanced pattern; relatives tended to be more accurate in the fear emotion block during the time-unlimited presentation type (F(1,52) = 3.99, P = .05, η² = 0.07) and less accurate during the happy emotion block during the time-limited presentation type (F(1,52) = 3.59, P = .06, η² = 0.07).

To further explore behavioral performance between groups, a 2 presentation type × 4 emotion block × 3 group mixed-model ANCOVA was conducted on d-prime scores. D-prime scores were used as a signal detection measure to assess a participants’ ability to distinguish between targets and distracters, with higher d-prime scores suggested better sensitivity. There was a significant presentation type by emotion block by group interaction on the d-prime scores (F(6,215) = 2.22, P = .05, η² = 0.06). Given the significant 3-way interaction, 2 presentation type × 3 group ANCOVAs were conducted on each emotion block to determine whether a particular emotion was specifically affected. Only the blocks with happy expressions as target stimuli demonstrated an overall significant presentation type by group interaction (F(2,77) = 6.64, P = .002, η² = 0.15). Specific comparison of patients and controls (2 presentation type × 2 group ANCOVA) demonstrated that schizophrenia patients had poorer sensitivity compared with controls in the time-limited condition, but not the time-unlimited condition (F(1,57) = 7.70, P = .007, η² = 0.12). Compared with controls, relatives had lower d-prime scores in the time-limited condition, but higher d-prime scores in the time-unlimited condition (F(1,52) = 13.07, P = .001, η² = 0.20). There was no interaction between presentation type and group for schizophrenia patients and relatives (η² = 0.002) because patients had worse performance regardless of presentation type (F(1,52) = 4.84, P = .03, η² = 0.10).

Associations between Temporal Lobe Structures, Task Performance, Symptoms, and IQ

First, fusiform, hippocampus, and middle temporal gray matter volumes were combined across hemispheres to reduce the number of comparisons because all these regions demonstrated main effects of group on the total volume. However, because the left fusiform region was of particular interest, correlations between this region and behavioral task conditions were also evaluated. In schizophrenia patients, greater bilateral fusiform (Spearman’s ρ = .41, P = .04) and middle temporal (Spearman’s ρ = .56, P = .003) gray matter volumes were associated with greater accuracy on the facial emotion recognition task during the time-limited presentation type (see figure 3). There were significant correlations between the left fusiform volume and the time-limited (Spearman’s r = .34, P = .05) and time-unlimited (Spearman’s r = .34, P = .05) emotion recognition conditions. These relationships did not exist in within the other groups and were not present for the age recognition conditions.

Fig. 3.

Relationship between Fusiform and Middle Temporal Grey Matter Volumes and Facial Emotion Recognition Accuracy in Schizophrenia Patients.

Second, Fisher’s r-to-z transformations were used to test for differences between correlations between groups and task conditions. The correlation between the fusiform region (z = 1.66, P = .04; left fusiform z = 1.75, P = .04) and middle temporal region (z = 1.86, P = .03) and time-limited facial emotion recognition were greater in schizophrenia patients than in controls. Furthermore, in schizophrenia patients, the correlation for the middle temporal region with facial emotion recognition was greater than the correlation between the middle temporal region and age recognition (z = 2.35, P = .009). Neither the combined fusiform volume nor the left fusiform volume specifically differed in its correlation between task conditions.

Third, there were no noteworthy correlations between positive and negative symptoms and temporal lobe brain regions or behavioral task performance. In schizophrenia patients, IQ was related to left fusiform gray matter volume (Pearson’s r = .48, P = .02). In controls, IQ was related to facial emotion recognition (Spearman’s ρ = .43, P = .01) and age recognition (Spearman’s ρ = .37, P = .03) during the time-limited condition.

Discussion

Given the relationship between emotion recognition and community and social functioning, an understanding of the neural underpinnings of emotion recognition is crucial (for review, see Couture et al⁴⁷). This study investigated the volume of temporal lobe gray matter regions, including the fusiform face perception area, and facial emotion recognition deficits in schizophrenia patients and first-degree nonpsychotic relatives. Fusiform volume reductions were evident in schizophrenia patients and appeared related to the genetic (familial) liability for the disorder because similar reductions were observed in nonpsychotic relatives. Additionally, reduced fusiform and middle temporal gray matter volumes were associated with facial emotion recognition deficits in schizophrenia patients. Middle temporal gray matter volume failed to be associated with age recognition, suggesting that sparse gray matter in the region may be specifically involved with impaired facial emotion recognition in schizophrenia.

Results of the present study are consistent with genetic liability most strongly affecting the temporal lobe, with both patient and relatives showing less gray matter volume in primarily the left fusiform region than controls. A number of studies have found smaller bilateral fusiform volumes in both first-episode⁴⁸ and chronic schizophrenia.^35,36 Similar to our findings, a review of 15 voxel-based morphometry studies showed that approximately a quarter of the studies found abnormalities in the left fusiform, whereas only one study found abnormalities in the right fusiform.¹³ The findings of the present study appear to be the first observation that reduced fusiform gray matter volume may reflect unexpressed genetic liability for schizophrenia.

The additional finding of less gray matter volume in the hippocampi of schizophrenia patients concurs with replicable evidence of hippocampal volume loss in schizophrenia patients.⁴⁹ Also, the middle temporal gyrus has found to be reduced in drug-naïve patients⁵⁰ and first-episode patients.¹⁴ Although hippocampal volumes⁵¹ and middle temporal¹⁷ volumes have been found to be abnormal in relatives of patients, this was not evident in the present sample of relatives using this specific methodology.

Results provide evidence for a specific deficit of schizophrenia patients in facial emotion recognition that is independent of speed of processing facial percepts. With an unlimited response time, patients were able to improve their performance on facial age recognition but not emotion recognition. This effect was unrelated to reaction times differences because there were no main effects of or interaction with group status on speed of response. This task is a modification of Schneider and colleagues²⁰ paradigm where the faces were presented as in the time-limited condition of this study. Schneider and colleagues²⁰ found a specific deficit for facial emotion recognition compared with age recognition and recognition memory in patients compared with controls. Specific deficits in emotion recognition have also been found after controlling for general facial perceptual difficulties.^31,32 These results extend previous research suggesting facial emotion recognition may be an important aspect of the disorder or associated neural abnormalities because patients cannot improve their emotion recognition deficits with additional processing time, though they can improve their general facial perception.

The finding of a specific facial emotion recognition deficit in schizophrenia carries added significance given that it was associated with regional reductions in temporal lobe gray matter. The fusiform has intricate connections with the medial temporal lobe regions with neural pathways being largely left lateralized. These neural connections may be particularly important for the recognition, recall, and processing of emotionally expressive faces.⁷ Perhaps the combined abnormalities of left fusiform, hippocampus, and middle temporal lobe underlie facial emotion recognition deficits in schizophrenia patients. Indeed, a meta-analysis of functional neuroimaging studies during emotion recognition found that schizophrenia patients had less activity in the bilateral fusiform and parahippocampus/amygdala.⁵² In the present study, smaller bilateral fusiform and middle temporal regions were associated with greater impairments in facial emotion recognition in the time-limited condition. For the middle temporal lobe, this relationship was specific to emotion recognition and absent for more general aspects of face perception. Therefore, middle temporal gray matter volume appears to correspond with the capacity to more quickly integrate emotionally relevant facial features.

In the current sample of nonpsychotic relatives, we generally failed to find evidence for deficits in facial emotion recognition, with the exception of lower d-prime scores for the happy condition during the time-limited presentation type. Other studies have indicated that relatives have intermediate abnormalities in multiple emotion recognition domains falling between patient and control samples,^29,53 although not all studies have found impairments.⁵⁴ More complex emotion recognition tasks that involve accurately labeling the expressed facial emotion or using subtle stimuli may increase sensitivity to behavioral deficits in relatives. Interestingly in this study, we found that although patients failed to improve their performance with unlimited time, relatives were able to increase their performance under the self-paced condition to be better than controls, especially in the fear and happy condition. This may reflect compensatory neural and behavioral mechanisms.

Limitations include that our sample size was modest; however, effect sizes were large enough to result in significant findings. Also, the relative sample was somewhat older than the patient and control sample; however, there was a great deal of overlap in the age distributions and results were significant with and without the inclusion of the participant age covariate. This study used a chronic, middle-aged schizophrenia sample as opposed to a first-episode sample, which may have made it more difficult to detect associations with symptoms. However, we found an association with task performance and structure. One possible explanation is that cognitive tasks may be more closely related to discrete brain regions than symptoms, hence increasing our power to detect an association. Lastly, we used an automated methodology to measure the temporal lobe volumes. There are limitations associated with automated labeling of magnetic resonance imaging data, but we used a well-validated program (Freesurfer) and diligently followed guidelines provided to ensure production of topologically accurate surfaces and parcellations.⁴⁵ Additionally, using an automated methodology may increase the potential for reliability by removing the potential for human bias.

In summary, only schizophrenia patients exhibited a specific deficit in emotion recognition not attributable to a generalized impairment in face perception. Less gray matter in the fusiform gyrus, a core region of the facial recognition system, was related to the genetic liability for the disorder. Nonpsychotic relatives of patients have an increased genetic vulnerability and an intermediate level of structural neural abnormalities; however, they often do not present behavioral abnormalities. Therefore, family members may be an ideal sample to investigate successful compensatory neural and behavioral mechanisms that could be the focus of novel pharmacological and psychosocial interventions.

Funding

Thesis Research Grant from the University of Minnesota, Postgraduate Scholarship Doctoral Award from the Natural Sciences and Engineering Research Council of Canada, and the University of Calgary Start-up Grant to V.M.G.; Clinical Science Merit Review Program Grant of the Department of Veterans Affairs to S.R.S.; National Institutes of Health (R24 MH069675 to S.R.S.; R21 MH079262 to A.W.M.; P41 RR008079, P30 NS057091 to the Center for Magnetic Resonance Imaging at the University of Minnesota).