Configural processing in face recognition in schizophrenia

Abstract

Introduction. There is currently substantial literature to suggest that patients with schizophrenia are impaired on many face-processing tasks. This study investigated the specific effects of configural changes on face recognition in groups of schizophrenia patients. Methods. In Experiment 1, participants identified facial expressions in upright faces and in faces inverted from their upright orientation. Experiments 2 and 3 examined recognition memory for faces and other non-face objects presented in upright and inverted orientations. Experiment 4 explored recognition of facial identity in composite images where the top half of one face was fused to the bottom half of another face to form a new face configuration. Results. In each experiment, the configural change had the same effect on face recognition for the schizophenia patients as it did for control participants. Recognising inverted faces was more difficult than recognising upright faces, with a disproportionate effect of inversion on faces relative to other objects. Recognition of facial identity in face-halves was interfered with by the formation of a new face configuration. Conclusion. Collectively, these results suggest that people with schizophrenia rely on configural information to recognise photographs of faces.

Faces provide essential cues that guide our interpersonal communication and behaviour. Deficits in identifying a facial expression or a person’s identity can have dramatic effects on an individual’s social interactions. There is a substantial literature showing that people with schizophrenia are impaired on several facial-processing tasks. These individuals have difficulty judging and labelling facial expressions (e.g., Archer, Hay, & Young, 1994; Berndl, von Cranach, & Grusser, 1986; Cutting, 1981; Walker, Marwit, & Emory, 1980; Walker, McGuire, & Bettes 1984; see Mandal, Pandey, & Prasad, 1998 and Morrison, Bellack, & Mueser, 1988 for reviews). They are also impaired on tests of perceptual face matching, recognition memory for unfamiliar faces, and recognition of facial identity (Archer, Hay, & Young, 1992; Feinberg, Rifkin, Schaffer, & Walker, 1986; Gruzelier et al., 1999; Hellewell, Connell, & Deakin, 1994; Kerr & Neale, 1993). Because people with schizophrenia often exhibit poor interpersonal and social skills, it is important to understand the basis of their face recognition deficits.

This work addressed the possibility that facial-processing deficits in individuals with schizophrenia are due to impairments in processing information about the configuration of faces. According to findings in the face perception literature, faces are perceived as “gestalts” or as wholes and not simply as a collection of isolated features such as the eyes, nose, or mouth. An expression often used to make this point is that the whole does not equal the sum of its arts. Studies reveal that faces are represented by discrete facial features and the configuration formed by these features (Rhodes, 1988; Sergent, 1984). The configuration of a face refers to the spatial relations among internal facial features. For example, the distance between the eyes, chin contour, or interactive properties of different facial features convey configural information.

One line of evidence that shows the importance of configural processing for face recognition comes from studies examining the effects of inverting faces (i.e., rotating faces 180°) from their upright orientation. The finding that face recognition is more impaired by this inversion procedure than is recognition for other non-face objects has been demonstrated under a variety of conditions (Valentine, 1988). One explanation for the pronounced inversion effects in face recognition is that upright faces are analysed in terms of both their parts and configural properties, whereas inverted faces are analysed only in terms of their parts. The idea is that it is difficult to encode information about the configuration of inverted faces because inverting the face disrupts the spatial arrangement of its internal features. These inversion effects suggest that memory representations include information about the global structure of a face, namely, the facial gestalt or configuration.

Several investigators argue that perceptual deficits emerge in people with schizophrenia in tasks that require gestalt, or holistic, processing. The claim is that these individuals perceive objects or scenes in a fragmented or piecemeal way. Cutting (1989) provides several accounts of how Gestalt theory is applied to symptoms of schizophrenia. In one account of delusional perception, Matussek (1987; cited in Cutting, 1989) described a perceptual distortion in which elements in the environment appear to be “lifted out of the remainder of the context and stand out” (Cutting, 1989, p. 431). This attention to detail limits the ability to perceive the overall structure of patterns. Along these same lines, empirical studies indicate that schizophrenia patients have difficulty engaging top-down processes to perceptually organise or integrate visual information (Cox & Leventhal, 1978; John & Helmsley, 1992; Knight, 1992; Place & Gilmore, 1980; Silverstein et al., 1996; Silverstein, Bakshi, Chapman, & Nowlis, 1998). It is suggested that impairments in perceptual grouping result from patients’ inability to form new representations in which discrete elements are combined into wholes (Knight & Silverstein, 1998; Silverstein et al., 1998).

Given such deficits, it is possible that people with schizophrenia are unable to integrate features of the face to form a cohesive whole. Visual analysis may be directed at discrete facial features at the expense of perceiving the face as a whole. Several findings in the literature support this possibility. Mandal and Palchoudhury (1989) observed that schizophrenia patients were impaired in making judgements about facial expression (e.g., sadness or fear) when the entire face was presented but not when segments of a face (e.g., upper segment of eyes, cheeks, and nose) were shown. These results suggest that patients relied only on parts of a face to identify expression. In another study, Frith et al. (1983) used a task in which faces and non-face objects could be sorted on the basis of their component features. They found that participants in their control group sorted faces more accurately than objects, whereas patients with schizophrenia sorted aces similarly to objects. These investigators suggested that control participants sorted faces more accurately than objects because the holistic quality of faces allowed for the integration of multiple facial features simultaneously. Schizophrenia patients did not appear to benefit from the holistic quality of faces, as evidenced by their similar sorting of faces and objects. In a third study, Grusser, Kirchhoff, and Naumann (1990) examined the effects of inverting stimuli on recognition memory for faces with emotional expressions. Schizophrenia patients were generally impaired for recognising upright faces but were not impaired for inverted faces. Grusser et al. (1990) proposed that the patients’ recognition of inverted faces was normalised because inverting the face diminished its emotional quality and "faceness". Another interpretation is that patients were unimpaired in recognising inverted faces because inversion of a face promotes analysis of component but not configural features. Together, these findings point to the possibility that schizophrenia patients are impaired in using configural information to recognise faces.

In this series of experiments, we sought to determine whether people with schizophrenia were sensitive to configural changes in faces when making three types of judgements: identifying facial expression, recognising prior occurrence of unfamiliar faces, and recognising facial identity. According to theoretical models of face recognition, distinct types of processes mediate these different judgements of face recognition (Bruce & Young, 1986). Identifying expressions, identifying personal identity, and recognising recently seen faces involve the interaction of different functional components of the face recognition system. Yet perception of the face configuration has been shown to be important for each of these recognition judgements. If people with schizophrenia have deficits in processing configural information, we would expect to see these deficits across different facial processing tasks.

In each of the following experiments, we manipulated a variable that is assumed to distort information about the configuration of a face (e.g., inverting a face) and examined its effects on recognition. In the case of inverting faces, we reasoned that if schizophrenia patients’ recognition is dependent on configural information, then their recognition would be disrupted by presenting a face upside-down. If schizophrenia patients do not rely on information about the configuration, their performance would be less affected, or not affected at all, by manipulations that limit processing of this type of information. For instance, their recognition of upright faces might not differ from their recognition of inverted faces. The first experiment examines the effects of inverting faces on identification of facial expression.

EXPERIMENT 1

The arrangement of facial features and their spatial relations among one another contribute to the emergence of facial expression. Ekman, Friesen, and Ellsworth (1972) proposed that parts of the face and different facial features interact to produce emotional expression. This claim is supported by findings showing that participants identify expressions by perceiving a combination of facial features. McKelvie (1973) found that movement of different parts of the face, such as simultaneous movement of the brow and mouth, influenced the ability to label a facial emotion more than did movement of individual features alone. Calder, Young, Keane, and Dean (2000) showed that participants relied on configural information to recognise facial expressions, using composite images of different facial expressions. They developed composite expressions by aligning the top half of one facial expression (e.g., fear) with the bottom half of another facial expression (e.g., happiness). Participants were slower to identify expressions in the top and bottom halves of composite images compared to a condition in which face halves were misaligned (non-composites). Calder et al. (2000) suggest that expressions are more difficult to identify from composite images because the composite formed a new emotional expression that was inconsistent with, and therefore interfered with, perception of the expression in the top and bottom segments. These results suggest that people recognise facial expressions by processing the face as a whole, not by detecting single facial features.

As described earlier, people are notoriously poor at recognising upside-down faces. Perception of facial expression is also sensitive to changes in orientation. An expression that is apparent in an upright face can be difficult to discriminate in an inverted face (e.g., Thompson, 1980). Experiment 1 compared identification of facial expressions in upright and inverted faces. It is expected that accuracy will be higher for upright than for inverted faces, because inverting faces will, in part, attenuate the ability to encode configural information. However, if schizophrenia patients identify facial expression by analysing individual features, their performance might not differ for upright and inverted faces.

Method

Participants. There were 16 patients with a diagnosis of schizophrenia (15 males, 1 female) and 10 non-psychiatric control participants (7 males, 3 females). These individuals participated in an earlier study of visual scanning in schizophrenia (Schwartz, Rosse, Johri, & Deutsch, 1999). The patients met Diagnostic and Statistical Manual IV (DSM-IV) (American Psychiatric Association, 1994) criteria for schizophrenia. The diagnosis was based on a semi-structured interview with the patient performed by a psychiatrist (RBR) and a review of the patient’s chart. All patients had chronic courses with multiple psychiatric hospitalisations. During the clinical interview, the severity of the patients’ current symptoms was assessed using the Brief Psychiatric Rating Scale (BPRS; Overall & Gorham, 1962). A rating for each item on the BPRS was determined in consensus by two of the authors who have established reliability. The mean BPRS score for the group was 54.4 (range: 30-73). Fourteen patients were recruited from an inpatient psychiatry ward and two from an outpatient partial hospitalisation programme. The two outpatients also presented with psychotic symptoms (hallucinations and/or delusions). Patients were screened for medical and neurological problems. Any patient with a history of substance dependence or who met criteria for alcohol or drug abuse within the 6 months prior to the study was excluded. All patients were treated with neuroleptic medications at the time of testing.

Control participants were paid volunteers who were recruited from the Washington DC metropolitan area and from the hospital staff The mean age for participants in the control group did not differ reliably from the mean age of those in the patient group, t(24) = 1.85, p > .05. However, the mean educational level of control participants was significantly higher than that of the patients, t(24) =-2.33, p < .05. Control participants were screened to exclude people with medical or psychiatric problems. All participants in Experiments 1-4 signed an informed consent prior to their participation in these studies. Table 1 shows characteristics of the participants in all four experiments.

TABLE 1.

Demographic features for patients and control participants in Experiments 1, 2, 3, and 4: Means and standard deviations

Group	Age (years)	Education (years)	Duration of illness (years)	BPRSa
Exlperiment 1
Control	35.2	14.5	-	-
	9.9	2.9
Schizophrenia	40.8	12.6	15.56	54.4
	5.4	1.4	5.6	11.4
Experimient 2
Control	38.3	13.9	-	-
	8.8	1.8
Schizophrenia	41.5	13.3	16.2	48
	5.9	1.3	8	12.8
Schizophreniab	45.6	13.2	15.7	58.2
	5.58	1.7	7.2	9.3
Experimient 3
Non-psycliiatric control	42.5	12.7	-	-
	9.6	0.9
Clinical control	45.6	12.7	13.2	42.4
	7.1	1.6	12.1	6.4
Schizophrenia	44.3	13.6	21	52.4
	8.5	2.1	9.3	8.8
Experimentt 4
Control	39.6	13.7	-	-
	10.9	2.1
Schizophreriia	39.6	13.1	16.32	49.56
	7.1	1.4	7.54	10.1

^aBrief Psychiatric Rating Scale

^bPatients with acute exacerbation (N =10).

Materials. The stimuli consisted of 64 faces from the Ekman and Friesen (1976) slide collection. Critical items were composed of eight faces from each of seven types of facial expression: happy, sad, surprise, disgust, anger, fear, and neutral. Eight additional faces from this collection were selected randomly to serve as buffer items.

Design. The design was a 2 × 2 (group × orientation) mixed factorial, with group (control and patient) as a between-subjects variable and orientation (upright and inverted) as a within-subjects variable.

Procedure. Each participant was tested individually. They were shown 32 upright and 32 inverted faces. The orientation condition was blocked and presented in a fixed order such that upright faces were presented before inverted faces. The first two and last two faces in each orientation condition were buffers. Facial expressions occurred in a random order within each orientation. Each face was shown using a slide projector, and people were asked to identify the emotion depicted on the face from one of the seven expressions. The seven choices were printed on a page that was kept in full view of the participant during the test. The test was self-paced.

Results and Discussion

Table 2 shows accuracy for both groups for identifying facial expressions depicted in upright and inverted faces. An alpha level of .05 was selected for all analyses in the paper. A 2 (group) by 2 (orientation) analysis of variance revealed a significant main effect of orientation, F(1, 24) = 67.42, p < .001. The main effect of group was only marginally significant, F(1, 24) = 3.93, p = .06, presumably because of the small sample sizes. There was no interaction of group by orientation conditions, F(1, 24) = .19, p > .60. Inspection of the individual data indicated that accuracy for upright faces was higher than that for inverted faces for 14 of the 16 patients and 10 of the 10 control participants. Although the patients had greater difficulty identifying facial expressions in general, inverting faces had the same effect on the patients’ identification as it did on the controls’ identification.

TABLE 2.

Proportion of facial expressions correctly identified in upright and inverted faces for patients and control participants in Experiment 1

		Face condition
Group		Upright	Inverted
Control	Mean	0.81	0.58
	SD	0.12	0.14
Patient	Mean	0.69	0.49
	SD	0.19	0.12

One interpretation of these data is that identifying facial expression for inverted faces was impaired because participants could not process spatial relations among facial features in inverted faces. The problem with this interpretation is that inverting a face also changes the appearance of its individual features, which could impair people’s ability to judge expressions even if one were using only discrete facial features to do so. It is unclear from these findings whether or not deficits in identifying expressions in inverted faces were due to problems in processing the configuration of the face or due to problems in discriminating individual features. The next experiment addresses whether schizophrenia patients are disproportionately impaired in the recognition of inverted faces compared with the recognition of other inverted objects.

EXPERIMENT 2

Recognition of all stimuli is more difficult when presented upside down versus right-side up. However, it is well established that face recognition is more impaired by inversion than is recognition for other classes of objects (Scapinello & Yarmey, 1970; Yin, 1969, 1970; see Valentine, 1988 for review). One hypothesis for the disproportionate effect of inversion on facial processing is that inverting faces distorts the integrity of the face configuration, rendering it more difficult to perceive the face as a whole. Because objects can be represented in terms of their individual parts as well as their overall structure (Bruce & Humphreys, 1994; Farah, Wilson, Drain, & Tanaka, 1998), object recognition is less sensitive to changes in orientation than is face recognition.

Experiment 2 was designed to examine the effects of inverting stimuli on recognition memory for faces and objects. Faces and houses were presented in an upright orientation at study, and were tested in both upright and inverted orientations. If patients with schizophrenia rely on configural information to recognise faces, as suggested by Experiment 1, then inverting stimuli should lead to a larger decrement in recognition of faces than it does for recognition of houses.

Method

Participants. There were 20 patients with schizophrenia (18 males, 2 females) and 20 control participants (18 males, 2 females). All patients were recruited from the outpatient (full-day) partial hospitalisation programme in psychiatry. The diagnoses and symptom ratings, were assigned as in Experiment 1. On the basis of the clinical interview, 14 of the 20 patients presented with at least one psychotic symptom (hallucinations and/or delusions). The remaining six patients were remitted with respect to psychotic symptoms but had others, such as negative symptoms (e.g., affective flattening, alogia) and disorganised speech and behaviour. All patients were treated with neuroleptic medications at the time of testing. The mean BPRS score for the patients was 48 (range: 30-71). All patients had a chronic course with multiple psychiatric hospitalisations. The control participants were recruited from nearby communities and the hospital staff. Individuals in both groups fulfilled the same criteria as those who participated in Experiment 1. There was no reliable difference between the groups in their age, t(38) = -1.36, p > .15, and years of education completed, t(38) = 1.22, p > .20.

Materials. The critical materials consisted of black and white photographs of 32 faces and 32 houses. Pictures of faces were selected from a high-school yearbook. The faces were relatively similar in that they lacked outstanding features such as facial hair, glasses, or distinguishing marks. All poses were full face and cropped at shoulder level. The houses were selected to have similar architectural features with few outstanding characteristics such as lawn ornaments or street numbers. The houses were photographed from a front view, with the picture cropped along the outside edge of the house.

One-half of the faces and one-half of the houses were assigned to one list and the other half of the stimuli were assigned to a second list. One list of faces and houses was used for target items and the other list was used as distractors in the recognition test. In addition, six photographs of faces and houses (three each) were used as buffer items that were presented at the beginning and end of the study list.

Design. The design was a 2 (group) × 2 (stimulus type) × 2 (orientation) mixed factorial with group (control and patient) as a between-subjects variable and stimulus type (face and house) and orientation (upright and inverted) as within-subjects variables.

Procedure. Each person was tested individually. All stimuli were presented as slides on a Telex Caramate projector. In the study phase, people viewed a mixed list of 38 faces and houses in the upright orientation, and were instructed to rate the stimuli on the basis of how much they liked them using a scale from 1 to 3, with 3 meaning most likeable. They performed the rating task at their own pace, with each item presented for about 5 seconds. Items were shown in a pseudorandom order with the constraint that one type of stimulus (face or house) was not shown on more than three consecutive trials.

At test, participants viewed a mixed list of 16 face pairs and 16 house pairs, and their memory was assessed in a two-alternative forced-choice recognition test. Participants were instructed to point to the item shown during the study phase. One item in each pair was a target and the other was a distractor. One-half of the face pairs and one-half of the house pairs were shown in an upright orientation, with the remaining pairs shown in an inverted orientation. Test items were shown in a pseudorandom order such that no more than three items from the same condition (i.e., upright and inverted or face and house) were shown consecutively. Across participants, all stimuli were shown equally often as targets and distractors, in upright and inverted orientations, and as a correct response on the right or left side.

Results and discussion

Table 3 shows proportions of correctly recognised upright and inverted stimuli for the patients and control participants. A 2 (group) by 2 (stimulus type) by 2 (orientation) analysis of variance on the accuracy data indicated significant main effects for group, F(1, 38) = 14.05, p < .001, and orientation, F(l, 38) = 97.67, p < .0001. Of greater interest was the significant interaction of stimulus type by orientation, F(l, 38) = 6.92, p < .05, suggesting that recognition of faces was disrupted more by inversion than was recognition of houses. A marginally significant interaction of group by orientation, F(l, 38) = 4.03, p = .052, revealed that patients recognised fewer inverted stimuli in general. The interactions of group by stimulus, F(l, 38) = .98, p > .30, and group by stimulus by orientation, F(1, 38) = .37, p > .50, were not significant. These data indicate that recognition memory for faces was disproportionately affected by inversion compared with recognition memory for houses. Despite their greater difficulty in recognising inverted stimuli (both faces and houses), schizophrenia patients did not appear to be less sensitive to face-specific orientation effects. All 20 patients in this study had higher accuracy for recognising upright compared with inverted faces.

TABLE 3.

Proportion of faces and houses recognised in upright and inverted conditions for patients and control participants in Experiment²

		Faces		Houses
Group		Upright	Inverted	Upright	Inverted
Control	Mean	0.93	0.73	0.87	0.76
	SD	0.09	0.15	0.1	0.11
Patient	Mean	0.91	0.61	0.78	0.63
	SD	0.11	0.21	0.14	0.17

It could be argued that schizophrenia patients displayed normal sensitivity to configural changes in faces because the group was not composed exclusively of patients in an acute psychotic state. Perceptual distortions of the face might occur only in acute psychosis (Cutting, 1990, Harrington, Oepen, & Spitzer, 1989). To address this concern, we tested an additional 10 patients on the identical paradigm described here (see Table 1 for demograplhic details). These 10 patients presented with an acute exacerbation of psychotic symptoms: 9 patients were hospitalised on the inpatient ward and 1 patient continued treatment in the partial hospitalisation programme. The average BPRS score for these patients was 58.2 (range: 43-70). The mean proportion of items correctly recognised for these patients was .90 (SD = 0.18) for upright faces, .68 (SD = 0.16) for inverted faces, .66 (SD = 0.17) for upright houses, and .64 (SD = 0.18) for inverted houses. Analyses on the recognition data for patients with an acute exacerbation of their illness revealed the same pattern of results as those reported earlier for the group of 20 patients. They recognised more faces than houses, F(1, 9) = 8.44, p < .02, and more upright than inverted stimuli, F(1, 9) = 18.00, p < .01. Importantly, recognition of faces was more disturbed by inverting stimuli than was recognition of houses, F(l, 9) = 12.52, p < .01. Accuracy was higher for upright than for inverted faces for all .10 patients. These data replicated the pronounced inversion effects in face recognition for patients with an acute exacerbation of their symptoms, suggesting that patients with acute psychosis are sensitive to configural changes in faces.

An unexpected finding in this experiment was that inverting stimuli produced a larger impairment in recognition for patients than it did for control participants. One reason for this might be that people try to mentally rotate or normalise faces and objects before they recognise the stimuli. This mental rotation process could involve demands on working memory by requiring that rotated images be held in mind while judging their familiarity. Inverting items might reduce the patients’ performance more than the controls’ performance because the patients have greater difficulty performing mental transformations that involve working memory.

However, the design of Experiment 2 makes it difficult to interpret the effects of inverting items on recognition. In this experiment, all items were studied in an upright orientation with half of the items tested upright and half tested inverted. Thus inverted items might have been more difficult to recognise because they were both inverted and tested in a different orientation from the one in which they were studied. If the patients have greater difficulty in mental transformations related to rotation then this could have reduced their recognition of inverted items. More importantly, the patients’ poorer recognition of inverted faces may have resulted from confounding stimulus orientation with study procedure and not from issues related to configural processing. The next experiment was designed to address this issue.

EXPERIMENT 3

As in the previous experiment, the aim was to determine whether schizophrenia patients’ recognition memory for faces is more impaired by inverting stimuli than is their recognition for other objects. In this experiment, participants studied faces and houses in an upright orientation and were tested on those same items in an upright orientation. Using another set of stimuli, they studied inverted faces and houses and were tested on those items in the inverted orientation. Another aim of the experiment was to determine whether group differences in face recognition were due to differences in the groups’ intellectual abilities. We therefore included a clinical control group for the purpose of matching schizophrenia patients and clinical control participants on the revised National Adult Reading Test (NART; Blair & Spreen, 1989), a measure of pre-morbid intelligence.

Method

Participants. The groups were composed of 16 patients with schizophrenia or schizoaffective disorder (14 males and 2 females), 16 clinical control participants (14 males and 2 females), and 16 non-psychiatric controls (14 males and 2 females). All patients were recruited from the outpatient and inpatient programmes in psychiatry. The assignment of diagnoses and the assessment of symptoms for the schizophrenia patients are the same as those described in Experiment 1. The mean BPRS score for the group was 52.4 (range: 37-67). Of the 16 patients in the schizophrenia group, 15 had at least one active psychotic symptom (hallucinations and/or delusions). The one remaining patient had negative symptoms. Fifteen patients had a chronic course with multiple psychiatric hospitalisations and one patient was diagnosed with schizophrenia 12 months prior to his participation in the study, with two psychiatric hospitalisations. All of the patients in the schizophrenia group were treated with neuroleptic medications at the time of testing. The average NART score for the group was 100.44 (SD = 8.5).

The patients in the clinical control group were diagnosed with a mood disorder according to the criteria of the DSM-IV. Eight patients had a diagnosis of major depression, seven had a diagnosis of bipolar mood disorder, and one had a diagnosis of cyclothymia. The diagnoses were based on a semi-structured interview conducted by a psychiatrist and a review of the patient’s chart. Clinical control participants were interviewed using the BPRS to assess the severity of their psychiatric symptoms. The mean BPRS score for the group was 42.4 (range: 29-53). Eight patients had at least one active psychotic symptom (hallucinations and/or delusions). At the time of testing, all patients in this group were taking mood stabilisers/antidepressants and six were treated additionally with neuroleptic medications. Patients in this group were screened for medical and neurological problems. The average NART score for this group was 103.73 (SD = 9.35).

Participants in the non-psychiatric control group were recruited from the community and hospital staff. They met the same inclusion and exclusion criteria as those described in Experiment 1. The average NART score for these control participants was 100.91 (SD = 10.37). The results of statistical tests on demographic variables showed that the three groups did not differ in terms of age, F(2, 47) = .55, p > .05, education, F(2, 47) = 1.62, p > .05 or performance on the NART, F(2, 47) = .57, p > .05.

Materials. The materials consisted of black and white photographs of 64 faces and 64 houses. These photographs were selected using the criteria described in the previous experiment. The materials were divided into four lists, each consisting of 16 faces and 16 houses. Two lists were shown in the upright orientation and two were shown in the inverted orientation. Of the two lists shown in each orientation, one served as target items presented at study and the other served as distractor items shown only in the recognition test.

Design. The design was a 3 (group) × 2 (stimulus type) × 2 (orientation) mixed factorial with group (schizophrenia, clinical control, and non-psychiatric control) as a between-subjects variable and stimulus type (face and house) and orientation (upright and inverted) as within-subjects variables.

Procedure. Each person was tested individually. All stimuli were presented on a Telex Caramate projector. In the study phase, subjects viewed a mixed list of 16 faces and 16 houses in either an upright or inverted orientation. As in the previous experiment, participants were instructed to rate each item according to how much they liked it using a scale from I to 3, with 3 meaning most likeable. Participants rated the items at their own pace.

At test, a mixed list of 16 face pairs and 16 house pairs was presented for a two-alternative forced-choice recognition test. One item in the pair was the target and the other was the distractor. Participants were asked to point to the item they had seen during the study phase. Test items were presented in the same orientation as that presented during the study phase. Once individuals completed the first test, the second set of materials was presented for study and test in the opposite orientation to that presented for the first set. The identical procedures were repeated for this second set of materials. The order of presentation of stimulus orientation was counterbalanced across participants. In addition, all stimuli were shown equally often as target and distractors, in an upright and inverted orientation, and as a correct response on the right and left side in the recognition test across the participants.

Results

Table 4 shows the proportion of correctly recognised faces and houses for schizophrenia patients, clinical control participants, and non-psychiatric control participants. A 3 (group) by 2 (stimulus type) by 2 (orientation) analysis of variance yielded significant main effects for group, F(2, 45) = 3.37, p < .05, stimulus type, F(l, 45) = 18.26, p < .001, and orientation, F(1, 45) = 52.16,p < .001. Post-hoc analyses revealed that schizophrenia patients’ recognition was lower than the recognition performance of non-psychiatric control participants, F(1, 30) = 7.85, p < .01. There were no other reliable differences among the three groups.

TABLE 4.

Proportion of faces and houses recognised in upright and inverted conditions for patients and control participants in Experiment 3

		Faces		Houses
Group		Upright	lnverted	Upright	Inverted
Non-psychiatric control	Mean	0.95	0.74	0.84	0.76
	SD	0.07	0.12	0.13	0.12
Clinlical control	Mean	0.94	0.7	0.82	0.73
	SD	0.09	0.15	0.18	0.11
Schiizoplhrenia	Mean	0.91	0.71	0.76	0.64
	SD	0.98	0.13	0.1	0.17

The key finding in this experiment was a significant two-way interaction between stimulus type and orientation, F(l, 45) = 19.78, p < .001, indicating that inverting stimuli had a larger effect on recognising faces than it did on recognising houses. The two-way interactions of group by stimulus type, F(2, 45) = 2.13, p > .10, and group by orientation, F(2, 45) = .04, p > .90, were not statistically significant. In addition, the three-way interaction of group by stimulus type by orientation was not significant, F(2, 45) = .72, p > .40.

Discussion

The main finding of this experiment was that recognition of faces was impaired by inverting stimuli to a greater extent than was recognition of houses. This pattern of data was obtained for the three groups of participants whose performance on a measure of pre-morbid intelligence did not differ. These findings show that schizophrenia patients are as sensitive to changes in the configuration of the face as are patients with mood disorders and normal volunteers. In contrast to the findings of Experiment 3, schizophrenia patients were not more impaired in recognising inverted items than upright items relative to control participants. It is likely therefore that this previous finding occurred because stimulus orientation was not balanced at study and test phases.

Inversion effects have been interpreted as functional evidence to suggest that face recognition is mediated by different processes than those involved in object recognition (e.g., Farah, Wilson, Drain, & Tanaka, 1995; Farah et al., 1998; Moscovitch, Winocur, & Berhmann, 1997). For instance, Farah and her colleagues (Farah, Tanaka, & Drain, 1995; Farah et al., 1998) have proposed that face recognition differs from object recognition to the extent that each depends on part decomposition. Face recognition depends primarily on holistic representations, whereas object recognition can depend on either holistic or part-based analysis. In contrast to faces, objects can be decomposed into elemental parts. According to this hypothesis, faces are more sensitive to inversion because they are normally encoded as undifferentiated wholes and represented holistically. Facial processing is special because it involves little or no part decomposition.

Theoretical notions of a specialised face processor have been challenged by findings of inversion effects for non-face stimuli. For instance, dog experts (breeders and judges) showed comparable inversion effects for recognising dogs and faces, whereas non-experts showed inversion effects for faces only (Diamond & Carey, 1986). In addition, people who were given training in the discrimination of non-face objects, called Greebles, were sensitive to configural changes in these objects such as inverting stimuli, whereas novices were not (Gauthier & Tarr, 1997). This line of research suggests that inversion effects in face recognition are due to extensive experience in recognizing faces and not to specialized processes mediating facial processing.

The question of whether or not faces are “special” is a matter of some debate (see Ellis & Young, 1998; Farah et al., 1998; Moscovitch et al., 1997; Tovee, 1998 for recent reviews), and we do not propose that the results of this experiment provide evidence for or against this issue. It is clear that inverting faces disrupted recognition memory for schizophrenia patients. Face-specific orientation effects suggest that patients have acquired expertise in recognising upright, as compared with inverted, faces. As with normal volunteers, it appears that these patients are unable to perceive spatial relations among facial features in upside-down faces. Their recognition of faces depends on processing configural information in faces viewed in an upright orientation.

EXPERIMENT 4

This experiment was designed to determine whether people with schizophrenia relied on configural information to recognise a person’s identity. Young et al. (1987) demonstrated the importance of configural information and preserving the integrity of internal facial features when recognising facial identity. These investigators developed a technique in which the top and bottom halves of different faces were combined to create a new face configuration, called a composite. They found that fusing two well-known faces together created a new unfamiliar face, which interfered with the recognition of its separate (familiar) parts. Their results suggested that perception of the face configuration had precedence over perception of its individual parts in recognising the identity of a person.

This experiment tested whether schizophrenia patients were impaired in recognising the top and bottom halves of familiar faces when a new facial composite was formed from these parts. If schizophrenia patients process the whole face, then recognition of facial identity in face parts should be impaired by this newly formed configuration. In contrast, if patients’ processing of configural information is impaired, recognition of identity might be unaffected by composite faces.

Method

Participants. There were 19 patients with a diagnosis of schizophrenia (19 males) and 21 control participants (17 males, 4 females) in the study. The patients were recruited from the partial hospitalisation programme in psychiatry. Participants were recruited from the same pool and fulfilled the same criteria as those in Experiment 1. Of the 19 patients, 15 experienced at least one psychotic symptom (hallucinations and/or delusions). The remaining four patients were remitted with respect to psychotic symptoms and had predominantly negative symptoms and disorganised speech. All patients received neuroleptic medications at the time of testing. The mean BPRS score for the patients was 49.56 (range: 29-73). The groups did not differ significantly in terms of their age, t(38) = .01, p > .90, and years of education completed, t(38) = 1.07, p > .25. Six patients from this experiment were also tested in Experiment 2. There was an average of 7.5 months (range: 6-9 months) between the two test administrations for these six individuals.

Materials. The materials consisted of five computerised images of famous people obtained from sources on the internet: John F. Kennedy, Bill Clinton, Ronald Reagan, Jimmy Carter, and Richard Nixon. The images were full-face common poses of these individuals. The composite and non-composite faces were designed according to the procedure reported in Young et al. (1987). For the composite faces, each face was split into a top and bottom segment by cutting a horizontal line below the eyes. Then, using a graphics program, the top half of one face was joined to the bottom half of a different face. This procedure yielded 20 composite faces. The non-composite faces were created by positioning the bottom-face segment to the left or right of the top-face segment such that the centre of the nose of the bottom segment was aligned to the right or left edge of the top segment. The non-composite faces were centred within the image.

There were two sets of 20 non-composite faces. Each top half of one face was shown with every other bottom half of another face in the right and left positions across the two sets. For example, in set one, the top of John F. Kennedy was shown to the right of the bottom halves of Bill Clinton and Jimmy Carter and to the left of the bottom halves of Richard Nixon and Ronald Reagan. In set two, the top of John F. Kennedy was shown to the right of the bottom halves of Richard Nixon and Ronald Reagan and to the left of the bottom halves of Bill Clinton and Jimmy Carter.

Design. The design was a 2 (group) × 2 (composite type) × 2 (face part) mixed factorial with group as a between-subjects variable and composite type (composite and non-composite face) and face part (top and bottom) as within subjects variables.

Procedure. In the first phase of the experiment, participants viewed the original intact faces of the five people on a television screen, and were asked to name the person. In the next phase of the experiment, they were shown a mixed list of 20 composite faces and 20 non-composite faces in a random order and were asked to name the person in the top part as quickly as possible. Then, the identical list of 40 items was re-presented, and participants were asked to name the person in the bottom part as quickly as possible. One-half of the participants viewed composite faces with the 20 non-composite faces from set one and the remaining half viewed composite faces with the 20 non-composite faces from set two. The presentation order of naming the top person followed by naming the bottom person was fixed for all participants. One experimenter recorded the participants’ verbal responses, and a second experimenter recorded response times by pressing a key on a timer. The second experimenter was blind to the stimuli presented on the television screen.

Results and discussion

Table 5 shows naming accuracy for top and bottom parts of composite and non-composite faces. Accuracy was generally high across conditions and did not differ for patients and control participants, F(l, 38) = 1.4, p > .20.

TABLE 5.

Proportion of top and bottom parts of familiar faces identified in the composite and non-composite conditions for patients and control participants in Experiment 4

		Top		Bottom
Group		Composite	Non-composite	Composite	Non-composite
Control	Mean	0.98	0.99	0.87	0.91
	SD	0.04	0.03	0.13	0.12
Patient	Mean	0.95	0.94	0.82	0.91
	SD	0.07	0.07	0.19	0.14

Figure 1 shows mean response times to name top and bottom parts of composite and non-composite faces. Response times for one patient were lost due to an error in recording responses. A 2 (group) × 2 (composite type) × 2 (face part) analysis of variance performed on response time data indicated significant main effects of group, F(1, 37) = 7.64, p < .01, composite type, F(l, 37) = 8.04, p < .01, and face part, F(1, 37) = 12.70, p < .001. The interaction of composite type by face part was significant, F(1, 37) = 5.47, p < .05, indicating a difference in the composite effect for top and bottom face halves. The response time difference between composite and non-composite conditions was larger for naming bottom face parts. The reduced composite effect for top halves may relate to ceiling effects in these conditions. The accuracy for naming top halves of faces was over 90% for both controls and patients. It can also be noted that response times for bottom face parts were not contaminated by speed-accuracy tradeoffs. The slower response times for composite faces did not occur with higher accuracy. The interactions of group by composite type, F(1, 37) = .70, p > .40, group by face part, F(1, 37) = .05, p > .80, and group by composite type by face part, F(l, 37) = .37, p > .50, were not significant.

Figure 1.

Mean response time in milliseconds (+SE) to name the top and bottom parts of composite and non-composite faces for control participants (A) and patients with schizophrenia (B).

These findings indicated a difference in response times for composite and non-composite faces. Familiar halves of faces were more difficult to recognise when presented in composite images because fusing parts of different faces formed a new unfamiliar face configuration. Perception of the face as a whole diminished the ability of participants to recognise the facial identity in the top and bottom segments. Although the patients had slower response times overall, the pattern of their data in the composite and non-composite conditions was similar to the pattern for control participants. Configural information in composite faces interfered with the patients’ recognition of facial identity in the top and bottom parts. These data suggest, again, that the patients process the configuration of a face as a whole.

GENERAL DISCUSSION

This research examined whether schizophrenia patients’ deficits in recognising faces stem from a problem in processing the configuration of the face. Answering this question seemed to be a critical step in understanding how people with schizophrenia perceive faces because so much has been written in the face-perception literature about the importance of processing configural information. The data from four experiments showed that schizophrenia patients were sensitive to changes in the configuration of the face. In each experiment, the pattern of face recognition for patients did not differ from that for control participants. Identifying facial expressions from inverted faces was more difficult than identifying expressions from upright faces (Experiment 1). Recognition memory for inverted faces was also poorer than recognition of upright faces. As with many previous findings, this orientation effect was greater for faces than for other objects (Experiments 2 and 3). Recognising facial identity was disrupted in a composite image where the top half of one person’s face was fused to the bottom half of another person’s face compared to a condition where the face halves were separated (non-composite) (Experiment 4).

These findings suggest that people with schizophrenia do indeed rely on configural information to recognise faces. Schizophrenia patients, like normal volunteers, had difficulty processing the spatial relations between facial features in upside-down faces. The results with the composite technique suggest that schizophrenia patients do not process parts of the face to recognise facial identity; they process the face as a whole. Although one cannot accept the null hypothesis, the pattern of the data also suggests that patients with schizophrenia were as sensitive to configural changes as were normal and clinical controls. Thus it appears that schizophrenia patients’ face recognition is not based solely on selecting isolated features but that they too perceive a face as more than a sum of its parts.

This observation is consistent with other findings in the perceptual organisation literature. According to Knight and Silverstein (1998), perceptual organisation refers to the ability to “rapidly and automatically organise stimulus components into wholes” (Knight & Silverstein, 1998, p. 261). Recent findings indicate that schizophrenia patients perform normally on tasks that use Gestalt grouping principles and symmetrical patterns composed of contiguous components (Chey & Holzman, 1997; Knight & Silverstein, 1998). These findings suggest that patients combine stimulus components with strong configural properties into wholes rather than perceiving them as individual elements. It has been proposed that perceptual organisation is normal in schizophrenia patients because processing of some visual forms is carried out automatically before attention is distributed to specific features. This type of visual processing occurs with little experience, is seen early in development, and is possibly innate (Chey & Holzman, 1997; Knight & Silverstein, 1998; Silverstein, Osborn, West, & Knight, 1998).

It is possible that schizophrenia patients had normal sensitivity to configural changes in the face because processing face configurations is a visual processing skill that is also automatic and occurs early in development. Research has shown that people can detect faces at lower thresholds (less than 40 ms) than the thresholds required to classify faces, with upright faces detected at lower thresholds than upside-down faces (Purcell & Stewart, 1988). This suggests that detection of the face configuration occurs during early stages of visual processing and is sensitive to the arrangement of facial features. It has also been shown that rapid visual search of stimuli is influenced by whether or not stimuli formed a face configuration. Suzuki and Cavanagh (1995) showed that visual search of stimuli was determined by the global facial structure and not by individual features. In their study, global structure had precedence over search of discrete features. Developmental evidence of preferential tracking of face patterns in newborns suggests that there is innate or early knowledge of face configurations (Johnson, 1999; Morton & Johnson, 1991). Although this evidence points to early holistic processing of faces that develops with little experience, other findings fail to support the notion that faces are processed automatically. For example, faces do not show perceptual “pop-out” effects indicative of parallel or preattentive processing (Nothdurft, 1993).

Another possible reason for why schizophrenia patients show intact processing of configural information is that they have acquired basic knowledge about face patters through early and extensive exposure to faces viewed in a normal upright orientation. As with normal volunteers, people with schizophrenia develop expertise in recognising upright faces. Their difficulty in recognising upside-down faces or parts of composite faces may simply reflect sensitivity to changes in a highly familiar, overlearned stimulus.

The present research was undertaken to explore one possibility for the well-established findings that schizophrenia patients are impaired in a number of facial processing tasks. Based on our findings, it does not appear that deficits in processing configural information account for these patients’ difficulties in recognising faces. These data provide little support for the idea that patients perceive faces in a piecemeal or fragmented manner.

If facial processing deficits do not arise because of problems in processing the face as a whole, then what are deficits due to? The mechanisms underlying judgements about facial identity, facial expression, and recognition of unfamiliar faces differ from one another (Bruce & Young, 1986). It is likely therefore that different mechanisms are responsible for the different face recognition problems seen in people with schizophrenia. Poor recognition memory for faces may be better understood in the broader context of declarative memory failures frequently observed in this population.

For identification of facial expression, one of the many unanswered questions about this phenomenon in schizophrenia is whether or not it is more impaired than other types of face recognition. Some evidence suggests that it is (e.g., Archer et al., 1994; Borod et al., 1993; Walker et al., 1984) although a differential deficit is not always observed (e.g., Kerr & Neale, 1993; Kohler et al., 2000). Given its significance to social behaviour, it is important to understand this problem in schizophrenia irrespective of whether or not it is differentially impaired.

Schizophrenia patients may have difficulty identifying facial expressions because these judgements involve different processes than those involved in perceiving the configuration of a face. One suggestion is that recognising facial affect involves greater attention to a constellation of features. Information about the shape of facial features (e.g., widening or narrowing of eyes, open or closed mouth) and the positioning of facial features relative to one another is important for recognising facial expression (Bruce & Young, 1998, Ch. 6). This idea was demonstrated using a procedure to develop caricatures of facial expressions. Participants’ recognition of facial expressions was enhanced when differences between locations of features in an expression face and a reference-norm face (e.g., neutral face) were accentuated (Calder, Young, Rowland, & Perrett, 1997). In contrast, participants’ recognition of expressions was decreased when the differences (or distances) between features in the expression face and reference norm face were reduced. Thus recognising facial expressions depends on perceiving a combination of facial features, with a specific combination of features associated with communicating one emotion (e.g., fear) more readily than other emotions.

Evidence from neuroscience also suggests that analysing facial expressions differs from processing the face as a whole. A recent study of single cell recording showed that face-responsive neurons in the inferior temporal cortex have different response patterns for global information about faces and detailed information about faces (Sugase, Yamnane, Ueno, & Kawano, 1999). Neuronal activity for global information, such as whether a stimulus is a face or geometric shape, occurred earlier than activity associated with detailed information about facial expression and personal identity. Sugase et al. (1999) suggest that the delayed neural response for detailed information allows for input from other brain structures that have reciprocal connections with the inferior temporal cortex, such as the parahippocampal cortex and orbitofrontal and prefrontal cortex. Taking this evidence together, schizophrenia patients may be impaired in identifying facial expressions because expression analysis depends on greater attention and discrimination to changes in a constellation of facial features compared with processing global features, and because it involves brain areas and systems that are impaired in the disorder. This idea is consistent with findings that recognition of facial affect in schizophrenia patients is correlated to cognitive deficits including attention (Addington & Addington, 1998; Kohler et al., 2000; Poole, Tobias, & Vinogradov, 2000).

Our data provide evidence that people with schizophrenia utilise information about the face configuration when recognising natural photographs of faces. Despite the evidence that the patients displayed normal sensitivity to configural changes in the face, there are many unanswered questions concerning facial processing deficits in these patients. One challenge is to understand how deficiencies in different perceptual and cognitive mechanisms influence these patients’ ability to recognise faces. Face recognition problems may be due to impairments in attending to changes in subtle relations among facial features. But it is also possible that deficits occur because patients misinterpret the meaning of facial features as a result of their difficulties injudging the emotions, intentions, and mental states of others (Frith & Corcoran, 1996). An understanding of face perception and its role in social functioning in people with schizophrenia will inevitably require study of both face recognition mechanisms and the array of cognitive and emotional deficits found in this illness.