Skip to main content
NCBI home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2010 Mar 3;30(9):3544–3552. doi: 10.1523/JNEUROSCI.4863-09.2010

Internal and External Features of the Face Are Represented Holistically in Face-Selective Regions of Visual Cortex

Timothy J Andrews 1,2,✉, Jodie Davies-Thompson 1,2, Alan Kingstone 3, Andrew W Young 1,2
PMCID: PMC2839485  EMSID: UKMS28922  PMID: 20203214

Abstract

The perception and recognition of familiar faces depends critically on an analysis of the internal features of the face (eyes, nose, mouth). We therefore contrasted how information about the internal and external (hair, chin, face outline) features of familiar and unfamiliar faces is represented in face-selective regions. There was a significant response to both the internal and external features of the face when presented in isolation. However, the response to the internal features was greater than the response to the external features. There was significant adaptation to repeated images of either the internal or external features of the face in the fusiform face area (FFA). However, the magnitude of this adaptation was greater for the internal features of familiar faces. Next, we asked whether the internal features of the face are represented independently from the external features. There was a release from adaptation in the FFA to composite images in which the internal features were varied but the external features were unchanged, or when the internal features were unchanged but the external features varied, demonstrating a holistic response. Finally, we asked whether the holistic response to faces could be influenced by the context in which the face was presented. We found that adaptation was still evident to composite images in which the face was unchanged but body features were varied. Together, these findings show that although internal features are important in the neural representation of familiar faces, the face's internal and external features are represented holistically in face-selective regions of the human brain.

Introduction

Although recognizing human faces is a simple process for most people, the differences between faces are small compared with the variation that occurs between many nonface objects. A variety of evidence suggests that the mechanisms involved in face processing are distinct from those used to process other categories of objects (Yin, 1969; McNeil and Warrington, 1993; Tanaka and Farah, 1993; Pitcher et al., 2009). These mechanisms are thought to primarily analyze the internal features of the face (Valentine, 1991; Leopold et al., 2001; Maurer et al., 2002; Yovel and Kanwisher, 2004), with the ability to make use of differences in internal features improving as faces become familiar (Hancock et al., 2000; Burton et al., 2005). For example, behavioral studies have shown that internal features become more salient than external features when recognizing familiar faces, but that perceivers make use of both internal and external features of unfamiliar faces (Ellis et al., 1979; Young et al., 1985; O'Donnell and Bruce, 2001).

Despite the clear importance of the internal features in the perception and recognition of faces, other studies have suggested the importance of holistic processing, in which both the internal and external features of the face are combined to create an overall representation. For example, the same internal features of a face can be perceived differently according to the face setting in which they are viewed (Young et al., 1987; Sinha and Poggio, 1996, 2002)—combining even highly familiar internal features (eyes, nose, and mouth) with an inappropriate set of external features (hair, chin, and face outline) produces a composite face whose constituent parts can be difficult to recognize.

Our aim was to draw on these behavioral findings to elucidate how information about the internal and external features of the face is represented in face-selective regions of the human brain (Haxby et al., 2000; Fairhall and Ishai, 2007). Several studies suggest that one region—the fusiform face area (FFA)—is important for processing facial identity. For example, the response in the FFA is modulated by successful face recognition (Grill-Spector et al., 2004) and shows a reduced response (adaptation) to repeated images with the same identity (Grill-Spector et al., 1999; Andrews and Ewbank, 2004; Rotshtein et al., 2005; Yovel and Kanwisher, 2005). Despite evidence for the role of the FFA in face recognition, the relative importance of the internal and external features of the face in activating this region has not been tested explicitly, and the majority of studies have used unfamiliar faces as stimuli. First, we measured the responses to the internal and external features of familiar and unfamiliar faces when presented in isolation. Next, we asked to what extent the internal and external features of the face have independent or holistic neural representations, using composite face images in which the internal features of one face were combined with the external features of another face. Finally, we determined whether the holistic response to faces could be influenced by the context (such as a body) in which a face is presented.

Materials and Methods

Subjects.

Twenty-one participants took part in experiment 1 (12 females; mean age, 22), twenty-three subjects took part in experiment 2 (14 females; mean age, 24), and twenty-one subjects took part in experiment 3 (13 females; mean age, 23). All participants were right handed and had normal to corrected-to-normal vision. Written consent was obtained for all subjects and the study was approved by the York Neuroimaging Centre Ethics Committee. Visual stimuli (∼8° × 8°) were back-projected onto a screen located inside the magnetic bore, ∼57 cm from subjects' eyes.

Imaging parameters.

All experiments were performed using a GE 3 tesla HD Excite MRI scanner at the York Neuroimaging Center at the University of York. A Magnex head-dedicated gradient insert coil was used in conjunction with a birdcage, radio-frequency coil tuned to 127.4 MHz. A gradient-echo EPI sequence was used to collect data from 38 contiguous axial slices (experiments 1 and 2: TR = 3 s, TE = 25 ms, FOV = 28 × 28 cm, matrix size = 128 × 128, slice thickness = 3 mm; experiment 3: TR = 2 s, TE = 30 ms, FOV = 24 × 24 cm, matrix size = 128 × 128, slice thickness = 4 mm).

Stimuli.

Face images were taken from the Psychological Image Collection at Stirling (PICS; http://pics.psych.stir.ac.uk/), and a variety of internet sources. Body images were taken from a body image collection at Bangor (http://www.bangor.ac.uk/~pss811/). Images of other categories were taken from a variety of web-based sources. Images were presented in gray scale. In the adaptation scans, the mean change in image intensity across images was calculated by taking the average of the absolute differences in gray value at each pixel for successive pairs of images within a block. Images were presented using Neurobehavioural Systems Presentation 13.0.

Familiar faces were chosen on the basis of familiarity ratings obtained from a separate group of subjects. The familiarity of the faces used in the adaptation scan was confirmed by a test that was performed outside the scanner in which subjects were asked whether the faces were familiar and whether they could report the name and occupation. In the familiarity test, participants from experiments 1 and 2 reported 95% (±0.9) to be familiar. They were also able to report the correct occupation (92% ± 1.6) and name (87% ± 1.6) from the majority of the familiar face images. Different images of the familiar identities were used in the familiarity test and in experiments 1 and 2. The unfamiliar faces were chosen to match familiar faces for variation in age and appearance.

Localizer scan.

To identify regions responding selectively to faces in the visual cortex, a localizer scan was performed for each subject. Subjects viewed 20 blocks of 10 images. Each block contained images from one of the five different categories: faces, bodies, objects, places, or Fourier scrambled images of the former categories. Each image was presented for 700 ms followed by a 200 ms fixation cross. Stimulus blocks were separated by a 9 s fixation gray screen. Each condition was repeated four times, and arranged in a counterbalanced block design.

Experiment 1—adaptation to internal and external features of the face.

To determine how the internal and external features of the face are represented in face-selective regions of the brain, we used an adaptation paradigm that included six conditions: (1) same internal features, (2) different internal features, (3) same external features, (4) different external features, (5) same whole face, and (6) different whole faces. A blocked design was used to present the stimuli. Each stimulus block consisted of nine images from one of the conditions. In each block, images were shown for 800 ms followed by a 200 ms fixation cross. The mean image change between successive stimuli in each condition is shown in supplemental Table 1 (available at www.jneurosci.org as supplemental material). Stimulus blocks were separated by a 9 s fixation gray screen. Each condition was repeated eight times in a counterbalanced order, giving a total of 48 blocks per scan, and each scan was repeated for each subject with familiar and unfamiliar faces. To monitor attentional load across stimulus conditions, subjects had to press a button to report the occurrence of a red dot, which was superimposed on 15% of the images.

Experiment 2—adaptation to composite (internal/external) face images.

To determine whether the internal and external features of the face are represented independently or holistically in face-selective regions of the brain, we used an adaptation paradigm with four image conditions: (1) same internal features, same external features; (2) same internal features, different external features; (3) different internal features, same external features; and (4) different internal features, different external features. Composite images were generated using image editing software. A blocked design was used to present the stimuli. Each stimulus block consisted of nine images from one of the conditions. In each block, images were shown for 800 ms followed by a 200 ms fixation cross. The mean image change between successive stimuli in each condition is shown in supplemental Table 2 (available at www.jneurosci.org as supplemental material). Stimulus blocks were separated by a 9 s fixation gray screen. Each condition was repeated six times, giving a total of 48 (24 unfamiliar, 24 familiar) blocks. To monitor attentional load across stimulus conditions, a red dot task was used.

Experiment 3—the role of context in face processing.

To determine whether the neural representation in face-selective regions could be influenced by changes in the immediate context in which the face was presented, we used composite images in which the faces could be superimposed on different bodies or busts. There were four image conditions: (1) same face, same bust; (2) same face, different bust; (3) different face, same bust; (4) different face, different bust. The faces were unfamiliar to the subjects. A blocked design was used to present the stimuli. Each stimulus block consisted of 10 images from one of the conditions. In each block, images were shown for 800 ms followed by a 200 ms fixation cross. The mean image change between successive stimuli in each condition is shown in supplemental Table 3 (available at www.jneurosci.org as supplemental material). Stimulus blocks were separated by a 10 s fixation gray screen. Each condition was repeated six times, giving a total of 24 blocks. To monitor attentional load across stimulus conditions, a red dot was superimposed on one or two images in each block. Subjects were required to respond, with a button press, as soon as they saw the image containing the target.

fMRI analysis.

Statistical analysis of the fMRI data was performed using FEAT (http://www.fmrib.ox.ac.uk/fsl). The initial 9 s of data from each scan were removed to minimize the effects of magnetic saturation. Motion correction was followed by spatial smoothing (Gaussian, FWHM 6 mm) and temporal high-pass filtering (cutoff, 0.01 Hz). For the localizer scan, face-selective regions of interest (ROIs) were determined by the contrast face > place thresholded at p < 0.001 (uncorrected). The time series of the resulting filtered MR data at each voxel was converted from units of image intensity to percentage signal change by subtracting and then normalizing the mean response of each scan [(x − mean)/mean × 100]. All voxels in a given ROI were then averaged to give a single time series in each ROI for each subject. Individual stimulus blocks were normalized by subtracting every time point by the zero point for that stimulus block. The normalized data were then averaged to obtain the mean time course for each stimulus condition. The peak response calculated as an average of the response at 9 and 12 s in experiments 1 and 2 and the average of the response at 8, 10, and 12 s in experiment 3 after the onset of a block. Repeated-measures ANOVA were used to determine significant differences in the peak response to each stimulus condition in each experiment.

Results

Localizer scan

Figure 1 shows the location of three different regions in the occipital and temporal lobe that showed face-selective activity (face > place): FFA; occipital face area (OFA); and superior temporal sulcus (STS) (Haxby et al., 2000). The coordinates of each region are shown in Table 1. Each region was defined separately for each individual, and all further analyses were performed on the mean time courses of voxels in these ROIs. The average time courses of activation in the face-selective regions are also shown in Figure 1. The curves confirm that the model used to analyze the data (face > place) provides a good description of the raw MR activity and illustrate the differences in responses between faces and other categories of objects. There was no difference in the pattern of response between the right and left hemispheres. Accordingly, all subsequent analyses were based on a pooled analysis in which the right and left hemisphere voxels were combined.

Figure 1.

Figure 1.

Open in a new tab

Localizer scan. Top, Location of face-selective regions (FFA, OFA, STS) across all subjects in a whole-brain group analysis. These scan images follow radiological convention, with the left hemisphere shown on the right. Bottom, MR time course during localizer scans, showing activity averaged across hemispheres and subjects for each stimulus category in face-selective areas. The shaded regions represent the duration of each stimulus block. Error bars represent ±SE.

Table 1.

Mean MNI coordinates of face-selective regions of interest

Region Hemisphere Experiment 1
 Experiment 2
 Experiment 3
n x y z n x y z n x y z
FFA L 18 −40 −61 −19 22 −34 −55 −18 14 −39 −67 −17
R 21 43 −55 −20 23 40 −54 −26 18 41 −55 −29
OFA L 12 −40 −85 −13 18 −43 −84 −16 5 −44 −74 −20
R 16 46 −77 −9 21 46 −78 −12 16 39 −76 −18
STS R 17 52 −50 6 19 51 −62 3 14 51 −57 6
Open in a new tab

Experiment 1—adaptation to internal and external features of the face

Figures 2 and 3 show the responses in the different regions to internal and external features of familiar and unfamiliar faces. The FFA was localized in all (21) subjects, whereas the OFA and STS were localized in 17 subjects. The data were first analyzed using a 3 × 2 × 2 × 3 repeated-measures ANOVA with ROI (FFA, OFA, STS), familiarity (familiar, unfamiliar), identity (same, different), and feature (internal, external, whole) as the main factors. There were significant effects of ROI (F(2,28) = 75.0, p < 0.0001), familiarity (F(1,14) = 4.5, p < 0.05), feature (F(2,28) = 25.0, p < 0.0001), and identity (F(1,14) = 85.2, p < 0.0001). There was a significant interaction between ROI and identity (F(2,28) = 52.1, p < 0.0001) and between familiarity and identity (F(2,28) = 10.1, p < 0.01). The data were then analyzed in each region using a 2 × 3 repeated-measures ANOVA with identity (same, different) and feature (internal, external, whole) as the main factors.

Figure 2.

Figure 2.

Open in a new tab

Experiment 1—adaptation to internal and external features of familiar faces. Left, Examples of familiar faces. Right, The average time course of response across subjects in the FFA, the OFA, and the STS. The shaded regions represent the duration of stimulus presentation. Error bars represent ±SE.

Figure 3.

Figure 3.

Open in a new tab

Experiment 1—adaptation to internal and external features of unfamiliar faces. Left, Examples of unfamiliar faces. Right, The average time course of response across subjects in the FFA, the OFA, and the STS. The shaded regions represent the duration of stimulus presentation. Error bars represent ±SE.

First, we determined whether there was a significant effect of feature (whole, internal, external) in the face-selective regions. In the FFA, there was an effect of feature for familiar (F(2,40) = 33.4, p < 0.001) and unfamiliar (F(2,40) = 27.9, p < 0.001) faces. The effect of feature in the FFA was due to a larger response to the internal features than to the external features (familiar: t(20) = 6.3, p < 0.001; unfamiliar: t(20) = 5.1, p < 0.001). Similarly, the response to the whole face was greater than the response to external features for both the familiar (t(20) = 7.59, p < 0.001) and unfamiliar (t(20) = 7.2, p < 0.001) faces. However, there was no difference in the response of the FFA to the internal features compared to the whole face for either the familiar (t(20) = 1.2, p = 0.51) and unfamiliar (t(20) = 1.5, p = 0.28) faces.

There was a significant effect of feature for familiar (F(2,32) = 3.6, p < 0.05) and unfamiliar (F(2,32) = 6.9, p < 0.005) faces in the OFA. This was due to a progressive change in the response to the external features (familiar: 1.05 ± 0.10%, unfamiliar: 0.95 ± 0.12%), internal features (familiar: 1.42 ± 0.10%, unfamiliar: 1.25 ± 0.10%), and whole faces (familiar: 1.43 ± 0.12%, unfamiliar: 1.33 ± 0.12%).

There was also an effect of feature in the STS for familiar (F(2,32) = 20.8, p < 0.001) and unfamiliar (F(2,32) = 28.7, p < 0.001) faces. This was due to a lower response to the external features than to the internal features (familiar: t(16) = −3.8, p < 0.005; unfamiliar: t(16) = −8.15, p < 0.001) and the whole face (familiar: t(16) = −5.3, p < 0.001; unfamiliar: t(16) = −5.1, p < 0.001).

Next we determined the effect of identity (same, different) in the face-selective regions. There was a significant effect of identity in the FFA for familiar (F(1,20) = 82.7, p < 0.001) and unfamiliar (F(1,20) = 69.6, p < 0.001) faces. The effect of identity was due to a smaller response (adaptation) to the same-face conditions than to the corresponding different-faces conditions for familiar (diff. whole > same whole: t(20) = 7.2, p < 0.001; diff. internal > same internal: t(20) = 8.8, p < 0.001; diff. external > same external: t(20) = 7.04, p < 0.001) and unfamiliar (diff. whole > same whole: t(20) = 6.6, p < 0.001; diff. internal > same internal: t(20) = 5.94, p < 0.001; diff. external > same external: t(20) = 5.3, p < 0.001) faces.

There was also a significant effect of identity in the OFA for familiar (F(1,16) = 116.7, p < 0.001) and unfamiliar (F(1,16) = 18.6, p < 0.001) faces. This was due to a smaller response (adaptation) to the same-face conditions than to the corresponding different-faces conditions for familiar faces (diff. whole > same whole: t(16) = 6.61, p < 0.001; diff. internal > same internal: t(16) = 9.6, p < 0.001; diff. external > same external: t(16) = 6.09, p < 0.001), and unfamiliar faces (diff. whole > same whole: t(16) = 4.3, p < 0.005; diff. external > same external: t(16) = 2.5, p < 0.05). There was a trend, but no significant differences, between the internal features (diff. internal > same internal: (t(16) = 2.3, p = 0.07) of unfamiliar faces.

There was a significant effect of identity in the STS for familiar (F(1,16) = 15.67, p < 0.001) but not unfamiliar (F(1,16) = 0.02, p = 0.90) faces. The effect of identity for familiar faces was due to the difference between the same and different whole faces and internal features (diff. whole > same whole: t(16) = 4.33, p < 0.001; diff. internal > same internal: t(16) = 2.63, p < 0.05). No adaptation was evident for external features (diff. external > same external: t(16) = 0.88, p = 0.78).

Finally, we determined whether there was a significant interaction between feature and identity. In the FFA, although there was no significant interaction (F(2,40) = 0.97, p = 0.39) for unfamiliar faces, there was a trend for an interaction between identity and feature (F(2,40) = 2.67, p = 0.08) for familiar faces (Fig. 4). A post hoc 2 × 2 ANOVA specifically aimed to examine differences between internal and external representations revealed a significant interaction in the FFA between feature (internal, external) and identity (same, different) for familiar faces (F(1,20) = 5.52, p < 0.05) but not unfamiliar faces (F(1,20) = 0.75, p = 0.40). This was caused by a larger difference in the adaptation effect (different − same) for the internal (0.63 ± 0.07%) than for the external (0.48 ± 0.07%) features of familiar faces, but no difference between adaptation to internal (0.49 ± 0.08%) and external (0.40 ± 0.07%) features of unfamiliar faces.

Figure 4.

Figure 4.

Open in a new tab

Experiment 1—adaptation to internal and external features of faces. The average adaptation response in the FFA to internal and external features of familiar and unfamiliar faces in experiment 1. There was a significant interaction between feature and identity for familiar faces, which was explained by a larger adaptation effect (diff. − same) for internal than for external features of the face. A similar difference was not evident for unfamiliar faces.

Unlike the FFA, there was no interaction in the OFA between identity and feature for familiar (F(2,32) = 0.82, p = 0.45) or unfamiliar (F(2,32) = 1.67, p = 0.21) faces. Similarly, there was no interaction between identity and feature for either familiar (F(2,32) = 1.12, p = 0.34) or unfamiliar (F(2,32) = 0.03, p = 0.97) faces.

Experiment 2—adaptation to composite (internal/external) face images

Figures 5 and 6 show the responses in the FFA, OFA, and STS to different composites of the internal and external features of either familiar or unfamiliar faces. The FFA was localized in all (23) subjects, the OFA was localized in 21 subjects, and the STS was localized in 19 subjects. The data were analyzed using a 2 × 2 repeated-measures ANOVA with internal (same, different) and external (same, different) as the main factors.

Figure 5.

Figure 5.

Open in a new tab

Experiment 2—Adaptation to composite (internal/external) images of familiar faces. Top, Example of images from different conditions (row 1: same internal, same external; row 2: same internal, different external; row 3: different internal, same external; row 4: different internal, different external). Bottom, The average time course of response across subjects in face-selective regions. Error bars represent ±SE.

Figure 6.

Figure 6.

Open in a new tab

Experiment 2—adaptation to composite (internal/external) images of unfamiliar faces. Top, Example of images from different conditions (row 1: same internal, same external; row 2: same internal, different external; row 3: different internal, same external; row 4: different internal, diff. external). Bottom, The average time course of response across subjects in face-selective regions. Error bars represent ±SE.

In the FFA, there was a significant effect of external for both familiar (F(1,22) = 11.2, p < 0.005) and unfamiliar (F(1,22) = 8.2, p < 0.01) faces. Although there was no effect of internal for either familiar (F(1,22) = 2.65, p = 0.12) or unfamiliar (F(1,22) = 0.36, p = 0.56) faces, there was a significant interaction (familiar: F(1,22) = 18.6, p < 0.001; unfamiliar: F(1,22) = 8.6, p < 0.01). A post hoc analysis revealed that there was a smaller response to the same-internal, same-external condition than to all other conditions for familiar (same internal, diff. external: t(22) = 6.23, p < 0.001; diff. internal, same external: t(22) = 3.46, p < 0.005; diff. internal, diff. external: t(22) = 4.65, p < 0.001) and unfamiliar (same internal, diff. external: t(22) = 4.22, p < 0.001; diff. internal, same external: t(22) = 2.24, p = 0.07; diff. internal, diff. external: t(22) = 2.43, p < 0.05) faces. There was no significant difference between any of the other conditions.

In the OFA, there was a significant effect of external for familiar (F(1,20) = 8.4, p < 0.01), but not unfamiliar (F(1,20) = 2.2, p = 0.15) faces. Although there was no effect of internal for either familiar (F(1,20) = 0.09, p = 0.77) or unfamiliar (F(1,20) = 0.06, p = 0.81) faces, there was a significant interaction (familiar: F(1,20) = 21.9, p < 0.001; unfamiliar: F(1,20) = 6.9, p < 0.05). A post hoc analysis revealed there was a smaller response (adaptation) to same internal features, same external features than to same internal, diff. external (t(20) = 5.66, p < 0.001) and diff. internal, same external (t(20) = 2.68, p < 0.05) but not to diff. internal, diff. external (t(20) = 2.10, p = 0.10) features of familiar faces. For unfamiliar faces, the only significant difference was between the same-internal, same-external condition and the same-internal, diff.-external condition (t(20) = 2.92, p < 0.05).

There was no significant effect of external (familiar: F(1,18) = 0.51, p = 0.49; unfamiliar: F(1,18) = 0.59, p = 0.45) or internal (familiar: F(1,18) = 0.02, p = 0.90; unfamiliar: F(1,18) = 0.01, p = 0.92) in the STS. There was also no interaction between internal and external features (familiar: F(1,18) = 0.92, p = 0.35; unfamiliar: F(1,18) = 2.4, p = 0.14).

Experiment 3—the role of context in face processing

Figure 7 shows the effect of changing the context in which a face is presented on the responses in the FFA, OFA, and STS. The FFA was localized in 21 subjects, the OFA was localized in 17 subjects, and the STS was localized in 14 subjects. The data were analyzed using a 2 × 2 repeated-measures ANOVA with face (same, different) and bust (same, different) as the main factors.

Figure 7.

Figure 7.

Open in a new tab

Experiment 3—the role of context in face processing. Top, Example of composite images (row 1: same face, same bust; row 2: same face, diff. bust; row 3: diff. face, same bust; row 4: diff. face, diff. bust). Bottom, The average time course of response across subjects in the core face-selective regions. Error bars represent ±SE.

There was a significant effect of face (F(1,20) = 25.9, p < 0.0001) in the FFA. Post hoc analysis revealed that this pattern of results was due to a smaller response (adaptation) to the same-face, same-bust condition than to the different-face conditions (diff. face, same bust: t(20) = 4.1, p < 0.001; diff. face, diff. bust: t(20) = 5.5, p < 0.0001). Similarly, there was a smaller response to the same-face, diff.-bust condition than to the different-face conditions (diff. face, same bust: t(20) = 2.1, p < 0.05; diff. face, diff. bust: t(20) = 3.9, p < 0.001). There was also a significant, but smaller, effect of bust (F(1,20) = 6.4, p < 0.05) in the FFA. This effect was due to a small release from adaptation in the same-face, diff.-bust condition compared to the same-face, same-bust condition (t(20) = 3.2, p < 0.005). However, there was no significant difference between the diff.-face, same-bust and diff.-face, diff.-bust conditions (t(20) = 0.8, p = 0.4).

There was a significant effect of face (F(1,16) = 18.1, p < 0.001), but no effect of bust (F(1,16) = 0.45, p = 0.51) in the OFA. This pattern of results was due to due to a smaller response (adaptation) to a difference between conditions in which the same face was repeated (same face/same bust: 1.33 ± 0.16%, same face, diff. bust: 1.40 ± 0.17%) than to when different faces were presented (same face, same bust: 1.62 ± 0.20%, same face, diff. bust: 1.65 ± 0.19%).

There was no significant effect of face (F(1,13) = 0.33, p = 0.57) or bust (F(1,13) = 0.14, p = 0.71) in the STS face region.

Discussion

The aim of this study was to probe how the internal (eyes, nose, and mouth) and external (hair, face shape) features of a face are represented in face-selective regions of the human brain. The main results from this study are as follows: (1) there was a larger response to the internal features of the face than to the external features; (2) there was more adaptation to the internal features than to the external features of familiar faces in the FFA, but there was no difference in adaptation between the internal and external features of unfamiliar faces; (3) changes to either the internal or external features of a face caused a complete release from adaptation, demonstrating that these features are represented holistically; and (4) the holistic response to faces does not extend to the context in which the face is presented.

The importance of the internal features of the face in the recognition of familiar faces has been shown in a number of behavioral studies (Ellis et al., 1979; Young et al., 1985; O'Donnell and Bruce, 2001). For example, Ellis et al. (1979) used a memory task in which participants had to remember a set of famous or unfamiliar faces. Participants were then presented with either the internal or external features of faces, and asked to recall whether they had seen the face during the learning phase. The results showed that participants were better able to recognize the identities of familiar faces from the internal features than from the external features. In contrast, there was no difference in recognition rate between the internal and external features of unfamiliar faces. Young et al. (1985) found similar results using a matching task in which participants had to judge whether two images were the same person. Participants were faster at making this judgment using the internal features than using the external features with familiar faces. However, there was no difference between the internal and external features when matching unfamiliar faces. Interestingly, Moscovitch and Moscovitch (2000) reported that the patient CK, who has a severe object agnosia with preserved face recognition, showed a more pronounced difficulty in recognizing external features than control subjects. Our imaging results fit well with these behavioral studies by showing greater response to the internal features than to the external features of the face. Moreover, there was significantly more adaptation to the internal features of the face than to the external features in the FFA for familiar faces, but no difference in adaptation for unfamiliar faces (see Fig. 4). These results are also consistent with recent results showing a difference in the way that familiar and unfamiliar faces are represented in the FFA (Ewbank and Andrews, 2008).

Although face-selective regions responded more to the internal features of the face, significant responses to the external features were evident in these regions. This fits with behavioral results, which show that, while recognition is better when the internal features of the face are shown, it is still possible to identity faces based solely on the presence of the external features (Ellis et al., 1979). Indeed, it is possible for the same internal features to be recognized as different individuals just based on differences in the appearance of the external features of the face (Sinha and Poggio, 1996, 2002). More recently, Freiwald et al. (2009) reported that neurons in the macaque temporal lobe are particularly sensitive to the aspect ratio or shape of the face. So, although our results demonstrate that the external features of faces are represented in face-selective regions, they are not specific about which aspects of the external features are preeminent.

To determine whether the internal and external features of faces are represented independently, we generated composite face images in which the internal features of one face were combined with the external features of another face. If face regions represent internal and external features of faces independently, a reduced response (adaptation) would be expected when either the internal or external features of the face are repeated, regardless of any change to the image. However, if the FFA relies on some form of holistic representation, then changes to either the internal or external features of the face should result in a release from adaptation.

Most models of face processing emphasize the importance of the configuration of features of the face, but there are different senses in which the idea of “configuration” has been invoked. Particularly important in this respect is the difference between a holistic configurational representation, in which information from all parts of the face is indissolubly combined, and a “second-order” configurational representation that specifies how the various parts of the face are positioned with respect to each other (Maurer et al., 2002; Yovel and Kanwisher, 2005; Maurer et al., 2007). In terms of this distinction, recombining the internal and external features of different faces (as in our experiment 2) disrupts any holistic representation, but should have little effect on a second-order configurational representation (since the spacing of the critical internal features themselves remains unchanged). Our findings that there was a release from adaptation in the FFA to composite images in which the internal features were unchanged but the external features varied on successive images, or when the external features were unchanged and the internal features were varied, strongly suggest that the FFA processes information using a holistic code in which the internal and external features of the face are both important. These findings fit with other evidence that face parts are processed interactively rather than independently (Yovel and Axelrod, 2009) (but see Betts and Wilson, 2010). For example, the combination of highly familiar internal features with an inappropriate set of external features produces a composite face that can be difficult to recognize accurately (Young et al., 1987). The importance of a holistic representation of faces is also apparent when the lower half of one face is aligned with the upper half of another face to produce a composite image (Young et al., 1987); the resulting novel configuration impairs recognition of the two halves of the face, suggesting that face perception is based on a holistic process. Consistent with these behavioral findings and the results in our study, a larger adaptation effect in the FFA is apparent for misaligned than for aligned composite images in which one half of the face is unchanged (Schiltz and Rossion, 2006).

Finally, we determined whether holistic response to faces could be influenced by the nonface context in which the face was presented. The importance of context has been shown in the response of the FFA to images (such as a body) that imply the presence of a face (Cox et al., 2004). However, it has been argued that these findings could be explained by activity of body-selective neurons in the fusiform gyrus (Peelen and Downing, 2005; Schwarzlose et al., 2005). To determine the role of “context,” we generated composite face images in which the face was unchanged, but the body features on which it was superimposed varied. In the FFA, there was a slight increase in response (i.e., release from adaptation) for this composite image compared to repetitions of the same face and body. However, the response was still significantly lower than stimulus blocks in which the face varied across successive presentations. We then determined the response to composite images in which the face varied, but the body was unchanged. This stimulus generated a response that was not significantly different from stimulus blocks in which different faces and bodies were shown. In other words, the FFA shows a holistic response to faces, but does not show a holistic response to different combinations of faces and bodies. Together, these results suggest that the neural representation in the face-selective regions such as the FFA is predominantly selective for faces.

Adaptation to repeated images of internal or external features of the face was evident in the FFA and OFA, but not in the STS. This fits with models of face processing that emphasize the difference between inferior temporal processes involved in facial recognition and superior temporal processes involved in understanding dynamic aspects of faces (Bruce and Young, 1986; Haxby et al., 2000; Hoffman and Haxby, 2000; Andrews and Ewbank, 2004; Winston et al., 2004; Ewbank and Andrews, 2008). The STS did, however, show a larger response to internal than to external features of the face. This is consistent with the importance of the internal features in providing cues about expression and gaze.

In conclusion, the response to isolated features of the face provide clear neurological support for the idea that the dominant role of internal features in the neural representation of familiar faces. However, the release from adaptation when viewing composite faces suggests that both the internal and external features are important in face perception. Finally, the absence of a holistic response between faces and nonface objects suggests that the FFA is primarily involved in processing information about faces.

Footnotes

This work was supported by a grant from the Wellcome Trust (WT087720MA). We thank Chris Alderson, Nick Arthur, Johan Carlin, Rick Chen, Gemma Cheney, Dettie Freedman, Andre Gouws, Anna Gebhardt, Rachel Harlow, Ama Hirsch, Anneka Holden, Ed Ingamells, and Helen Williams for their help during the initial stages of this project. J.D.-T. is supported by an Economic and Social Research Council studentship.

References

  1. Andrews TJ, Ewbank MP. Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe. Neuroimage. 2004;23:905–913. doi: 10.1016/j.neuroimage.2004.07.060. [DOI] [PubMed] [Google Scholar]
  2. Betts LR, Wilson HR Advance online publication. Heterogeneous structure in face-selective human occipito-temporal cortex. J Cogn Neurosci. 2010 doi: 10.1162/jocn.2009.21346. [DOI] [PubMed] [Google Scholar]
  3. Bruce V, Young A. Understanding face recognition. Br J Psychol. 1986;77:305–327. doi: 10.1111/j.2044-8295.1986.tb02199.x. [DOI] [PubMed] [Google Scholar]
  4. Burton AM, Jenkins R, Hancock PJB, White D. Robust representations for face recognition: the power of averages. Cogn Psychol. 2005;51:256–284. doi: 10.1016/j.cogpsych.2005.06.003. [DOI] [PubMed] [Google Scholar]
  5. Cox D, Meyers E, Sinha P. Contextually evoked object-specific responses in human visual cortex. Science. 2004;304:115–117. doi: 10.1126/science.1093110. [DOI] [PubMed] [Google Scholar]
  6. Ellis HD, Shepherd JW, Davies GM. Identification of familiar and unfamiliar faces from internal and external features. Perception. 1979;8:431–439. doi: 10.1068/p080431. [DOI] [PubMed] [Google Scholar]
  7. Ewbank MP, Andrews TJ. Differential sensitivity for viewpoint between familiar and unfamiliar faces in human visual cortex. Neuroimage. 2008;40:1857–1870. doi: 10.1016/j.neuroimage.2008.01.049. [DOI] [PubMed] [Google Scholar]
  8. Fairhall SL, Ishai A. Effective connectivity within the distributed cortical network for face perception. Cereb Cortex. 2007;17:2400–2406. doi: 10.1093/cercor/bhl148. [DOI] [PubMed] [Google Scholar]
  9. Freiwald WA, Tsao DY, Livingstone MS. A face feature space in the macaque temporal lobe. Nat Neurosci. 2009;12:1187–1196. doi: 10.1038/nn.2363. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Grill-Spector K, Kushnir T, Edelman S, Avidan G, Itzchak Y, Malach R. Differential processing of objects under various viewing conditions in human lateral occipital complex. Neuron. 1999;24:187–203. doi: 10.1016/s0896-6273(00)80832-6. [DOI] [PubMed] [Google Scholar]
  11. Grill-Spector K, Knouf N, Kanwisher N. The fusiform face area subserves face perception, not generic within-category identification. Nat Neurosci. 2004;7:555–562. doi: 10.1038/nn1224. [DOI] [PubMed] [Google Scholar]
  12. Hancock PJB, Bruce V, Burton AM. Recognition of unfamiliar faces. Trends Cogn Sci. 2000;4:330–337. doi: 10.1016/s1364-6613(00)01519-9. [DOI] [PubMed] [Google Scholar]
  13. Haxby JV, Hoffman EA, Gobbini MI. The distributed human neural system for face perception. Trends Cogn Sci. 2000;4:223–233. doi: 10.1016/s1364-6613(00)01482-0. [DOI] [PubMed] [Google Scholar]
  14. Hoffman EA, Haxby JV. Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nat Neurosci. 2000;3:80–84. doi: 10.1038/71152. [DOI] [PubMed] [Google Scholar]
  15. Leopold DA, O'Toole AJ, Vetter T, Blanz V. Prototype-referenced shape encoding revealed by high-level aftereffects. Nat Neurosci. 2001;4:89–94. doi: 10.1038/82947. [DOI] [PubMed] [Google Scholar]
  16. Maurer D, Le Grand R, Mondloch CJ. The many faces of configural processing. Trends Cogn Sci. 2002;6:255–260. doi: 10.1016/s1364-6613(02)01903-4. [DOI] [PubMed] [Google Scholar]
  17. Maurer D, O'Craven KM, Le Grand R, Mondloch CJ, Springer MV, Lewis TL, Grady CL. Neural correlates of processing facial identity based on features versus their scrambling. Neuropsychologia. 2007;45:1438–1451. doi: 10.1016/j.neuropsychologia.2006.11.016. [DOI] [PubMed] [Google Scholar]
  18. McNeil JE, Warrington EK. Prosopagnosia: a face-specific disorder. Q J Exp Psychol A. 1993;46:1–10. doi: 10.1080/14640749308401064. [DOI] [PubMed] [Google Scholar]
  19. Moscovitch M, Moscovitch DA. Super face-inversion effects for isolated internal or external features, and for fractured faces. Cogn Neuropsychol. 2000;17:201–219. doi: 10.1080/026432900380571. [DOI] [PubMed] [Google Scholar]
  20. O'Donnell C, Bruce V. Familiarisation with faces selectively enhances sensitivity to changes made to the eyes. Perception. 2001;30:755–764. doi: 10.1068/p3027. [DOI] [PubMed] [Google Scholar]
  21. Peelen MV, Downing PE. Selectivity for the human body in the fusiform gyrus. J Neurophysiol. 2005;93:603–608. doi: 10.1152/jn.00513.2004. [DOI] [PubMed] [Google Scholar]
  22. Pitcher D, Charles L, Devlin JT, Walsh V, Duchaine B. Triple dissociation of faces, bodies, and objects in extrastriate cortex. Curr Biol. 2009;19:319–324. doi: 10.1016/j.cub.2009.01.007. [DOI] [PubMed] [Google Scholar]
  23. Rotshtein P, Henson RN, Treves A, Driver J, Dolan RJ. Morphing Marilyn into Maggie dissociates physical and identity face-representations in the brain. Nat Neurosci. 2005;8:107–113. doi: 10.1038/nn1370. [DOI] [PubMed] [Google Scholar]
  24. Schiltz C, Rossion B. Faces are represented holistically in the human occipito-temporal cortex. Neuroimage. 2006;32:1385–1394. doi: 10.1016/j.neuroimage.2006.05.037. [DOI] [PubMed] [Google Scholar]
  25. Schwarzlose RF, Baker CI, Kanwisher N. Separate face and body selectivity on the fusiform gyrus. J Neurosci. 2005;25:11055–11059. doi: 10.1523/JNEUROSCI.2621-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Sinha P, Poggio T. I think I know that face. Nature. 1996;384:404. doi: 10.1038/384404a0. [DOI] [PubMed] [Google Scholar]
  27. Sinha P, Poggio T. ‘United’ we stand. Perception. 2002;31:133. doi: 10.1068/p3101no. [DOI] [PubMed] [Google Scholar]
  28. Tanaka JW, Farah MJ. Parts and wholes in face recognition. Q J Exp Psychol A. 1993;46:225–245. doi: 10.1080/14640749308401045. [DOI] [PubMed] [Google Scholar]
  29. Valentine T. A unified account of the effects of distinctiveness, inversion and race in face recognition. Q J Exp Psychol A. 1991;43:161–204. doi: 10.1080/14640749108400966. [DOI] [PubMed] [Google Scholar]
  30. Winston JS, Henson RNA, Fine-Goulden MR, Dolan RJ. fMRI-adaptation reveals dissociable neural representations of identity and expression in face perception. J Neurophysiol. 2004;92:1830–1839. doi: 10.1152/jn.00155.2004. [DOI] [PubMed] [Google Scholar]
  31. Yin RK. Looking at upside-down faces. J Exp Psychol. 1969;81:141–145. [Google Scholar]
  32. Young AW, Hay DC, McWeeny KH, Flude BM, Ellis AW. Matching familiar and unfamiliar faces on identity and expression. Perception. 1985;14:737–746. doi: 10.1068/p140737. [DOI] [PubMed] [Google Scholar]
  33. Young AW, Hellawell D, Hay DC. Configurational information in face perception. Perception. 1987;16:747–759. doi: 10.1068/p160747. [DOI] [PubMed] [Google Scholar]
  34. Yovel G, Axelrod V. Non-preferred stimuli modify the representation of preferred stimuli in the fusiform face area. Soc Neurosci Abstr. 2009;35:603–4. [Google Scholar]
  35. Yovel G, Kanwisher N. Face perception: domain specific, not process specific. Neuron. 2004;44:889–898. doi: 10.1016/j.neuron.2004.11.018. [DOI] [PubMed] [Google Scholar]
  36. Yovel G, Kanwisher N. The neural basis of the behavioral face-inversion effect. Curr Biol. 2005;15:2256–2262. doi: 10.1016/j.cub.2005.10.072. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES