The Fusiform Face Area: In Quest of Holistic Face Processing

What role do external facial features (hair, moustaches, beards, etc.) play in face recognition? Many of us have experienced the difficulty of recognizing a friend or a colleague who had changed her hairstyle or had shaved his beard. Such a change seems like not just a change in the facial hair, but rather the whole face looks different. This phenomenon of face processing is called holistic processing, meaning that we perceive the face as a whole and not as a set of separate, independently processed features. Holistic processing is generally accepted to be unique to faces and provides strong support for the notion that faces are processed differently relative to all other object categories (Farah et al., 1998). While there are a large number of fMRI studies exploring the neural mechanisms of face processing, few of them have focused directly on holistic processing mechanisms (e.g., Schiltz and Rossion, 2006). Therefore, the study of Andrews et al. (2010), who investigated holistic processing by manipulating external and internal (eyes, nose, and mouth) facial features, provides a valuable contribution to our understanding of the neural basis of holistic face processing.

Andrews et al. (2010) conducted three fMRI experiments to examine how external and internal features are represented in face-selective occipitotemporal brain areas. They used a block-design fMR-adaptation paradigm (Grill-Spector and Malach, 2001). This method is based on the finding that repeated presentation of the same stimulus results in reduced fMRI response. In the fMR-adaptation paradigm used by Andrews et al. (2010), each condition is represented by two types of blocks: completely identical pictures (same) and pictures that differ in a single given dimension (different). Stronger neural response to the different relative to the same blocks (release from fMR-adaptation) suggests that the region of interest is involved to some extent in processing the manipulated dimension. Conversely, equal neural response to different and same blocks (fMR-adaptation) suggests that the region of interest is not sensitive to the dimension manipulated in the different block. One should be aware, however, of the possible pitfalls in the interpretation of fMR-adaptation results (Bartels et al., 2008). For example, Sawamura et al. (2006) have shown that neural selectivity, which is frequently inferred from fMR-adaptation experiment results, might be inconsistent with neural selectivity measured in electrophysiology single-unit recording.

The research of Andrews et al. (2010) focused on three regions of interest: the fusiform face area (FFA), the occipital face area, and the superior temporal sulcus, all of which were localized in separate scans as face-selective regions. The behavioral task in all three fMR-adaptation experiments was to monitor occasional appearance of a red dot superimposed on the image (exact location of the dot was not specified by the authors).

The first experiment had three conditions, each with same and different stimulus blocks: faces with internal and external facial features (whole faces), faces with external facial features cropped (internal features), and images with only hair, ears, and face outline, with all internal facial features deleted (external features). The stimulus blocks (duration of 9 s) were interleaved by fixation gray screen blocks (duration of 9 s). In each stimulus block, there were nine images from one of the conditions. Presumably, all the images in the different blocks were different, though this was not specified explicitly by authors. The experiment was conducted using familiar faces, i.e., those of famous people, and unfamiliar faces. The goal of this experiment was to test what features face-selective regions are sensitive to. The authors report a strong release from fMR-adaptation in the FFA, not only for whole faces and internal features conditions, but also for external features [Andrews et al. (2010), their Fig. 2 and Fig. 3]. Previous studies have primarily focused on the processing of internal features, with external features usually excluded from the stimulus. The important finding of Andrews et al. (2010) is that the FFA can discriminate between faces based on external features only.

Next, Andrews et al. (2010) directly compared the neural responses for internal and external features. They found the following: (1) that the general level of activation was higher for internal than external features; and (2) that the fMR-adaptation effect was stronger for internal features than for external features, but only for familiar faces [Andrews et al. (2010), their Fig. 4]. The stronger fMR-adaptation for internal features demonstrates that despite sensitivity to external features, the FFA is more specialized in processing internal features. It is important, though, to understand the method that Andrews et al. (2010) used to compare fMR-adaptation levels (result 2). For each condition, the authors calculated the neural signal adaptation index as an absolute difference score: different − same. However, the standard way to calculate the magnitude of adaptation is a normalized measure (a ratio), for example, same/different (Grill-Spector and Malach, 2001), different/same (Kourtzi et al., 2003), or (different − same)/different (Sawamura et al., 2005). The normalized measure ensures that different general levels of fMRI signal across conditions do not confound the adaptation score. Given that the general activation levels differed between the conditions of internal and external features (result 1), applying a normalized adaptation score may have revealed a different pattern of results.

In their second experiment, Andrews et al. (2010) directly tested whether a change in the external features may influence perception of the internal features. The experiment had four conditions: completely identical blocks of faces (same internal, same external), completely different blocks of faces (different internal, different external), blocks with same internal face parts but with different face outline, hair, and ears (same internal, different external), and blocks with different internal face parts but with same outline (different internal, same external). The experiment design was identical to the experiment 1, and it was also conducted in two versions (familiar and unfamiliar faces). The results revealed that all three image conditions in which a feature differed showed release from fMR-adaptation relative to blocks in which identical images were presented [Andrews et al. (2010), their Fig. 5 and Fig. 6]. In addition, no difference was found between the neural responses of the three conditions in which different images were presented. The authors suggested that these results demonstrate holistic face processing of the external features. Although based on previous studies of the FFA region (e.g., Schiltz and Rossion, 2006) it is possible that external features are processed holistically, the findings of Andrews et al. (2010) could have an equally valid alternative nonholistic explanation, as discussed below.

The main advantage of the fMR-adaptation method is a higher sensitivity relative to standard fMRI study designs. For example, whereas in a standard fMRI experiment the faces of Marilyn Monroe and Margaret Thatcher elicit similar neural activations, only by using the fMR-adaptation method is it possible to differentiate between the neural responses elicited by viewing these two identities (Rotshtein et al., 2005). However, despite this high sensitivity, the method does not provide an explanation of why the region of interest was sensitive. Let us consider only two conditions of the Andrews et al. (2010) study: “same internal and external” and “same internal, different external.” The authors suggest that the release from fMR-adaptation in the FFA was a result of holistic processing of the external features. In other words, the faces with different outline, hair, and ears, but identical internal features, were perceived as different faces, and this perceptual phenomenon was a result of neural processing that took place in the FFA brain region. This may be a reasonable conclusion, since the faces indeed look different [similar to the composite effect shown by Young et al. (1987)], and at the neural level, the FFA is thought to play an important role in face recognition (for example, Rotshtein et al., 2005). However, in their first experiment Andrews et al. (2010) showed that the FFA successfully discriminated external features alone, without the involvement of any holistic processing. This means that we cannot determine whether the observed release from fMR-adaptation for different external features is the manifestation of the holistic face effect or sensitivity to external features per se. An additional possibility is that both these effects contributed independently rather than interactively (as a holistic hypothesis would predict) to the release from fMR-adaptation.

In their final experiment, Andrews et al. (2010) tested whether holistic perception of the face is influenced by context (the body). The experiment design was similar to the previous experiment, but instead of the external features condition, there was a bust condition [Andrews et al. (2010), their Fig. 7, top]. In this experiment, the duration of stimulus and fixation block was 10 s each, and the stimulus block consisted of 10 images from one of the conditions. This experiment was conducted using images of unfamiliar faces only. The FFA showed a pattern of activity similar to that of experiment 2, with one exception: the release from fMR-adaptation in the FFA for the same face, different bust condition was smaller than for the two conditions that included different faces [Andrews et al. (2010), their Fig. 7, bottom]. The authors concluded accordingly that the bust, as opposed to external facial features, does not influence holistic representation of the face. But an alternative explanation to these findings is that the lower response for the same face, different bust condition results from effects of attention. That is, it is possible that the subjects simply did not attend to the lower part of the image that included the bust. For example, it was recently shown that faces attract attention even when they are irrelevant to the task (Cerf et al., 2009). Since we cannot estimate the extent of this attentional bias, there is no way to know how fMR-adaptation response in the same face, different bust condition would have looked without this bias.

In summary, holistic face processing has been a focus of numerous behavioral studies in the past few decades (Young et al., 1987) (for more references, see Andrews et al., 2010), and it is thought to be one of the core components of face perception in general (Farah et al., 1998). Recent development of brain imaging methods has enabled the exploration of neural correlates of these processes. The research of Andrews et al. (2010) provides a new and interesting way to study neural mechanisms of holistic face processing by manipulating external and internal face features. Future studies are required to better understand the nature of the holistic representation of the face and its neural manifestation.