What does a “face cell” want?’

Abstract

In the 1970s Charlie Gross was among the first to identify neurons that respond selectively to faces, in the macaque inferior temporal (IT) cortex. This seminal finding has been followed by numerous studies quantifying the visual features that trigger a response from face cells in order to answer the question; what do face cells want? However, the connection between face-selective activity in IT cortex and visual perception remains only partially understood. Here we present fMRI results in the macaque showing that some face patches respond to illusory facial features in objects. We argue that to fully understand the functional role of face cells, we need to develop approaches that test the extent to which their response explains what we see.

Understanding the neural mechanisms of face perception in the primate brain remains a major goal in neuroscience. The discovery of “face cells” in the inferior temporal (IT) cortex of macaques was a critical turning point in progress towards revealing the neural underpinnings of visual perception that continues to influence the focus of the field today (C. Bruce, Desimone, & Gross, 1981; Gross, Rocha-Miranda, & Bender, 1972). Initial investigations of the cortical visual system revealed hierarchical stages of processing. Single neurons in V1 were discovered to respond to simple visual stimuli such as oriented bars of light, with neurons in higher-order visual areas responding to increasingly more complex visual features such as direction-of-motion and binocular disparity (Hubel & Wiesel, 1962, 1974). In 1972, Charlie Gross and colleagues serendipitously discovered a more complex form of feature tuning in IT cortex (Gross et al., 1972). During a recording session, while failing to drive a neuron they had isolated, a member of the research team inadvertently waved a hand in front of the monkey and elicited a strong response from the neuron. Many hours of probing with different stimuli revealed that this neuron was selectively tuned to a detailed cut-out of a monkey hand, oriented in a direction consistent with the monkey looking at its own hand. Thus, unlike neurons in primary visual cortex known to respond to low-level visual properties such as oriented edges, the response of this neuron appeared to be correlated with high-level perception. This ground-breaking discovery marked the beginning of the on-going quest to distill the nature of the visual properties that trigger brain activity in IT cortex (Gross, 2005).

Nearly a decade later in 1981, Charlie Gross and colleagues reported finding seven “face cells” in the superior temporal sulcus (STS) of the rhesus macaque and interrogated their tuning properties in an attempt to better understand their role in face perception (C. Bruce et al., 1981; see Figure 1). They measured how the neurons responded to modified faces with the eyes removed, monkey versus human faces, and cartoon faces. Although these manipulations reduced the neuronal firing rate, the response was not eliminated until a face stimulus was physically cut up into smaller pieces and presented in a scrambled format, suggesting that these neurons were tuned to whole faces per se, rather than a particular local feature (C. Bruce et al., 1981; Desimone, Albright, Gross, & Bruce, 1984; Perrett, Rolls, & Caan, 1982). Direct empirical support for the hypothesis that the response of face cells in IT cortex underlies the perception of faces followed many years later in 2006 with the demonstration of a causal link between brain activity and face categorization behavior (Afraz, Kiani, & Esteky, 2006; also see Sadagopan, Zarco, & Freiwald, 2017). While macaques performed a face / nonface categorization task on noisy visual stimuli, the researchers used a micro-stimulation technique to artificially increase the firing rate of neurons at a site of face-selective multiunit activity in IT cortex. They found that triggering face cells increased the likelihood that the monkeys would categorize an ambiguous noise stimulus as a face, demonstrating a link between neuronal firing and behavior. This result is consistent with the idea that the excitation of face cells reflects the perception of a face in the visual environment. However, despite substantial progress in characterizing the tuning of face cells, the question of “What does a face cell want?” remains a puzzle in the field, to a large degree because the answer is turning out to be more complex than initially assumed.

Figure 1.

A historical overview of the stimuli that have successfully driven face cells (i.e. examples of the “best” stimuli from single cell investigations) with examples of their non-preferred counterparts (i.e. examples of the “worst” or “control” stimuli). Dashed line indicates that the study measured brain activity at the population level using fMRI.

At the single-unit level, the empirical approach of systematically manipulating face stimuli to observe the effects on firing rate has been frequently used with the aim of revealing the critical features for face perception (see Figure 1). For example, many face cells are tuned for both contrast polarity and the geometry of facial features (W. A. Freiwald, Tsao, & Livingstone, 2009; Leopold, Bondar, & Giese, 2006; Ohayon, Freiwald, & Tsao, 2012). An intriguing observation is that face cells often respond (although to a lesser degree) to particular non-face objects such as fruit or clocks that share a visual feature with faces, e.g., a round shape (Tsao, Freiwald, Tootell, & Livingstone, 2006). This observation points to a critical question about the interpretation of neuronal firing rate: what does it mean when a neuron responds to a visual stimulus, but not maximally? For example, face cells fire at a slower rate when their preferred stimulus, a face, is turned upside down (Taubert, Van Belle, Vanduffel, Rossion, & Vogels, 2015a, 2015b; Taubert, Van Belle, Vogels, & Rossion, 2018; Tsao, Freiwald, Tootell, & Livingstone, 2006). Yet, an upside-down face is still perceived as a face. This disconnect between the downregulation of neuronal activity and a limited effect on behavior reveals the current limitations in our understanding of what these neurons want, and how their tuning properties influence perception and behavior.

The empirical efforts to understand what face cells want at the single-unit level share a common approach that begins with an assumption about the best category of stimuli (i.e. faces) for driving the firing rate of face cells. A consequence of this method is that our understanding of what a face cell wants is highly constrained (see Figure 1). Further, this approach also assumes that face cells have a direct role in our ability to perceive faces. A complementary approach considers whether these cells are also responsible for the times we see facial features, when none actually exist. We have recently used face pareidolia as one step towards this aim (Taubert et al., 2018; Taubert, Wardle, Flessert, Leopold, & Ungerleider, 2017). Face pareidolia is the common misperception of illusory facial features on an otherwise inanimate object. It can be thought of as a natural error of face detection (see Figure 2A). Human and nonhuman primates readily respond to these non-face objects as if they are faces, yet these are false positives of detection and thus potentially revealing about the broad sensitivity of face cells. Our initial question was whether this illusory perceptual experience is associated with face-selective activity in IT cortex.

Figure 2:

(A) Illustrative examples of illusory faces and matched non-face objects. (B) Four male monkeys were trained to watch blocks of stimuli in a standard on/off block design. (C) Illusory Face Index values were calculated for all voxels in 4 animals (Illusory Face Index = average response to illusory faces – average response to matched objects/average response to illusory faces + average response to matched objects). (D) Whole-brain data for one subject projected onto a partially inflated cortical surface (Seidlitz et al., 2018). Face patches in the Superior Temporal Sulcus (left hemisphere) are outlined in bright green and labelled (Tsao, Freiwald, Knutsen, Mandeville, & Tootell, 2003). In the independent localizer experiment, the contrast used to identify the face patches along the STS (i.e. {monkey faces + human faces} > {objects + scenes + scrambled human faces + scrambled monkey faces}, with a statistical threshold set at p = 1×10⁻¹¹) also yielded face patches in the frontal cortex (outlined bright green). Abbreviations: STS, superior temporal sulcus; PL, posterior lateral face patch; ML, middle lateral face patch; AL, anterior lateral face patch; MF, middle fundus face patch; AF, anterior fundus face patch. Red color indicates the location of voxels with an IFI value = 1.0.

This question has to be asked at the system level rather than at the single-unit level because, given the dual identity of these stimuli as faces and objects, it is not clear whether the face-selective or object-selective regions of IT cortex will respond more to illusory faces compared to matched non-face objects (see Figure 2A). In the context of mapping face-selective activity in IT cortex, an important advance followed the application of functional MRI to awake nonhuman primates. Functional imaging uncovered that face cells in the macaque brain cluster within six spatially-distinct regions, the “face patches” (Baylis, Rolls, & Leonard, 1987; Logothetis, Guggenberger, Peled, & Pauls, 1999; Tsao, Freiwald, Knutsen, Mandeville, & Tootell, 2003). This observation is consistent with a popular model of face perception that proposes a distributed function across multiple regions (V. Bruce & Young, 1986; Haxby, Hoffman, & Gobbini, 2000). This proposal was an important advance in the study of IT cortex neurons, as it suggested that the anatomical location of a face cell might account for its response profile and that different populations of face cells might be triggered by different visual properties. For example, while the face patches on the lateral edge of the superior temporal sulcus (STS) are thought to be involved in achieving viewpoint invariance (W. A. Freiwald & Tsao, 2010) and building robust representations of facial identity (Chang & Tsao, 2017), those in the fundus are engaged by changes in facial expression and dynamic cues (Furl, Hadj-Bouziane, Liu, Averbeck, & Ungerleider, 2012; Hadj-Bouziane, Bell, Knusten, Ungerleider, & Tootell, 2008). This separation or dual pathway model fits with the numerous behavioral studies in both humans (Bate & Bennetts, 2015) and monkeys (Taubert, Flessert, Liu, & Ungerleider, 2019) providing evidence that the recognition of facial expression is independent of the recognition of facial identity. Here, we use functional magnetic resonance imaging to determine whether activity in the face patch system is evoked by the presentation of face pareidolia stimuli to rhesus monkeys.

To identify whether an illusory face in an object modulates activity in the macaque face patches, we scanned four nonhuman primates (Macaca mulatto) in an MRI scanner while they viewed blocks of stimuli consisting of illusory faces in objects or matched object sets without faces (Figure 2A). As the response of the primate visual system to illusory faces in objects is not known, we used whole-brain MRI in order to reveal sensitivity across all of the face patches and compare this with the response in object-selective regions. We defined face- and object-selective voxels in each animal based on an independent localizer using the following contrasts; face-selective = ({monkey faces + human faces} > {objects + scenes + scrambled human faces + scrambled monkey faces}) and object-selective = ({objects} > {monkey faces + human faces + scrambled human faces + scrambled monkey faces}). In all four subjects, object-selective voxels responded more to matched non-face objects than to illusory faces, indicating that the appearance of illusory facial features on an otherwise inanimate object suppresses the population response of object-selective voxels (Figure 2C). A small number of face-selective voxels had an illusory face index of 1.0, indicating that they were activated exclusively by illusory faces (Figure 2C). These voxels were located within the face-selective patch system on the lower lateral edge of the STS (areas AL and ML; see Figure 2D). Therefore, a real face is sufficient but not necessary to engage face-selective voxels. Notably, illusory faces did not engage the face patches in the fundus of the STS.

This observation at the system level shows that illusory facial features engage the lateral edge patches (ML and AL) but not others, including the fundus patches (MF and AF). This indicates that the visual properties that excite one population of face cells will not necessarily excite another, a result consistent with those showing a distinction between the fundus and the lateral edge patches (W. Freiwald, Duchaine, & Yovel, 2016; Furl et al., 2012; Hadj-Bouziane et al., 2008). Further, the finding that the lateral edge face patches respond more to illusory faces than matched objects demonstrates that these cell populations are not tightly tuned to the visual properties that typically define the faces of conspecifics, such as skin color or the round shape of a face. Instead, these lateral edge patches respond to both the real faces of conspecifics and illusory facial features in objects, demonstrating a high degree of tolerance to variance in the visual properties that constitute a face.

The whole brain data also revealed a region in ventrolateral prefrontal cortex (VLPFC) that was activated more by objects with illusory facial features than non-face objects (see Figure 2D). This result indicates that the frontal cortex may contribute to the false detection of illusory facial features in examples of face pareidolia. Although face patches have been identified within the ventrolateral, orbitofrontal and pre-arcuate regions of frontal cortex (Haile, Bohon, Romero, & Conway, 2019; Tsao, Schweers, Moeller, & Freiwald, 2008) their relationship to the face patches in ITC and their impact on the perception of faces is poorly understood. For example, although tracer studies have found anatomical connections between ITC and VLPFC, it is not yet clear whether these connections are bidirectional (Grimaldi, Saleem, & Tsao, 2016) or specific to face patches (Saleem, Miller, & Price, 2014). Therefore, while it is possible that activity in frontal cortex can upregulate the response of face cells in visual cortex via feedback connections, this remains speculative. By extension, until more is understood about the connectivity between the face patches in ITC and other neural structures, it is difficult to know whether the differential response to illusory faces reported in Figure 2D originated in ITC or somewhere else in the macaque brain.

The finding that some face patches respond to illusory facial features in examples of face pareidolia align with the results of another recent study showing that facial structure is sufficient but not necessary to elicit a response from face cells. Faces cells were found to respond to combinations of objects that would normally predict a face, even when the face was absent (i.e. a hat above shoulders; Arcaro, Ponce, & Livingstone, 2019). Therefore, face cells are not passive detectors of a particular constellation of low-level visual characteristics. Rather, face cells in the macaque face patch system can infer the presence of a face from the association with other objects (also see Martin, 2016). Together with the results of the imaging experiment presented here, these recent findings give new insight into the potential function of the face patches in the macaque brain. The fact that populations of face cells respond to visual stimuli in which the context is suggestive of faces and to illusory faces is indicative of their broad tuning. This broad tuning of the primate face system is likely adaptive. For example, a system that is tightly tuned to respond to specific input is likely to miss faces in a cluttered natural visual environment. In contrast, a system that is highly sensitive to any feature combination or contexts that might indirectly indicate that a face is present is less likely to miss a face. Therefore, the observation that face cells clustering on the lower lateral edge of the superior temporal sulcus respond to real, illusory and absent faces suggests that these cells may be a responsible for the speed and ease with which we detect faces in the environment (Crouzet, Kirchner, & Thorpe, 2010; McKone, 2004; Taubert, Apthorp, Aagten-Murphy, & Alais, 2011) and the well-known visual preference for faces shared among primates (Goren, Sarty, & Wu, 1975; Elizabeth A. Simpson, Maylott, Mitsven, Zeng, & Jakobsen, 2020; E. A. Simpson, Paukner, Pedersen, Ferrari, & Parr, 2019; Sugita, 2008; Taubert, Wardle, Flessert, Leopold, & Ungerleider, 2017; Vinken & Vogels, 2019). This highlights that further examination of the variance in what face cells respond to is a fruitful avenue for advancing our understanding of the role of the temporal cortex in visual perception and primate behavior.

Highlights.

• We review the evidence that face cells respond more to faces than objects

• We show evidence that face patches respond to objects with illusory facial features

• This approach connects the response of face cells to visual perception