Neuronal Mechanisms Underlying Face Recognition in Non-human Primates

Abstract

Humans and primates rely on visual face recognition for social interactions. Damage to specific brain areas causes prosopagnosia, a condition characterized by the inability to recognize familiar faces, indicating the presence of specialized brain areas for face processing. A breakthrough finding came from a non-human primate (NHP) study conducted in the early 2000s; it was the first to identify multiple face processing areas in the temporal lobe, termed face patches. Subsequent studies have demonstrated the unique role of each face patch in the structural analysis of faces. More recent studies have expanded these findings by exploring the role of face patch networks in social and memory functions and the importance of early face exposure in the development of the system. In this review, we discuss the neuronal mechanisms responsible for analyzing facial features, categorizing faces, and associating faces with memory and social contexts within both the cerebral cortex and subcortical areas. Use of NHPs in neuropsychological and neurophysiological studies can highlight the mechanistic aspects of the neuronal circuit underlying face recognition at both the single-neuron and whole-brain network levels.

Over the last 40 years, many studies have revealed the neuronal mechanisms of face processing systems using non-human primates (NHPs), especially macaque monkeys. These findings offer several insights into face recognition systems. Here, we begin with an overview of visual and face processing in macaques compared to that in humans. We then review face processing in the cerebral cortical areas, contrasting the two functional routes that process the identity and status of faces. Next, we review the function of subcortical regions in terms of face processing modulated by emotion and value memory. Finally, we discuss several candidates for face processing networks based on these findings and the potential future directions for unraveling face processing in primates.

Face Processing in Non-Human Primates

Similar Face Processing Strategies Between Humans and Macaques

Face recognition is crucial for animals such as humans who live in social groups, as it helps us to identify each other and understand others’ emotional states. Macaque monkeys have been used to study the neuronal mechanisms underlying visual processing because their visual systems are similar to those of humans (Felleman & Van Essen, 1991; Horwitz, 2015). Though there is little empirical support for the claim that macaques can naturally recognize the individual faces of conspecifics (Rossion & Taubert, 2019), it has been reported that they can recognize individual faces (Schell et al., 2011) and distinguish emotional states based on facial expressions (Liu et al., 2023). Humans and macaques share common face recognition characteristics. They both show better discrimination performance on novel faces of conspecifics than on those of other species (i.e., species-specific effect) (Dufour et al., 2006; Pascalis & Bachevalier, 1998), better performance on upright faces than on inverted faces (i.e., face inversion effect; a schematic diagram of experimental setting in Figure 1) (Dahl et al., 2009; Tomonaga, 1994; Vermeire & Hamilton, 1998), and perceive inanimate objects with facial features as faces (i.e., face pareidolia) (Taubert et al., 2017). These similarities suggest that there are face processing strategies that are common between humans and macaques.

Figure 1.

Face image presentation

Note. The inverted face image is presented on a monitor display for a macaque subject.

Advantages of Studies with Macaques

Macaque monkeys offer three key advantages as subjects for investigating face perception and provide insights that may be challenging for humans and apes to obtain. First, studies on macaques have allowed us to explore how environmental factors during development affect the ability to perceive faces. For example, face deprivation studies that conducted hand-rearing of infant monkeys with no exposure to faces showed that exposure to faces is crucial for acquiring face discrimination ability and face-looking behavior (Arcaro et al., 2017; Sugita, 2008). Second, studies in macaques are valuable for understanding face processing systems at the single-neuron level. Electrophysiological studies conducted in the 1980s revealed face-selective neurons in the inferior temporal (IT) cortex (Bruce et al., 1981; Perret et al., 1982; Yamane et al., 1988). More recent electrophysiological studies from the 2000s have revealed the properties of face-selective neurons from particular face-selective regions identified by functional magnetic resonance imaging (fMRI) (Landi et al., 2021; Tsao et al., 2006). Further, longitudinal neuronal recordings using chronically implanted microwire electrodes enable us to track the dynamic changes in each identical face-selective neuron daily (Koyano et al., 2023; McMahon et al., 2014). Third, studies in macaques are important to address brain regions and circuits that affect face processing through neuronal manipulation. For example, reversible suppression of a face-selective region (i.e., face patch) in the IT cortex reduces face gender discrimination performance (Afraz et al., 2015), and electrical microstimulation of face patches affects the perception of facial identity (Moeller et al., 2017). In the subcortical areas, specific lesions of the amygdala reduce attention to threatening faces (Dal Monte et al., 2015). Recent advances in genetic tools have enabled cell- or pathway-selective manipulation of macaque brains using optogenetic or chemogenetic manipulation (Oyama et al., 2021; Tremblay et al., 2020). These tools can be applied to the pathway-selective manipulation of face processing circuits in the near future.

Visual Object Processing in the Macaque Brain

The temporal cortex, which is a disproportionately expanded cortical area in primates, evolved in early anthropoid primates (Braunsdorf et al., 2021). In primates, visual object recognition is processed through the V4, TEO, and TE areas in the IT cortex, and visual object information is sent to the caudate tail (CDt) and putamen tail (PUTt) in the basal ganglia (Figure 2; Kemp & Powell, 1970; Kravitz et al., 2013; Saint-Cyr et al., 1990). Subsequently, this information is transmitted to the superior colliculus (SC) for saccadic eye movements toward a visual object (Amita et al., 2020; Hikosaka et al., 2000). Face recognition is processed via the same ventral visual pathway, although face information is mainly represented in multiple face-specialized regions of the cerebral cortex. This basic functional organization for face processing shares common properties between humans and macaques (Tsao, Moeller et al., 2008), supporting the applicability of findings from macaque studies for understanding the human face identification process. However, it remains unclear how these face patches interact with subcortical areas (i.e., the basal ganglia, thalamus, amygdala, hippocampus, and superior colliculus) for social communication. Recent studies have revealed that face familiarity is acquired in the IT cortex based on the number of days of exposure to faces (Koyano et al., 2023) and that there are face familiarity neurons with object value memory in the CDt and PUTt (Kunimatsu et al., 2023). The face familiarity acquired through the cortical face network may be modulated by the subcortical value network for gaze shifts toward attractive faces, similar to the object value system (Hikosaka et al., 2014).

Figure 2.

Visual-oculomotor network in primates

Note. Visual object information is processed from the primary visual cortex (V1) to the superior colliculus (SC) through the inferior temporal cortex (TEO/TE) and the basal ganglia (CDt/PUTt).

Face Processing in the Cerebral Cortex

Cortical Face Patches

Primate brains, including those of humans, contain face-selective areas specifically dedicated to processing faces distributed across various brain regions (Deen et al., 2023; Hesse & Tsao, 2020a; Leopold & Park, 2020). In the 1980s, electrophysiological studies involving NHPs identified visually responsive neurons in the IT cortex that were selective for various categories, including faces (Baylis et al., 1985; Bruce et al., 1981; Desimone et al., 1984; Perrett et al., 1982). A subsequent optical imaging study revealed clusters of face-selective cells within a small region of the IT cortex (Wang et al., 1996). Following fMRI studies found face-selective areas distributed across the IT cortices of primates, including those of humans (Kanwisher et al., 1997), macaques (Logothetis et al., 1999; Tsao et al., 2003), and marmosets (Dureux et al., 2023; Hung et al., 2015). Studies have suggested the functional and anatomical homology of these face-selective areas between macaques and humans (Bell et al., 2009; Janssens et al., 2014; Pinsk et al., 2009; Rajimehr et al., 2009; Tsao, Moeller et al., 2008; Yovel & Freiwald, 2013), although it is still under investigation whether there is a clear one-to-one correspondence. Face-selective neurons frequently respond to both heterospecific and conspecific faces. However, conspecific faces often elicit faster responses (Kiani et al., 2005), and the species boundary tends to shift toward the own species (Sigala et al., 2011). Within a face area, neurons that prefer either conspecific or heterospecific faces form contiguous clusters (Sato et al., 2013).

In macaques, these face-selective regions are termed “face patches” (Moeller et al., 2008; Tsao, Moeller et al., 2008; Figure 3). The functionally defined face patches have been consistently located at specific positions in the cortex, displaying similarity across individual macaques. A detailed analysis of anatomical features indicates that face patches form at specific locations relative to the anatomical landmarks of cortical bumps (Arcaro, Mautz et al., 2020) and that the positioning of face patches is consistently adjacent to color (Lafer-Sousa & Conway, 2013) and body patches (Bell et al., 2009; Pinsk et al., 2005). Studies using fMRI-guided electrophysiological recordings have revealed dense clustering of face-selective cells within face patches (Aparicio et al., 2016; Tsao et al., 2006). The density of these face-selective neurons is the highest at the center of the face patch, especially for excitatory responsive neurons (Bell et al., 2011), and decreases gradually over a few millimeters toward the periphery of the face patch (Aparicio et al., 2016). Electrical microstimulation of the cluster of face-selective IT neurons elicits a bias toward the face during face/non-face categorization of object images (Afraz et al., 2006).

Figure 3.

Face network in the cerebral cortex

Note. There are multiple face patches distributed in the inferior temporal (IT) and frontal cortex. PL, posterior lateral; ML, middle lateral; MF, middle fundus; MD, middle dorsal; AL, anterior lateral; AF, anterior fundus; AD, anterior dorsal; AM, anterior medial; PRh, perirhinal; TP, temporal pole; PA, prefrontal arcuate; PV, prefrontal ventral; PO, prefrontal orbital.

Core Face Patches in the Inferior Temporal Cortex

Six distinct face patches in the IT cortex have been consistently identified in various studies (Tsao, Moeller et al., 2008; Moeller et al., 2008). These patches, labeled as posterior lateral (PL), middle lateral (ML), middle fundus (MF), anterior lateral (AL), anterior fundus (AF), and anterior medial (AM) (Figure 3), constitute a “core” system for the visual feature analysis of faces (Deen et al., 2023; Landi & Freiwald, 2017). They are hierarchically organized along the anterior-posterior axis, reflecting the structure of the ventral visual pathway. In the posterior regions, faces are encoded as view-specific representations that become more view-invariant and identity-selective in the anterior regions (Eifuku et al., 2004, Freiwald & Tsao, 2010; Meyers et al, 2015). The anterior face patches are particularly adept at representing visual categories and face identities (Liu et al., 2013), integrating temporal contexts (Russ et al., 2023), and serving as sources of top-down prediction error signals along a hierarchy (Issa et al., 2018; Schwiedrzik & Freiwald, 2017). An inactivation study showed that anterior face patches (AM/AF) require inputs from middle face patches (ML/MF) and that top-down signals from anterior face patches affect face selectivity in middle face patches (Liu et al., 2022).

These core face patches are located in the IT region, where the visual receptive field is large and no clear retinotopic organization is present. Neurons in AF face patches exhibit consistent responses to naturalistic movies during free viewing, a condition that does not require gaze fixation and results in varied eye movement patterns across trials (McMahon et al., 2015). This phenomenon can be partially attributed to the larger receptive fields of the neurons. However, evidence suggests that face patches also have contralateral and foveal biases, especially those located at the posterior position of the IT cortex. Neuroimaging studies have shown that PL face patches are located in the area TEO (Moeller et al., 2008), where retinotopy is present (Janssens et al., 2014). Consistent with these studies, the posterior face patches can more effectively decode facial positions (Taubert, Wardle, Tardiff, Patterson et al., 2022), indicating the existence of a retinotopic structure. Electrophysiological recordings have revealed response biases of individual neurons in the PL and ML face patches to the contralateral visual field (Issa & DiCarlo, 2012). When ML activity is suppressed, neglect of the contralateral eye is induced during the free viewing of face images (Azadi et al, 2024), and gender discrimination ability is impaired only on the contralateral side (Afraz et al., 2015). In addition to these biases in the contralateral visual field, foveal bias has been observed in ML face patches (Janssens et al., 2014; Rajimehr et al., 2014). This aligns with the reduction in face selectivity when a face is presented in the peripheral visual field with a distractor stimulus (Taubert, Wardle, Tardiff, Patterson et al., 2022). Considering the innate primate behavior of face-gazing, the foveal response bias of face patch neurons is understandable (Arcaro & Livingstone, 2021). Recent electrophysiological studies have further supported this idea by mapping the receptive fields of face patch neurons using rapid serial visual presentation (Arcaro, Ponce et al., 2020) and naturalistic movie presentation paradigms (Xiao et al., 2024).

Psychophysical studies in human participants have emphasized the importance of both local features, such as the eyes and mouth, and holistic features, such as the geometrical configuration of facial parts, in face recognition (Tanaka & Simonyi, 2016). Correspondingly, neurons in core face patches have been reported to be selective for both local and holistic features. Neurons in the PL/ML (Issa & DiCarlo, 2012), ML/MF (Freiwald et al., 2009), AM/AF (Waidmann et al., 2022), and AM face patches (Landi et al., 2021) are selective for local facial features, such as the eyes. Part-based processing has also been reported for pareidolia stimuli in the central (CIT) and anterior IT (AIT) areas (Sharma, Vinken et al., 2023). In addition, neurons in ML/MF patches are also selective for holistic facial features (Freiwald et al., 2009) and contrast patterns of the whole face (Ohayon et al., 2012). Facial inversion, which disrupts holistic processing, reduces the responses of face-selective neurons and alters face tuning (Freiwald et al., 2009; Sugase-Miyamoto et al., 2014; Taubert et al., 2015b; Taubert, Van Belle et al., 2018). The information on individual facial parts can be combined using nonlinear integration (Li & Chang, 2023) and spike correlations (Hirabayashi & Miyashita, 2005). Some studies have suggested differential roles of holistic processing across face patches. A study using swapping facial parts showed part-based tuning in AM and AF patches, whereas there was no representation of the combinational features of each part (Waidmann et al., 2022). Consistently, a variant of the inverted face, the Thatcher illusion effect, is found in the ML patch (Taubert et al., 2015a), while the effect is weaker in the AIT (Sugase-Miyamoto et al., 2014) and AL face patch (Taubert et al., 2015a). Therefore, holistic features may be selective in the middle face patches rather than in the anterior face patches. The holistic encoding and configurational effects were also observed in face patches when faces were presented in the context of body parts. Face patches show whole-agent selectivity (Fisher et al., 2015b) and a preference for naturalistic face-body configurations (Zafirova et al., 2022; Zafirova et al., 2024). ML and PL neurons respond to occluded faces and non-face objects when presented with bodies (Arcaro, Ponce et al., 2020).

Core face patches were generally found in similar locations across individual animals. This aligns with the idea of proto-architecture in the IT cortex, which suggests a common blueprint for neuronal organization (Srihasam et al., 2014). Core face patches tend to be found at specific locations relative to the bump of the superior temporal sulcus (Arcaro, Mautz et al. 2020). During infancy, particularly the first year of development, face patches gradually emerge in the IT cortex (Livingstone et al., 2017). Face experience during this period is necessary for face patch development (Arcaro et al., 2017). Face deprivation during early development impairs face patch formation and results in abnormal face-looking behavior (Arcaro et al., 2017; Sugita, 2008), similar to that of an adult monkey that naturally lacks face patch activation (Vinken et al., 2019).

The core face patches are strongly connected to each other. This was observed as direct projections with an anatomical tracer (Grimaldi et al., 2016), as effective connectivity by means of fMRI signals increasing with microstimulation (Moeller et al., 2008, Premereur et al., 2016), and as functional connectivity calculated from signal correlations during resting-state fMRI (Schwiedrzik et al., 2015). A majority of visual inputs from earlier visual areas (V2, V3, V4v, and other parts of the TEO) first enter the PL face patch, which serves as a gateway to core face patches (Grimaldi et al., 2016). Although the cortical layer analysis of the projections showed a mixture of feedback and feedforward projections within the core face patch network (Grimaldi et al., 2016), an inactivation study suggested that visual signals propagated from the middle to the anterior face patches through two different streams (Liu et al., 2022). Given the general architecture of the ventral visual pathway (Figure 2) and the functional hierarchy of the face patches (Freiwald & Tsao, 2010), it is logical to assume a posterior–anterior signal flow, which propagates from the PL face patches, then the ML/MF patches and the AL/AF/AM patches. The AM and AL face patches are also connected to the anteromedial temporal areas, including the perirhinal and parahippocampal cortices (Grimaldi et al., 2016; Schwiedrzik et al., 2015), which are thought to be important for memory processing. Functional connectivity with frontal face patches was shown in a resting-state fMRI study (Schwiedrzik et al., 2015), though anatomical tracing showed only weak connections from the frontal face patches to the lateral IT core face patches (Grimaldi et al., 2016). Frontal face patches may connect heavily with the dorsal part of the IT core face patches, which have not yet been systematically investigated in tracer studies.

The anterior face patches connect heavily with the amygdala (Grimaldi et al., 2016; Moeller et al, 2008), which is important for facial expressions (Leonard et al., 1985). Many face patches are also reciprocally connected to the pulvinar (Grimaldi et al., 2016; Kagan et al., 2021; Moeller et al, 2008), which show a selective response to faces (Maior et al., 2010; Nguyen et al., 2013). Connections with the claustrum have also been reported in multiple studies (Grimaldi et al., 2016; Moeller et al, 2008). Resting-state fMRI studies have revealed additional brain regions whose activities correlate with those of face patches (Schwiedrzik et al., 2015; Zaldivar et al., 2022). These regions include the superior temporal sulcus (STS), lateral geniculate nucleus (LGN), parietal regions, premotor regions (including F4 and F5), hippocampus, lateral prefrontal areas (including areas 12 and 46), caudate nucleus, insula, mediodorsal thalamus, and substantia nigra (SN). These “functionally” connected areas may not be directly connected with face patches but may influence each other through a network of the brain. The findings of a resting-state study that utilized simultaneous recordings of single-cell and whole-brain fMRI signals suggest that each neuron in the face patch shows different signs of functional connectivity with the cortical and some subcortical regions (Zaldivar et al., 2022); this indicates that the relationships between face patches and subcortical regions are more complicated than previously thought. A study that correlated single-neuron responses with whole-brain fMRI signals during naturalistic movie watching further revealed that face-patch neurons can be categorized into a few groups according to their functional connectivity patterns (Park et al., 2017). These groups of neurons are distributed across multiple face patches but are intermixed with other groups at each local face patch, suggesting parallel subnetworks across the face patches (Park et al., 2022).

Although neurons in the core face patches are highly responsive to faces compared to non-face objects, it is worth carefully considering whether these neurons are purely selective to real faces. They may be responsive to face-like stimuli and not to faces per se. Visually selective IT cortices can be classified according to the statistical features of visual objects (Bao et al. 2020). According to this idea, faces can be considered as a category of stubby-shaped animate objects. Artificial intelligence (AI)-generated optimal images for face cells contain face-like visual features but not face itself (Bardon et al., 2022). Microstimulation of face patches disrupts identity discrimination not only for faces, but also for non-face objects that contain face-like features (Moeller et al., 2017). Some face patches are responsive to pareidolia stimuli, which are objects that resemble faces (Taubert, Wardle, Tardiff, Koele et al., 2022; Sharma, Vinken et al., 2023. see also Taubert et al., 2017 for behavioral results). Furthermore, a recent study directly demonstrated the encoding of non-face object information in face patches (Vinken et al., 2023). Taken together, face patches in the IT cortex often respond to face-like features, even if it is not a real face, although the response is generally higher for real faces than for ordinary objects.

Ventral Feature Analysis System

The core face patches, particularly those located on the lateral sides, were considered for the feature-based analysis of the visual face structure. Here, we call this the “ventral system,” which includes PL, ML, AL, and AM face patches, according to the dorsal/ventral two-face pathway model of face processing (Liu et al., 2022; Pitcher & Ungerleider, 2021). The PL face patch is located at the most posterior part of this system and at the start of processing; it receives visual inputs from the earlier visual areas V2, V3, and V4 (Grimaldi et al., 2016). The details of the selectivity in the PL patch require further research, but one study showed a strong preference for the contralateral eye in the PL patch (Issa & DiCarlo, 2012). Subsequently, the information is thought to be further processed in other face patches, including the ML patch, which also receives direct inputs from earlier visual areas (Grimaldi et al., 2016). ML neurons also exhibit a preference for the contralateral eye (Issa & DiCarlo, 2012), and inactivation of this area induces the neglect of the contralateral eye during free viewing of faces (Azadi et al, 2024). One prominent characteristic of the ML face patch is its selectivity to head orientation (Freiwald & Tsao, 2010; Taubert et al., 2020): ML neurons encode specific views of the face, such as frontal or profile views. Although ML neurons are view variant and do not deeply encode the identity information of faces (Freiwald & Tsao, 2010), they are selective to specific facial features and configurations, such as the size of the iris and face aspect ratio, with ramp-shaped tuning, which enables one-to-one feature-response mapping (Freiwald et al., 2009). ML neurons are also sensitive to contrast patterns of faces, which can change under different lighting conditions (Ohayon et al., 2012). The selectivity of ML neurons is size tolerant (Taubert, Van Belle et al., 2018). Inactivation of the ML face patch causes deficits in gender discrimination (Afraz et al., 2015) and face detection (Sadagopan et al. 2017), which could arise from the impairment of facial feature processing. In summary, ML neurons perform facial feature processing based on both local and holistic features in a view- and contrast-dependent manner that can vary under different circumstances for the same identity. The AL face patches are located more anteriorly and receive important inputs from the ML face patch (Liu et al., 2022). AL neurons are also view dependent, but in a different way than ML neurons; they show mirror-symmetry response to face images (Freiwald & Tsao, 2010). AL neurons that respond to a rightward-profile face also respond to a leftward-profile face at the same angle. This is interpreted as an intermediate stage that combines the information from different viewpoints of faces to construct view-invariant identity coding of faces. AL neurons, as well as ML neurons, are selective for warmer colors (Duyck et al, 2021), similar to adjacent color-sensitive areas in the IT cortex (Chang et al., 2017); this may reflect a preference for natural face colors. Few studies have systematically evaluated the difference in feature coding between AL and ML face patches; however, one study showed that ML face patches encode the horizontal components of faces, whereas AL patches encode more vertical components in the long-lasting face response (Taubert et al., 2016). The view invariance and identity encoding are further developed in the AM face patch, the most anterior face patch of the ventral system (Freiwald & Tsao, 2010). AM neurons selectively respond to specific facial identities in any rotational view. In addition, selectivity of AM neurons is generally preserved at different sizes (Waidmann et al., 2022). Furthermore, AM neurons, as well as ML neurons, reflect conscious perception (Hesse & Tsao, 2020b). This implies that AM neurons represent one of the most advanced stages in the processing of faces, playing a pivotal role in encoding the facial features across varying viewpoint and distance. Through visual feature analyses of the ventral system, view-dependent images in the posterior face patches are converted into identity information of faces in the anterior face patches (Eifuku et al., 2004; Freiwald & Tsao, 2010; Meyers et al, 2015). This processed visual identity information can distinguish one face from another but may not involve knowledge of personal identity (Landi et al., 2021; Landi & Freiwald, 2017). To recognize a face of individuals we personally know, identity information seems to require further processing in the memory system.

How do ventral system face patches, especially identity-selective AM patch, encode facial identity? Two models have been proposed for encoding facial identity in a multidimensional face feature space: axis encoding (Chang & Tsao, 2017) and norm-based encoding (Koyano et al., 2021; Leopold et al., 2001; Leopold et al., 2006) models (Figure 4). Axis encoding explains face tuning using a linear combination of neurons, each of which encodes one or few feature dimensions of the faces. Each neuron exhibited a ramp-shaped increase in response according to the increase in features along the axis (Figure 4A). At the population level, face patch neurons can map the face feature space using a linear combination of these features. It has been suggested that this linear encoding of features is related to high-level semantic features (Higgins et al., 2021). Norm-based encoding explains face tuning by the distance from a norm, or prototype, which is thought to be created by averaging past experiences of faces (Valentine, 1991). Each neuron exhibited a tendency for reduced (or, in some cases, increased) responses to the norm face, creating a V-shaped tuning pattern along the identity levels (Figure 4B). By emphasizing the distance from the norm face, this V-shaped tuning can effectively encode the face feature space, subtracting the redundant features. These two models are not mutually exclusive; they can be combined to represent the face feature space with different time courses (Figure 4C). Axis encoding starts earlier, together with a transient increase in response after the presentation of faces, which exhibits ramp-shaped encoding to the best feature of each neuron. Norm-based encoding is integrated into this ramp-shaped tuning by suppressing the response to the average face with a delay of ~200 ms. This time course is consistent with other studies that reported delayed encoding of fine category information (Dehaqani et al., 2016; Matsumoto et al, 2005; Sugase et al., 1999) and identity (Freiwald & Tsao, 2010), which are accompanied by suppression (Salehi et al., 2020) and spatially distributed gamma local field potential (LFP) (Miyakawa et al., 2018).

Figure 4.

Facial feature encoding in the ventral face patches

Note. A. Encoding of principal face components with linear combination of ramp-shaped tuning. B. Encoding of a norm face with a V-shaped tuning. C. Delayed integration of the ramp and V-shaped tuning. Adopted and modified from Chang & Tsao, 2017 (A) and Koyano et al., 2021 (B, C), with permission.

Anterior Memory System

The anterior ventral IT region is known to represent familiar face identity (Eifuku et al., 2011) and memories associated with faces (Eifuku et al., 2010). Anterior to the core face patches, there are two additional face patches that encode memory information of faces in the perirhinal (PRh) and temporal pole (TP) cortex (Landi & Freiwald, 2017). We group these two with the AM face patches and refer to them as the “anterior system.” Although the anatomical connections of these specific face patches have not been fully examined, brain areas around these regions are heavily connected with the medial temporal areas, such as the hippocampus, and with the anterior IT regions, where the ventral system is present (Lavenex et al., 2002; Lavenex et al., 2004; Suzuki & Amaral, 1994). In contrast to the stable representation of faces in AF face patches (McMahon et al., 2014), the representation of face patches in the anterior system is flexible and can represent the memory information formed as a result of learning. AM face patch neurons develop visual familiarity through the repeated presentation of faces over weeks (Koyano et al., 2023) (Figure 5A). Neurons in the TP (Landi et al., 2021) (Figure 5B) and PRh (She et al., 2021) face patches show sensitivity to personally familiar faces, defined as the faces of conspecifics living next door or caretakers who have personal interactions with the individual every day. These areas could also serve as sources of prediction signal feedback for core face patches (Schwiedrzik & Freiwald, 2017) and may form broader memory-processing networks with the entorhinal cortex and hippocampus, where face-selective responses (Ku et al., 2011) and cross-modal representation of individual identity (Tyree et al., 2023) has been reported. Together with the ventral system, the anterior system forms a face processing route on the ventrolateral surface of the IT cortices, providing learned information regarding particular faces that is important for identifying the individual (Figure 3). It is crucial to rapidly determine the personal identity of conspecifics in the social lives of primates. We propose that this ventral face processing route evolved to integrate constant facial features and determine the individual identity, answering the question “who are you?”

Figure 5.

Memory representation of faces

Note. A. Development of visual familiarity in the AM face patch. B. Personal familiarity representation of identity in the temporal pole (TP) face patch. Adopted and modified from Koyano et al., 2023 under CC-BY 4.0 License (https://creativecommons.org/licenses/by/4.0/) (A) and Landi et al., 2021 with permission (B).

Dorsal Interaction System

IT face patches located dorsal to the ventral system include MF, AF, medial dorsal (MD), and anterior dorsal (AD) patches. We refer to these collectively as the “dorsal system,” according to the dorsal/ventral two-face pathway model of face processing (Liu et al., 2022; Pitcher & Ungerleider, 2021). Unlike the ventral system, the dorsal system is particularly adept at processing dynamic, interactive, and multimodal information. One unique characteristic of the dorsal system is its sensitivity to dynamic information, such as face motion. This sensitivity is prominent in the MD and AD face patches (Fisher & Freiwald, 2015a, Yang & Freiwald, 2023), which are identified as additional face patches for their sensitivity to motion, as well as AF face patch (Zhang et al., 2020; but also see Polosecki et al., 2013 and Russ & Leopold, 2015 for motion sensitivity in the ventral system). Neurons in these areas showed significant enhancement of responses to moving faces over static ones. The motion of faces is also thought to be essential for social communication, as it conveys social signals by forming facial expressions. Studies confirmed that the areas sensitive to face motion (Furl et al., 2012), as well as the face areas within the STS (Hadj-Bouziane et al., 2008; Taubert et al., 2020b), also respond to facial expressions. A single-unit recording study showed that MD neurons encoded facial expressions that are preserved even with changes in identity and head orientation (Yang & Freiwald, 2021), suggesting the importance of the motion-sensitive regions in the recognition of facial expressions. In addition to processing these facial motions, the dorsal system also processes a crucial information for understanding interactions with other individuals. A neuroimaging study showed the activation of dorsal STS areas, including the AD face patch, during the observation of social interaction scenes (Sliwa & Freiwald, 2017). Single-unit studies have further shown that MD neurons are selective to others’ gazes (Yang & Freiwald, 2021; Yang & Freiwald, 2023), suggesting their important role in eye contact communication. For more direct, physical interaction with others, the ability to accurately perceive real-world sizes and judgment of reachable distances is essential (Leopold & Park, 2020). AF neurons are sensitive to the face size during naturalistic viewing (McMahon et al., 2015) and can represent the sizes based on their actual dimensions in the real world, not just how they appear on the retina (Khandhadia et al., 2023). AF neurons also have sensitivity to three-dimensional structure of faces (Murphy & Leopold, 2019), which may enhance communications in the real world. Beyond its capability in these visual processing, the dorsal system can also integrate broader information for effective interaction. Neurons in AF face patches exhibit multimodal responses to both visual and auditory stimuli, combining these modalities for effective vocal communication (Khandhadia et al., 2021). Additionally, the dorsal system integrates faces with bodies, establishing whole-agent selectivity (Fisher et al., 2015b). This function overlaps with monkey patches in the STS areas that respond to naturalistic face-body configuration (Zafirova et al., 2022; Zafirova, Bognar & Vogels, 2024). These sensitivities to multimodal and whole-agent information would contribute to understanding facial attributes in a broader context. Taken together, face patches in the dorsal system process dynamic, interactive, and multimodal information, which is particularly important for navigating social interactions.

Although the functional differences and hierarchical organizations within the dorsal system are unclear, an inactivation study suggested a posterior-to-anterior signal flow from ML to AF patch, similar to the ventral system (Liu et al., 2022). Future studies that directly compare the dynamic and interactive functions will reveal signal processing mechanism in the dorsal system. It is also relevant to note that face patches in the dorsal system also show selectivity for static visual features. The MF face patch, often studied together with ML patch, is involved in visual feature processing (e.g., Chang & Tsao, 2017; Freiwald et al., 2009). Similarly, the AF face patch is known to be selective for facial identity (Koyano et al., 2021; McMahon et al., 2015; Waidmann et al., 2022), and the MD patch is selective for face orientation (Yang & Freiwald, 2021). However, the dorsal system’s unique sensitivity to dynamic and interactive features distinguishes them from the ventral system, which specializes in analyzing static and unchanging visual information such as identity. This sensitivity to dynamic and interactive cues in the dorsal system is essential for understanding the actions of others, potentially facilitated by cooperation with other brain regions, including the frontal, cingulate, and parietal cortices (Deen et al., 2023).

Frontal Social System

Face patches are not only found in the IT cortex, but also in the frontal cortices of macaques (Haile et al., 2019; Tsao, Schweers et al., 2008) and marmosets (Schaeffer et al., 2020). These face patches are characterized by weaker responses to non-face objects rather than strong responses to faces and are typically located next to the ventral color region of the prefrontal cortices (Haile et al., 2019). This “frontal system” includes the prefrontal orbital (PO), prefrontal ventral (PV), and prefrontal arcuate (PA) face patches. The PV face patch was originally termed a prefrontal lateral (“PL”) face patch (Tsao, Schweers et al., 2008); however, here, we use the term PV to avoid confusion with the PL (posterior lateral) face patch in the IT cortex (Deen et al., 2023). Unlike core face patches, the existence and location of frontal patches can vary among individuals (Haile et al., 2019; Janssens et al., 2014; Tsao, Schweers et al., 2008). While the anatomical connections of these specific face patches remain to be fully explored, the prefrontal areas around the frontal system have strong connections with the STS, where the dorsal system is located (Saleem et al., 2008; Saleem et al., 2014). In line with these connections, the functions of the frontal face patches partially overlap with those of the dorsal system. Neuroimaging studies have shown that the PO face patch, located in the lateral orbital sulcus, is important for emotional processing (Barat et al. 2018; Dureux et al., 2023; Liu et al., 2017; Tsao, Schweers et al., 2008). Areas in the ventrolateral prefrontal cortex (VLPFC), where the PV face patches are located, are important for social signal processing (Russ & Leopold, 2015; Schaeffer et al., 2020; Shepherd et al., 2018; Sliwa & Freiwald, 2017).

Although no studies have performed single-unit recordings from fMRI-defined frontal face patches, some studies have conducted electrophysiological recordings from face-selective neurons in the prefrontal cortices. Individual neurons in the orbitofrontal cortex encode facial expression, identity, motion, and vocal information with view-dependency (Rolls et al., 2006), as well as social category information, including expression, age, and gender (Barat et al., 2018). Lesions in the orbitofrontal cortex disrupt arousal in response to social stimuli (Goursaud & Bachevalier, 2020; Murphy & Bachevalier, 2020) and face-looking behaviors (Goursaud & Bachevalier, 2020). In the VLPFC, a cluster of face neurons has been reported below the principal sulcus, where a PV face patch is present (O’Scalaidhe et al., 1997). These neurons play an important role in various functions, including audiovisual integration (Sugihara et al., 2006), working memory (Hwang & Romanski, 2015), identity encoding (Diehl et al., 2022; Sharma, Diltz et al., 2024), expression (Diehl et al., 2022; Sharma, Diltz et al., 2023), and viewpoint selectivity (Romanski & Diehl, 2011). Inactivation of the VLPFC causes impairment of audiovisual working memory but not of visual face-only working memory (Plakke et al., 2015), suggesting its role in multimodal integration over extended timescales.

Although a wide variety of functions exist, the frontal system processes dynamic and high-level social information of faces, similar to the dorsal system. Together with the dorsal system in the IT cortex, the frontal system may form a route that analyzes dynamic and interactive facial features (Figure 3). This dorsal face processing route has likely evolved to understand the actions of an individual, answering the question “what are you doing?” Effective communication requires selecting a response based on interpreting another’s actions, which is complex and multi-step processes. Dorsal face processing routes likely collaborate with other brain regions, forming a larger network for this purpose. For example, a proportion of neurons in the AF face patch show strong functional connectivity with parietal regions including intraparietal areas during watching movies of conspecifics (Park et al., 2017). Interestingly, neurons in the anterior intraparietal areas selectively respond to body motions, including facial gestures like lip smacking and yawing (Lanzilotto et al., 2020). The face patches in the dorsal and frontal systems may operate in conjunction with other brain regions, such as the parietal cortex which is important for social affordance (Orban et al., 2021), as well as cingulate and other frontal areas, to interpret other’s actions and choose suitable reactions (Deen et al., 2023; Orban et al., 2021).

Face Information in the Subcortical Areas

Face Signals in the Basal Ganglia and Superior Colliculus

Many advanced studies have identified the key role of the basal ganglia in learning and decision making based on rewards (Graybiel, 1995; Hikosaka, 1991). However, it has recently been reported that they are also related to face recognition. For example, it has been reported that patients in the early stage of Huntington’s disease who have degeneration of CDt neurons (Vonsattel et al., 1985) show deficits in face recognition and discrimination (Grill-Spector et al., 2018). In addition, previous studies have reported that patients with Parkinson’s disease have impaired familiar and unfamiliar face detection (Ashby et al., 2003; Baggio et al., 2012; Sprengelmeyer et al., 2003). Notably, in Parkinson’s disease, dopamine neurons tend to be removed in the lateral part of the substantia nigra pars compacta, which tends to project to the striatum tail (STRt), including the CDt and PUTt (Hernandez et al., 2019). The STRt receives dense inputs from the IT cortex, which encodes rich facial and object information (Baizer et al., 1993; Saint-Cyr et al., 1990; Webster et al., 1995; Yeterian & Pandya, 1995). These reports suggest that the basal ganglia, particularly the STRt, are involved in facial information processing.

Recently, Kunimatsu et al. (2023) found that STRt neurons encode the faces of socially familiar persons. In this study, the authors presented monkeys with the face images of a familiar person (daily carer, social familiarity [+]) and an unfamiliar person (unknown person, social familiarity [–], Figure 6). They also tested responses to fractal images, which were consistently associated with large (object value [+]) and small rewards (object value [–]). STRt neurons responded to face images, especially images of familiar faces. These neurons also showed a stronger response to the object value (+) images than to the object value (–) images. Notably, the strength of the modulation of social familiarity and object value were positively correlated. These results indicate that social and value information are co-encoded in the STRt and that these signals are mediated by the same neuronal mechanism.

Figure 6.

The striatum tail neuron encodes both socially familiar faces and object value

Note. Response of STRt neuron to facial and fractal images. This neuron shows strong response to the image with social familiarity (+) and object value (+).

The STRt projects to the caudal-dorsal-lateral part of the substantia nigra pars reticulata (SNr) (Amita et al., 2019). The inhibitory connection to the SNr, which serves as a direct pathway, conveys high-valued object-dominant signals and inhibits SNr neurons, leading to the disinhibition of neurons in the intermediate layer of the SC (SCi), which is the output layer to oculomotor system, to facilitate saccades toward high-valued objects (see Figure 7) (Amita et al., 2020; Yasuda & Hikosaka, 2015). Previous studies have reported that object signals in STRt circuits mediate the rapid distinction of high-value objects among numerous objects and the tendency to look at high-value objects (Kunimatsu et al., 2019; Yamamoto et al., 2013). The same mechanism may underlie the facilitation of saccades toward face images so that we can quickly find a familiar person in a crowd.

Figure 7.

Hypothetical face network diagram

Note. This diagram indicates a hypothetical network structure of face processing. Amyg, amygdala; cla, claustrum; LGN, lateral geniculate nucleus; MD, mediodorsal thalamus; PFC, prefrontal cortex; Pul, pulvinar; SCi, intermediate layer of superior colliculus; SCs, superficial layer of superior colliculus; SNr, substantia nigra pars reticulata; STRt, striatum tail.

In contrast, the superficial layers of the SC (SCs), which is the input layer from the retina, appear to function as a subcortical pathway (retina-SCs-pulvinar) that bypasses the visual cortex for rapid processing of visual information (see Figure 7) (Johnson, 2005; Morris et al., 2001; Pegna et al., 2005; Tamietto & de Gelder, 2010). Recently, some studies have reported that SCs neurons quickly respond (< 50 ms) to face- or face-like pattern images (Le et al., 2020; Nguyen et al., 2014). These neurons are selective for facial expressions, even in abstract images. These results suggest that SCs neurons preferentially filter face-like patterns with short latencies to allow rapid processing of coarse visual information. These signals are sent to the lateral nucleus of the amygdala through the medial part of the pulvinar, not through the primary visual cortex (Inagaki et al., 2023). Response of pulvinar neurons is also elicited by face-like patterns with the shortest latencies (Maior et al., 2010; Nguyen et al., 2013). Because there is anatomical projection from the pulvinar to the STRt (Griggs et al., 2017), it is possible that facial information is conveyed to the STRt through this pathway. Once face images are captured by the retina, the inputs are processed through SCs. The “SCs-pulvinar-STRt” pathway may be involved in the rapid detection of familiar faces. By contrast, the “SCs-pulvinar-amygdala” pathway is primarily involved in rapid salient face detection (Hadj-Bouziane et al., 2012; Inagaki et al., 2023; Johnson, 2005; McFadyen et al., 2021). Injuries to this pathway often lead to impairments in emotional face detection (Morris et al., 2001). Perturbation in this pathway, particularly involving the amygdala, could also explain deficits in emotional face processing seen in post-traumatic stress disorder and autism spectrum disorder (Harms et al., 2010; Mahabir et al., 2015).

Face Signals in the Limbic Regions

In daily life, face signals are integrated into various types of information, such as emotions and personal experiences. Subcortical limbic regions, including the hippocampus and amygdala, which receive anatomical inputs from the face area in the temporal cortex (Aggleton et al., 1980; Cheng et al., 1997; Stefanacci & Amaral, 2000), are associated with these integrative processes. From human electrophysiological study, it is known that the neurons in the hippocampus represent familiar individuals in an abstract and invariant manner (Quiroga et al., 2005). Monkey studies have also shown that the hippocampus contains information about monkey faces and voices, suggesting multiple sensory information that indicates personality may be integrated in the hippocampus (Sliwa et al., 2016; Tyree et al., 2023).

A previous study reported that a majority of neurons in the amygdala respond to face images (Nakamura et al., 1992). Neurons that respond only to monkey or human faces are predominantly located in the lateral and basolateral nuclei (Gothard et al., 2007). These neurons also show strong selectivity, responding only to a specific individual human or monkey (Tazumi et al., 2010) and specific gaze and head orientations. In addition, they responded only to specific facial expressions, such as smiles or threats (Hoffman et al., 2007). The facial expression signal may be processed through the subcortical network because it is faster in the amygdala than in the primary visual cortex (Inagaki et al., 2023). Furthermore, Hadj-Bouziane (2012) showed that a legion of the amygdala disrupts the processing of emotional facial expressions in the high-level visual cortical areas involved in face recognition. Thus, the amygdala is involved in assessing the personal and emotional significance of faces.

The amygdala also contributes to critical behaviors for normal social development and social interaction (Rutishauser et al., 2015). For example, Munuera et al. (2018) reported that monkey social hierarchy is encoded by amygdala neurons that represent reward values. Furthermore, gaze-sensitive neurons in the amygdala become active only when a viewer actively makes eye contact with another monkey (Mosher et al., 2014). Bilateral amygdala lesions eliminate robust viewing preferences for both real and illusory faces (Taubert, Flessert et al., 2018). This demonstrates the fundamental role of the amygdala in guiding eye movements toward face stimuli. Consistent with this, amygdala activity is induced in autism, which impairs social skills such as reading facial expressions and making eye contact with people (Rutishauser et al., 2013). Thus, face processing in the amygdala plays a key role in social interactions.

Cortical-Subcortical Networks for Face Processing

Both cortical and subcortical face networks are essential for face processing, yet little is known about the interactions between them. The amygdala, pulvinar, and claustrum directly project to face patches in the IT cortex (Bouziane et al., 2012; Hadj- Grimaldi et al., 2016; Moeller et al., 2008). The similar function in recognizing facial expressions between face patches and the amygdala may be attributed to the interconnection of IT face patches with the amygdala. Lesions in the amygdala selectively eliminate responses to emotional faces in face patches of the IT cortex (Hadj-Bouziane et al., 2012), while the inactivation of IT face patch (AF face patch) reduces responses to faces and objects in the amygdala (Liu et al., 2022). Although the interaction between face patches in the prefrontal cortex and the amygdala remains unclear, anatomical studies indicate reciprocal projections between the prefrontal cortex and the amygdala (Ghashghaei et al., 2007; Saleem et al., 2008). These findings suggest that face patches communicate with the amygdala to facilitate social interaction. Further investigation is needed to elucidate the specific information that the amygdala transmits to and receives from these face patches.

While it remains to be determined which subcortical regions except the amygdala receive the input from the face patches, there might be some clues from fMRI studies that have investigated the functional connectivity of face patches. A study using single-unit fMRI mapping showed that face patches are functionally connected to the STRt, SC, LGN, amygdala, pulvinar, and claustrum (Park et al., 2017). Resting-state fMRI studies have indicated that face patches are functionally connected to the striatum, LGN, mediodorsal thalamus, hippocampus, amygdala, pulvinar, and claustrum (Schwiedrzik et al., 2015; Zaldivar et al., 2022). Therefore, these subcortical regions may be directly or indirectly connected to face patches, serving as a gateway or interface between them. Figure 7 shows a diagram of hypothetical face networks based on anatomical studies (Aggleton et al., 1980; Amita et al., 2019; Erickson et al., 2004; Griggs et al., 2017; Grimaldi et al., 2016; Jones & Burton, 1976; Kemp & Powell, 1970; Kravitz et al., 2013; Romanski et al., 1997; Saint-Cyr et al., 1990; Saleem et al., 2014; Selemon & Goldman-Rakic, 1988). Once face images are captured by the retina, the inputs are separately processed through the subsequent pathways, either via SCs or LGN. The “SCs-pulvinar-amygdala” pathway is mainly involved in fast salient face detection (Hadj-Bouziane et al., 2012; Inagaki et al., 2023; Johnson, 2005; McFadyen et al., 2021). Injuries to this pathway often result in the impairment of emotional face detection (Morris et al., 2001). The perturbation of this pathway, especially involving the amygdala, could also explain deficits in emotional face processing in post-traumatic stress disorder and autism spectrum disorder (Harms et al., 2010; Mahabir et al., 2015).

While the “SCs-pulvinar-amygdala” pathway primarily facilitates rapid detection of salient faces, the “LGN-V1-IT” pathway predominantly supports memory-based face identification (Grill-Spector et al., 2018), and its dysfunction may lead to prosopagnosia (Lohse et al., 2016). Facial information from both the amygdala and the IT cortex may be conveyed to the SCi via the STRt for the orientation of visual attention and looking behaviors (Amita et al., 2019; Amita et al., 2020; Griggs et al., 2017; Kunimatsu et al., 2023; Yasuda & Hikosaka, 2015). Additional research is needed to validate the involvement of cortical-subcortical networks in face looking behaviors.

Conclusions and Future Directions

NHP studies have revealed the neural correlates of psychological effects on face recognition. Activity in the middle face patch (ML), showing a stronger response to upright faces compared to inverted faces, is in line with the face inversion effect (Sugase-Miyamoto et al., 2014; Taubert et al., 2015a). Activity in the dorsal face patch (AF), modulated by the three-dimensional physical size of a face image, corresponds to Emmert’s law (Khandhadia et al., 2023; McMahon et al., 2015). The response of the ventral face patch system and the amygdala to illusory facial features provides neurophysiological evidence for face pareidolia (Sharma, Vinken et al., 2023; Taubert, Wardle, Tardiff, Koele et al., 2022). Additionally, the norm-based encoding of facial identity, derived from past face-related experiences, provides a mechanistic model for species-specific effect (Koyano et al., 2021; Leopold et al., 2006). However, several important questions remain unanswered. How do the face processing networks generate these neural correlates? How does the ventral face patch system interact with the dorsal face patch system in the IT cortex? How does the face patch system in the IT cortex interact with the face patch system in the frontal cortex? Which subcortical regions receive inputs from the face patches? To answer these questions, further NHP studies are necessary, because recent advances in genetic tools (optogenetics, chemogenetics, fluorescence imaging) enable pathway-selective manipulation and cell-type identification in NHPs (Oyama et al., 2021; Tremblay et al., 2020). These tools can be applied to the identification of face processing circuits in the future, and the NHP studies on face processing would help us to understand the neuronal mechanisms underlying the impairment of face processing, including prosopagnosia, schizophrenia, post-traumatic stress disorder, and autism spectrum disorder.