The Functional Neuroanatomy of Human Face Perception

Abstract

Face perception is critical for normal social functioning and is mediated by a network of regions in the ventral visual stream. In this review, we describe recent neuroimaging findings regarding the macro- and microscopic anatomical features of the ventral face network, the characteristics of white matter connections, and basic computations performed by population receptive fields within face-selective regions composing this network. We emphasize the importance of the neural tissue properties and white matter connections of each region, as these anatomical properties may be tightly linked to the functional characteristics of the ventral face network. We end by considering how empirical investigations of the neural architecture of the face network may inform the development of computational models and shed light on how computations in the face network enable efficient face perception.

1. INTRODUCTION

Face perception is critical for normal social functioning. For example, faces provide key visual information that we use to discriminate one person from another every single day. Because face perception is ecologically and evolutionarily relevant across species (Freiwald et al. 2016, Gross & Sergent 1992, Tsao & Livingstone 2008, Weiner & Grill-Spector 2015), a fundamental question is, What neuroanatomical and functional features of the human brain contribute to the visual perception and recognition of faces?

To tackle this question, many influential theories regarding the cognitive neuroscience of face perception (Behrmann & Plaut 2013, Duchaine & Yovel 2015, Freiwald et al. 2016, Haxby et al. 2000, Jiang et al. 2006, Kanwisher 2000, Rossion 2008) have examined how regions within the brain provide the representations proposed by classical theories of face perception in cognitive psychology (Bruce & Young 1986, Konorski 1967, Valentine 2001). This approach has been very successful in identifying a network of functional regions in the human occipito-temporal lobes that is specialized for processing faces. Indeed, many influential studies have functionally differentiated both ventral from dorsal components of this network (Freiwald et al. 2016, Pitcher et al. 2014) and functional regions from one another within these dorsal and ventral components, respectively (Kay et al. 2015; Pitcher et al. 2011a, 2012; Rossion et al. 2003; Schiltz et al. 2006; Weiner & Grill-Spector 2010; Weiner et al. 2010). The present article focuses on the ventral component of this network (Figure 1) and differs from prior reviews in two main ways. First, whereas prior reviews focus on the face network as an independent system, a repetitive theme throughout this review is that the ventral face network is embedded within the visual system more generally. Thus, we discuss and propose that processing in visual regions outside the face network and their interaction with the face network—for example, through long-range white matter connections—is meaningful and contributes to the efficiency of face processing. Second, in addition to understanding the functional characteristics of each region within the ventral face network, this article zooms into the microscopic structure of neural tissue composing each region. Both of these are important additions to prior reviews and are necessary stepping-stones toward building a mechanistic model that would inform how the anatomical structure of the face network subserves computations underlying fast and efficient face recognition.

Figure 1.

Face-selective regions in human ventral occipito-temporal cortex. (a) Face-selective regions are identified on the basis of higher responses to faces compared to a variety of other stimuli (faces > bodies, objects, places, and characters; t > 3, voxel level). As concluded in our prior work (Weiner & Grill-Spector 2010, 2011, 2012, 2013), including many categories in the statistical contrast (as opposed to faces > objects or faces > places) improves the accuracy of delineating face-selective regions in ventral temporal cortex (VTC). Additionally, localizing body-selective regions further guides the parcellation, as body- and face-selective regions together form a topographic map in VTC (Orlov et al. 2010; Weiner & Grill-Spector 2010, 2011, 2012, 2013). The figure shows an inflated cortical surface of an individual participant depicting the typical three clusters of face-selective regions in ventral occipito-temporal cortex. These clusters are found bilaterally and arranged from posterior to anterior. One cluster is on the inferior occipital gyrus (IOG) referred to as IOG-faces [also as the occipital face area (OFA)]; a second cluster is on the posterior aspect of the fusiform gyrus, extending to the occipito-temporal sulcus, referred to as pFus-faces [also fusiform face area one (FFA-1)]; a third cluster is located approximately 1–1.5 cm more anterior on the lateral aspect of the fusiform gyrus, overlapping the anterior tip of the mid-fusiform sulcus (MFS) and is referred to as mFus-faces (also referred to as FFA-2). White lines: boundaries of retinotopic areas. (b) Response amplitudes of mFus-faces from independent data showing the typical higher responses to faces compared to other stimuli. Adapted from Stigliani et al. (2015). (c) Responses to single images in pFus- and mFus-faces from electrocorticography recordings. Each cell shows the normalized responses in the high frequency broadband range (HFB) (30–150 Hz) to a single image averaged across 2–5 presentations of that image over a 100–350-ms time window. The first column shows responses in an electrode over pFus-faces/FFA-1 (indicated by 1 in panel a), and the second shows responses over mFus-faces/FFA-2 (indicated by 2 in panel a). Responses to face images that vary in gender, expression, size, pose, and background are higher than any of the nonface images. Adapted from Jacques et al. (2016). (d) Responses in ventral face-selective regions to face silhouettes are significantly higher than two-tone shapes and scrambled images. Adapted from Davidenko et al. (2012). (e) Responses in ventral face-selective regions are highest when faces are identified, intermediate when they are detected but not identified, and lowest when they are missed. Adapted from Grill-Spector et al. (2004).

From vision science, neuroscience, and computational perspectives, a complete mechanistic model explaining the functional neuroanatomy of face perception would (a) define each component of the ventral face network, (b) determine the anatomical features of each component, as well as their connections, (c) understand the functional characteristics (e.g., the representations and information) contained within each component, (d) derive the computations within and across components to the point that they can be modeled and cross-validated, and (e) provide an understanding regarding how anatomical features of the underlying neural circuits and their connections implement computations relevant for face perception and recognition.

This review synthesizes current knowledge of the face network in the human brain and shows that the field has made significant progress toward generating this ideal model. At the same time, this review also highlights remaining important unknowns that would bring the field even closer to making this ideal model a reality. First, we describe functional characteristics and organization of the human ventral face network. Second, we summarize microanatomical features of the ventral face network. Third, we describe features of white matter connections of the ventral face network and their role in face perception. Fourth, we discuss recent measurements of basic computations of the ventral face network performed by population receptive fields (pRFs). Fifth, we synthesize neural features and consider hypotheses regarding their computational role. Additionally, at the end of each section, we discuss key open questions for future research—questions whose solutions will further the field toward a complete mechanistic model of the functional neuroanatomy of face perception.

2. FUNCTIONAL CHARACTERISTICS AND ORGANIZATION OF THE VENTRAL FACE NETWORK

2.1. The Functional Characteristics of the Ventral Face Network

Face-selective regions, which exhibit higher neural responses to faces compared to other stimuli, have been identified with neuroimaging methods first with positron emission tomography (Sergent et al. 1992, Sergent & Signoret 1992), then with intracranial electroencephalography (Allison et al. 1994a,b, 1999; McCarthy et al. 1999; Puce et al. 1999), and later with functional magnetic resonance imaging (fMRI) (Kanwisher et al. 1997, McCarthy et al. 1997, Pinsk et al. 2009, Puce et al. 1996, Tong et al. 1998, Weiner & Grill-Spector 2010). On the basis of this functional characteristic, scientists identify a constellation of face-selective regions using fMRI (Haxby et al. 2000, Kanwisher 2010). In the occipital and temporal lobes, scientists identify regions in both ventral occipito-temporal cortex (Figure 1a) and superior temporal cortex (Freiwald et al. 2016, Weiner & Grill-Spector 2015). The former are associated with face perception and recognition (Fang & He 2005, Grill-Spector et al. 2004, Moutoussis & Zeki 2002, Tong et al. 1998) and the latter are associated with dynamic aspects of face perception (Andrews & Ewbank 2004, Calder & Young 2005, Calder et al. 2007, Pitcher et al. 2011a, Puce et al. 1998, Winston et al. 2004). As the focus of this review is understanding the neural basis of face recognition, we focus on three regions of the ventral face network: IOG-faces, pFus-faces, and mFus-faces (Figure 1a). The first region is synonymous with the occipital face area (OFA) (Gauthier et al. 2000, Pitcher et al. 2007). The other two regions are anatomically and functionally distinct components of the fusiform face area (FFA) (Kanwisher et al. 1997; Weiner & Grill-Spector 2010, 2012): pFus-faces is synonymous with FFA-1 (Pinsk et al. 2009), and mFus-faces is synonymous with FFA-2 (Pinsk et al. 2009). Additional face-selective regions have been identified in the anterior temporal lobe (Jonas et al. 2016, Rajimehr et al. 2009, Tsao et al. 2008), but these regions are not considered part of the core face network [but see Collins & Olson (2014)] as they are not just driven by visual stimulation and are more elusive to identify because of lower signals and susceptibility artifacts in fMRI.

The basic functional characteristic of regions within the ventral face network is higher neural responses to the visual presentation of faces compared to a variety of other stimuli, including animate stimuli (such as limbs, bodies, and animals), familiar and unfamiliar objects, scenes, characters, and textures (Figure 1b). Within each region, functional responses to face exemplars are higher than exemplars of other categories (Figure 1c) (Davidesco et al. 2014, Jacques et al. 2016, Mur et al. 2012, Privman et al. 2007). This characteristic response is maintained across sessions (Berman et al. 2010, Peelen & Downing 2005, Weiner & Grill-Spector 2010), tasks (Bugatus et al. 2017, Weiner & Grill-Spector 2010), and stimulus formats (Figure 2d), including photographs (Ishai et al. 2000, Kanwisher et al. 1997), line drawings (Ishai et al. 2000, Kanwisher et al. 1997), two-tone stimuli (Davidenko et al. 2012, Tong et al. 2000), texture (Farivar et al. 2009), and spatial frequency (Vuilleumier et al. 2003).

Figure 2.

Face-selective regions and cytoarchitectonic boundaries are predicted by cortical folding. (a) Inflated cortical surface zoomed on ventral temporal cortex (VTC) of the right hemisphere of a representative participant showing face-selective regions (faces > other categories; t > 3, voxel level) in red. White: mid-fusiform sulcus (MFS). Dashed black lines: 1-cm diameter disks aligned to the anterior and posterior tips of the MFS, respectively. (b) Inflated cortical surface zoomed on VTC of the right hemisphere of a representative postmortem brain showing cytoarchitectonic areas within the fusiform gyrus (FG), each in a distinct color (see legend). White: MFS. (c, top) Percentage of face-selective voxels predicted by disks in panel a. Dashed black line: percentage of face-selective voxels predicted by Talairach coordinates. Error bars: standard error of the mean across participants. There is a tighter coupling between the MFS and mFus-faces/FFA-2 located more anteriorly than pFus-faces/FFA-1, which is located more posteriorly. Adapted from Weiner et al. (2014). (Bottom) The distance between the MFS and the boundary between cytoarchitectonic areas of the medial FG (FG1 or FG3) and the lateral FG (FG2 or FG4). There is a tighter coupling between the MFS and the anterior cytoarchitectonic boundary between FG3 and FG4 compared to the posterior boundary between FG1 and FG2. Adapted from Lorenz et al. (2017). Abbreviations: CoS, collateral sulcus; OFA, occipital face area; OTS, occipito-temporal sulcus; pFus, posterior fusiform; mFus, mid fusiform.

Preferential responses to faces over other stimuli are maintained across image transformations such as changes to stimulus position, size, or viewpoint. Face information can also be read out from distributed responses across transformations (Anzellotti et al. 2014, Axelrod & Yovel 2012, Carlson et al. 2011, Kietzmann et al. 2012, Natu et al. 2010, Schwarzlose et al. 2008). However, functional responses of the ventral face network are modulated by stimulus position (Levy et al. 2001, Yue et al. 2011), size (Yue et al. 2011), illumination (Grill-Spector et al. 1999), contrast (Rotshtein et al. 2005, Yue et al. 2011), and viewpoint (Grill-Spector et al. 1999, Kietzmann et al. 2012, Natu et al. 2010, Vuilleumier et al. 2002). For example, responses in ventral face-selective regions are higher for upright than upside-down faces (Davidenko et al. 2012, Kanwisher et al. 1998, Yovel & Kanwisher 2005) and are higher for centrally presented than peripheral faces (Hasson et al. 2002, Levy et al. 2001). Additionally, responses in face-selective regions are modulated by top-down factors, including attention (Çukur et al. 2013b, Davidesco et al. 2013, O’Craven et al. 1999), expectation (Egner et al. 2010, Summerfield et al. 2008), and familiarity (Avidan & Behrmann 2009, Ewbank & Andrews 2008, Harris et al. 2016, Natu & O’Toole 2015, Weibert & Andrews 2015).

Critically, fMRI-adaptation experiments have been pivotal in showing that responses in ventral face-selective regions are sensitive to face identity. For example, repeating the same face produces reduced responses due to neural adaptation (Grill-Spector & Malach 2001; Grill-Spector et al. 1999, 2006; Rossion & Boremanse 2011) and parametrically increasing the dissimilarity among face identities systematically increases the responses in face-selective regions due to release from adaptation (Davidenko et al. 2012, Gilaie-Dotan & Malach 2007, Jiang et al. 2006, Loffler et al. 2005, Natu et al. 2016). Additionally, and consistent with behavioral aspects of face perception, (a) neural sensitivity to face identity is higher for upright than upside-down faces (Gilaie-Dotan et al. 2010) and (b) both changes in facial features and changes in the metric relation among features (Schiltz et al. 2006, Yovel & Kanwisher 2004) cause a recovery from fMRI-adaptation, which is associated with the perceived change in identity (Davidesco et al. 2014; Rotshtein et al. 2005; Schiltz et al. 2006, 2010). Additionally, neural responses to faces in ventral face-selective regions are correlated with the perception of individual participants (Fang & He 2005, Grill-Spector et al. 2004, Moutoussis & Zeki 2002, Tong et al. 1998). For example, neural responses within mFusand pFus-faces are low when faces are present but not detected, intermediate when faces are detected but not identified, and highest when they are identified (Figure 1e) (Grill-Spector et al. 2004). Finally, neural responses in ventral face-selective regions are also causally involved in the perception of faces (Allison et al. 1994a; Jonas et al. 2012, 2014; Parvizi et al. 2012; Pitcher et al. 2011b, 2012; Puce et al. 1999; Rangarajan et al. 2014; Rossion 2008).

Altogether, these foundational studies reveal that the amplitude of neural responses in the ventral face network are overall higher for faces than nonfaces across formats, and their activity is linked to perception of faces. Nonetheless, the amplitude of response is modulated both by low-level stimulus properties such as position and size, and top-down factors such as attention.

2.2. The Functional Organization of the Face Network Is Consistent Across Participants

A striking characteristic feature of the functional architecture of the ventral face network is that the cortical location of functional regions is highly consistent across people. At the centimeter scale, face-selective regions are identifiable on specific gyri: Occipital face-selective regions are located on the inferior occipital gyrus (IOG) (Figure 1a), whereas face-selective regions in ventral temporal cortex (VTC) are located on the lateral aspect of the fusiform gyrus (FG). The reliable coupling between different gyri and face-selective regions has been known since the late nineties, which is why researchers labeled these regions with particular macroanatomical structures [e.g., the fusiform face area (Kanwisher et al. 1997); the occipital face area (Gauthier et al. 2000)].

It is important to emphasize that gyri are not small—they are typically several centimeters long and wide and, thus, have a rather large surface area. Consequently, limiting a functional region to anywhere on these macroanatomical structures results in extensive across-subject variability in the localization of face-selective regions and low predictability of localizing these regions from cortical folding alone (Frost & Goebel 2012). However, in the last five years, we have gained new insights regarding the structural-functional coupling of face-selective regions at the millimeter scale. These advancements have been possible due to rigorous measurements of the variability of the cortical folding patterns of the FG and neighboring sulci (Weiner & Zilles 2016, Weiner et al. 2014) in addition to precise measurements of the relationship between functional regions and macroanatomical landmarks (Nasr et al. 2011, Weiner & Grill-Spector 2010, Weiner et al. 2014).

In terms of primary sulci, cortical folding patterns within VTC are rather obvious to identify: The collateral sulcus (CoS) bounds the FG on its medial aspects, whereas the occipito-temporal sulcus (OTS) abuts the FG on the lateral aspect. Nevertheless, the clarity of cortical folding in VTC is deceptive: Studies have reported that the identification of sulci within VTC is paradoxically difficult, with about 1.2 cm of error across individual raters (Sowell et al. 2002). Notably, this error is ameliorated and the variability reduced with a simple observation: There is an additional sulcus in the FG. Specifically, the mid-fusiform sulcus (MFS) is a shallow, longitudinal sulcus that bisects the FG into lateral and medial partitions and is identifiable in every brain (Figure 2a). Importantly, variable and predictable morphological features of the MFS are quantifiable. Analysis of 158 hemispheres revealed variability in the fractionation and length of the MFS, as well as predictability in the absolute depth and location of the anterior tip of the MFS. Specifically, the OTS and CoS are consistently more than twice as deep as the MFS, and the anterior tip of the MFS aligns well with the posterior extent of the hippocampus (Grill-Spector & Weiner 2014, Weiner et al. 2014).

The anatomical quantifications of the MFS not only contributed to improved understanding of stable and variable morphological features of sulcal patterns in the FG but also improved the predictability of the localization of ventral face-selective regions from cortical folding. That is, the stability of the anterior tip of the MFS is not only anatomically relevant but also functionally relevant, as it identifies more than 80% of mFus-faces across subjects in the right hemisphere (Figure 2c) (Weiner et al. 2014). Even though the posterior tip of the MFS is substantially more variable than the anterior tip of the MFS, the posterior extent of the MFS still identifies more than half of pFus-faces (Figure 2c) (Weiner et al. 2014). Discovery of this reliable organization of two functional regions relative to what is considered a tertiary macroanatomical structure is surprising because the prevailing thought of the field is that cortical folding patterns are not consistent enough to serve as landmarks for functional regions in high-level cortices such as the human FG. As such, the correspondence of the MFS relative to large-scale centimeter maps and fine-scale functional clusters has ignited a reconsideration of these prevailing views, especially in VTC (Grill-Spector & Weiner 2014).

These surprising findings naturally lead to the following question: Why are face-selective regions consistently located in the same macroanatomical locations across individuals? Understanding the factors governing this reproducible organization is important for two reasons. First, understanding these factors may shed light on a series of organizational constraints that contribute to the anatomical scaffolding of the functional architecture underlying the ventral face network. Second, understanding these factors more completely may also bring us closer to deriving computational and/or optimization strategies of the visual system that are reflected in this structural-functional correspondence. In the next section, we consider whether cytoarchitecture and white matter connections may serve as anatomical constraints contributing to the consistency of this functional architecture.

3. THE CYTOARCHITECTURE OF THE VENTRAL FACE NETWORK

Just as recent methods have advanced our understanding of the relationship between the ventral face network and cortical folding in VTC, methodological advancements have also illuminated our understanding of the coupling between cytoarchitectonic areas and cortical folding of the FG, as well as the correspondence between cytoarchitectonic areas and functional regions in VTC. In this section, we first describe the cytoarchitecture of the FG discovered in the last several years. Then, we discuss how cytoarchitectonic features differentiate face-selective regions of the ventral face network from one another.

3.1. The Cytoarchitecture of the Fusiform Gyrus

The size, shape, and organization of cells across the six-layered cortical ribbon (the combination of which is referred to as cytoarchitecture) is a well-established criterion to parcellate the brain into areas because differences in cellular structure across cortical layers are believed to be indicative of specialized neural hardware that is utilized for particular brain functions. For example, the size and shape of Betz cells in different portions of motor cortex are reflective of areas that are specialized for moving different parts of the body (Rivara et al. 2003). Just as recent methods have advanced our understanding of the relationship between the ventral face network and cortical folding, methodological advancements have also illuminated our understanding regarding the cytoarchitectonics of the ventral face network.

The most significant methodological advancement is the development of observer-independent cytoarchitectonic techniques (Amunts & Zilles 2015, Amunts et al. 2000, Caspers et al. 2013, Lorenz et al. 2017, Rottschy et al. 2007, Schleicher et al. 2000), which are more sensitive to identify cytoarchitectonic boundaries compared to classical, observer-dependent methods that relied on the eye of the anatomist (Bailey & von Bonin 1951, Brodmann 1909, von Economo & Koskinas 1925).

What is meant by observer-independent techniques? In stark contrast to earlier methods by Brodmann and others, in which an observer made subjective judgments regarding where cytoarchitectonic boundaries occur, observer-independent techniques use automated algorithms to identify and statistically test that neighboring pieces of cell-body-stained histological sections contain different profiles of gray-level indices (GLIs). GLI profiles quantitatively estimate laminar cell density—a classic hallmark of cytoarchitecture. These methods are more sensitive and objective than classical approaches to discriminate subtle cytoarchitectonic differences between areas. A further benefit of the observer-independent approach is that it is “blind” to cortical folding. Thus, finding a correspondence between a macroanatomical feature, such as a sulcus, cannot be induced by the algorithm and is therefore particularly meaningful.

In the last four years, observer-independent cytoarchitectonic techniques have revealed a microstructural heterogeneity of human VTC. Specifically, these techniques identify two cytoarchitectonic areas referred to as FG1 and FG2 in the posterior FG (Caspers et al. 2013) and two cytoarchitectonic areas referred to as FG3 and FG4 in the mid FG (Figure 2b) (Lorenz et al. 2017). Intriguingly, analysis of the location of cytoarchitectonic transitions shows that the boundary between FG1, medially, to FG2, laterally, is within 5.41 ± 1.6 mm from the posterior MFS and the border between FG3, medially, and FG4, laterally, is within 1.42 ± 0.54 mm from the anterior MFS (Figure 2c). Because the MFS also predicts the location of face-selective regions in the FG, as described in the prior section, these findings open the possibility of investigating the relationship between cytoarchitectonic areas (e.g., FG2 and FG4) and functional face-selective regions (e.g., pFus-faces and mFus-faces) in the lateral FG.

3.2. The Relationship Between Cytoarchitectonic Areas and Face-Selective Regions in the Fusiform Gyrus

Quantifying the relationship between cytoarchitectonic areas and face-selective regions is challenging because cytoarchitectonic areas are delineated in postmortem brains, whereas face-selective regions are defined in living brains. Thus, it is presently impossible to relate these cortical divisions within the same individual. Nevertheless, it is possible to quantitatively relate these structures by aligning them to a common reference frame using cortex-based alignment, which leverages cortical folding patterns to align one brain to another irrespective of whether the brains are from living or postmortem individuals (Fischl et al. 1999, Rosenke et al. 2017). Implementing this novel approach revealed that the functionally defined face-selective regions within the FG are cytoarchitectonically dissociable: 81% ± 24% (mean ± standard deviation) of mFus-faces/FFA-2 is located in FG4, whereas pFus-faces/FFA-1 is outside FG4 and 49.5% ± 24% is located within FG2 (Figure 3) (Weiner et al. 2017). These results indicate that pFus-faces/FFA-1 and mFus-faces/FFA-2 have different cellular architectures. For example, pFus-faces/FFA-1 displays features of FG2, which has a conspicuous layer III with larger pyramidal cells than those of mFus-faces/FFA-2, as well as a prominent and dense layer IV compared to mFus-faces/FFA-2, which has a thin and moderately dense layer IV.

Figure 3.

mFus- and pFus-faces are cytoarchitectonically dissociable. (a) mFus-faces/FFA-2 is located in cytoarchitectonic area FG4. (b) pFus-faces/FFA-1 is located in cytoarchitectonic area FG2. In each panel (top): superimposition of the probabilistic map of a cytoarchitectonic area (see colorbar) and the probabilistic definition of a functional area (contour) on the FreeSurfer average brain. In each panel (bottom): cell-body-stained histological section and the gray level index (GLI; line) across cortical layers from a representative 20-micron slice. Cytoarchitectonic maps used data from 10 postmortem brains. Functional definitions are based on data from 12 living subjects.

These results have three important theoretical ramifications. First, distinct cytoarchitecture is suggestive of a differential neural hardware that is optimized for specialized computations. Thus, it is likely that the cytoarchitectonic differences between mFus-faces/FFA-2 and pFus-faces/FFA-1 are reflective of different computations implemented by these regions. Second, as cytoarchitectonic differences are used to parcellate brain areas, our data suggest that the commonly defined fusiform face area (FFA) that spans a large swath of the lateral FG consists of two distinct cytoarchitectonic components. Therefore, these findings provide strong evidence to separate the FFA into distinct regions corresponding to pFus- and mFus-faces, respectively. Third, because IOG-faces is outside the FG, it does not overlap with any of the FG cytoarchitectonic areas. In turn, this suggests that IOG-faces/OFA is also cytoarchitectonically distinct from both pFus-faces/FFA-1 and mFus-faces/FFA-2. If so, these data would indicate that each of the three main regions of the ventral face network is cytoarchitectonically dissociable, which can be further examined in future research. A stringent test of this hypothesis would involve back-projecting functionally defined face-selective regions from living subjects to cell-body-stained histological sections from postmortem brains and then extracting the cytoarchitectonic profiles from the histological sections corresponding to the location of specific functional regions.

Although identifying cytoarchitectonic characteristics of face-selective regions is a fundamental stepping-stone, we acknowledge that future research is necessary to elucidate many remaining unknowns regarding the microanatomy of the ventral face network, including the following:

1. What is the three-dimensional (3D) structure of the cytoarchitecture?

2. What is the local connectivity of cells across cortical layers?

3. What are the cell types composing the cortical tissue of each region?

More importantly, a critical missing link is a theoretical foundation capable of deriving the exact computations produced by this elaborate microcircuitry.

4. WHITE MATTER CONNECTIONS OF THE VENTRAL FACE NETWORK AND THEIR RELATION TO PERCEPTION

4.1. White Matter Connections of the Ventral Face Network

Prior theories propose that the interplay among neural hardware, connectivity, and function may determine brain networks (Van Essen et al. 1992, Zeki & Shipp 1988). Specifically, it is well agreed upon that white matter connections constrain the functional architecture of the brain (Tallinen et al. 2014, Van Essen et al. 1992, Zeki & Shipp 1988). Recent evidence has begun to elucidate the nature of long-range white matter connections of the face network with four main findings. First, there are distinct white matter tracts associated with the ventral face network compared to networks associated with processing other domains. For example, long-range white matter connections of pFus- and mFus-faces are distinct from white matter connections of a place-selective region in the collateral sulcus [CoS-places/parahippocampal place area (PPA)] (Figure 4a) (Gomez et al. 2015, Pyles et al. 2013, Saygin et al. 2012, Tavor et al. 2013). Second, ventral face-selective regions IOG-, pFus-, and mFus-faces are highly interconnected to one another via direct white matter connections (Figure 4b) (Gschwind et al. 2012, Pyles et al. 2013, Weiner et al. 2016a). Third, longitudinal white matter tracts connect early visual retinotopic areas located outside the face network to ventral face-selective regions (Figure 4c) (Gschwind et al. 2012, Kim et al. 2006, Weiner et al. 2016a). Fourth, vertical white matter tracts connect dorsal stream visual regions located outside the face network to ventral face-selective regions. For example, portions of the vertical occipital fasciculus (VOF) (Takemura et al. 2016, Weiner et al. 2016b, Yeatman et al. 2014) connect a retinotopic region in the posterior intraparietal sulcus (IPS-0) and pFus-faces (Figure 4d) (Weiner et al. 2016a).

Figure 4.

White matter tracts of the ventral face network. (a) Separate parallel tracts are associated with mFus-faces/FFA-2 (red) and CoS-places/PPA (green). Tracts were identified on the basis of functional regions and an anatomical plane in the anterior ventral temporal lobe used to define the inferior longitudinal fasciculus. (b) White matter tracts directly interconnecting IOG-, pFus-, and mFus-faces. (c) Connections from early visual retinotopic regions (V1–V2) to IOG-faces (purple), pFus-faces (orange), mFus-faces (green), and a face-selective region in anterior temporal cortex (AT; blue). (d) A subset of the vertical occipital fasciculus (yellow) connects IPS-0 and pFus-faces (cyan). Data in panels a–d show representative data from individual participants. In each panel, the brain section is illustrated in gray and shown in a sagittal view. White matter tracts are illustrated as colored tubes. Face-selective regions illustrated in red were dilated from the gray matter to extend to the white matter. (e) A schematic summarizing results of tractography experiments. Dark red ovals: core face-selective regions; pink oval: non-core face-selective region; gray ovals: regions that are considered external to the face network. The schematic is arranged such that the hierarchical axis is from left to right. Abbreviations: A, anterior; AT, anterior temporal; CoS, collateral sulcus; I, inferior; IOG, inferior occipital gyrus; IPS, intraparietal sulcus; P, posterior; PPA, parahippocampal place area; S, superior.

A schematic summarizing these white matter connections is shown in Figure 4e. We note that this diagram is incomplete because it does not provide information regarding (a) the entire connectivity of the ventral face network, (b) the direction of connections [though research in animals suggests that they are bidirectional (Felleman & Van Essen 1991, Moeller et al. 2008)], or (c) the functionality of these white matter tracts. Nevertheless, it provides several important insights regarding the connectivity features of the ventral face network. First, these findings illustrate hierarchical connections from IOG- to pFus-faces and pFus- to mFus-faces, which may provide the connectivity scaffolding for hierarchical features of the ventral face network described in Section 5. Second, the network contains redundant connections: There are multiple white matter tracts reaching each region. Third, there are bypass connections that do not follow a strict hierarchical organization. For example, mFus-faces, which is considered a later stage of the processing hierarchy following pFus-faces, is connected not only to the preceding stage (pFus-faces) but also to earlier processing stages (IOG-faces and early visual cortex). Finally, there are vertical white matter tracts connecting IPS-0, which is thought to be part of the attention network (Hutchinson et al. 2014, Silver & Kastner 2009, Silver et al. 2005, Swisher et al. 2007), to pFus-faces. These vertical tracts may facilitate top-down processing (Kay & Yeatman 2017).

Findings of nonhierarchical connections to the ventral face network, as well as connections from dorsal to ventral stream regions, generated a rethinking of the initial theoretical model suggesting that IOG-faces/OFA is a gateway to two independent and parallel processing streams: one in ventral temporal cortex and one along the superior temporal sulcus (Haxby et al. 2000). Indeed, these findings led to revised cognitive neuroscience theoretical models of the face network (Duchaine & Yovel 2015, Pitcher et al. 2014, Rossion 2008, Weiner et al. 2016a), which suggest that nonhierarchical and vertical connections enable low-level and dynamic visual information, respectively, to reach ventral stream regions and facilitate form perception and recognition.

Although it is not yet clear whether and how bypass connections may be used for typical face processing, recent evidence from our lab suggests that an important functional feature of these connections is that they may provide resiliency to the network in the face of injury or disease. For example, recently, we had the unique opportunity to measure the face network twice before (1 month before and 4 days before) and twice after (1 month and 8 months after) surgical removal of the IOG in a patient (S.P.) who underwent surgery to treat intractable epilepsy (Weiner et al. 2016a). Surprisingly, downstream regions remained functionally intact despite the resection of the IOG, which would not have been predicted by a strict hierarchical organization (Haxby et al. 2000). Interestingly, this resiliency of FG face-selective regions is also reported in patients with long-term (>10 years) damage to inferior occipital cortex (Schiltz et al. 2006, Sorger et al. 2007, Steeves et al. 2006). Measurements of long-range white matter connections in S.P. identified the longitudinal and vertical tracts discussed above (Figure 4). These tracts may contribute to the resiliency of the downstream face network after the resection by enabling neural signals to reach these downstream regions via alternate routes extending from early visual cortex and/or parietal cortex to the ventral face network (Weiner et al. 2016a).

Although identifying long-range white matter tracts of the ventral face network is a major stepping-stone, we recognize that future work is necessary to uncover many remaining unknowns regarding the connectivity of the ventral face network, including the following:

1. What is the functional utility of white matter connections within the face network as well as to regions outside the network?

2. What is the role of subcortical connections for gating information? For example, although recent nonhuman primate research reveals connections between the face network to both the pulvinar and amygdala (Grimaldi et al. 2016), the function of these connections, and whether they exist in humans, is unknown.

3. How do white matter connections contribute to the spatial segregation of face-selective regions of the ventral face network?

4. How do white matter connections contribute to behavior?

In the next subsection, we discuss how recent findings have begun to provide insight into the latter of these open questions.

4.2. Performance in Face Recognition Is Correlated with Properties of White and Gray Matter of Fusiform Face-Selective Regions

Up to this point, we have examined the relationship between anatomical (e.g., cortical folding, cytoarchitecture, and white matter connections) and functional organization (that is, the cortical layout) of the ventral face network. Nevertheless, we have not yet considered whether and how the structural-functional organization of the face network may contribute to behavior. Intriguingly, recent findings in both typical and atypical participants suggest an interplay among anatomy, function, and behavior. These data show that face perception and recognition rely on the (a) function (see sidebar, What Is the Causal Role of Face-Selective Regions in Face Perception?), (b) local white matter properties, and (c) gray matter tissue properties of face-selective regions in the ventral face network.

In terms of white matter, recent evidence suggests that the white matter properties of the face network are correlated with performance in face tasks. For example, in typical participants, face recognition performance is correlated with white matter properties local to mFus-faces in the right hemisphere (Figure 5a) (Gomez et al. 2015, Song et al. 2015). This relationship is both stimulus- and region-specific: Place recognition is not correlated with white matter properties local to mFus-faces, and face recognition is not correlated with white matter properties local to CoS-places (Gomez et al. 2015). Additional insights can be gained from examination of white matter properties of the ventral face network in participants with developmental prosopagnosia (DP) (Avidan et al. 2005, Behrmann et al. 2005, Duchaine & Nakayama 2006). Although individuals with DP have the same large-scale organization of white matter tracts in ventral temporal regions as typical subjects have (Gomez et al. 2015, Song et al. 2015), individuals with DP have atypical white matter properties in tracts associated with mFus-faces (Gomez et al. 2015, Song et al. 2015). Individuals with DP show restricted diffusion [lower mean diffusivity and lower radial diffusivity in white matter adjacent to face-selective regions (Figure 5b) (Gomez et al. 2015, Song et al. 2015)] and, in some cases, lower functional anisotropy (Song et al. 2015). In contrast, diffusion properties in white matter adjacent to CoS-places are not different across DP and typical participants (Gomez et al. 2015).

Figure 5.

Properties of functionally defined white matter correlate with face recognition. (a) Correlation between performance (% correct) in the Benton Facial Recognition Test (Benton 1980) and the fractional anisotropy (FA) of white matter next to mFus-faces [referred to as functionally defined white matter] in typical participants. Each point represents data from a single subject. Line: the line-of-best-fit to the data showing a positive correlation between behavior and FA; shaded area: 95% confidence interval. (b) Whole: mean diffusivity (MD) averaged across a longitudinal white matter tract connected to mFus-faces (see Figure 4a). Local: MD averaged in the local white matter adjacent to mFus-faces. Red: typical adults; gray: adults with developmental prosopagnosia (DP). Each point represents a participant. MD is lower in adults with DP than in typical controls. Asterisk: p < 0.01. Adapted from Gomez et al. (2015).

Understanding how white matter properties may contribute to face perception requires consideration of anatomical layout of white matter in situ. Individual differences in white matter adjacent to face-selective regions likely reflect differences in the axonal properties of pyramidal neurons within face-selective regions. Many properties of axons, such as their myelination (Barres & Raff 1993) and internode myelin distance (Etxeberria et al. 2016), are thought to be dependent on neural activity. Thus, axon quantity or myelination may be an inherent reflection of the underlying functional characteristics of a given region of cortex and, in turn, play a causal role in computational aspects such as information relay speed (Etxeberria et al. 2016). Finding differences in the diffusion properties of white matter near a region of cortex, such as that observed between typical and DP individuals near mFus-faces, may reflect large-scale differences in axons emerging from or projecting to face-selective regions.

In terms of gray matter, recent evidence suggests that performance in face tasks is also correlated with the tissue properties of gray matter in FG face-selective regions. Some evidence comes from analyses of cortical thickness, which reveal that thinner gray matter of the FG in adults correlates with improved face perception (McGugin et al. 2016) and perceptual learning of faces (Bi et al. 2014) compared to thicker gray matter within the FG. However, cortical thickness does not indicate what aspect of the tissue is different across individuals. For example, less gray matter or more myelination could reduce cortical thickness estimates from MRI images. By comparison, recent advances in quantitative magnetic resonance imaging (qMRI) (Lutti et al. 2014, Mezer et al. 2013) now enable the quantification and comparison of the amount of brain tissue within a voxel, as well as the composition of that tissue (measured through relaxation time, T₁) between individuals. Insights as to how gray matter properties may contribute to function can be gleaned from qMRI measurements across development. Indeed, qMRI measurements show that the anatomical properties of gray matter develop in the ventral face (but not place) network from age 5 to 27. Specifically, T₁ in right pFus-faces is lower in adults than in children. Intriguingly, T₁ in pFus-faces is correlated with both face-selectivity and face recognition performance across subjects (Gomez et al. 2017). Because T₁ in pFus-faces is shorter in adults than in children, these findings suggest a mechanism of tissue growth, where increases in certain tissue types within gray matter are coupled with improved face processing. The tissue compartments contributing to this development are not yet known, but they likely include changes in multiple compartments, including increases in dendritic arbors (Elston & Fujita 2014), oligodendrocytes (Barres & Raff 1993), and myelination. Overall, these data suggest a fascinating new model by which emergent brain function and behavior result from changes in the anatomical properties of the cortical tissue, and these developmental changes in face-selective regions are associated with growth rather than pruning as proposed by the prevailing view (Rakic et al. 1986).

While substantial progress has been made examining the interaction of white matter, gray matter, function, and behavior of the ventral face network, additional anatomical factors, as well as the relationship among factors, are yet to be considered. One future topic to extend the examples of this section includes determining how additional anatomical factors constrain the development of the face network, and, in turn, affect behavior. For example, do aspects of the microstructure (e.g., dendrites/cell bodies/oligodendrocytes) or white matter associated with the face network also develop from childhood to adulthood? If so, is this development related to the development of gray matter, face selectivity, and/or face perception? What factors are perturbed in developmental disorders? Methodologically, this will likely require new in vivo tools or novel ways to link in vivo measurements to histological measures in postmortem tissue (Gomez et al. 2017, Weiner et al. 2017).

5. COMPUTATIONS BY POPULATION RECEPTIVE FIELDS IN THE VENTRAL FACE NETWORK

As described in the prior sections, the field has accrued a considerable body of knowledge regarding the functional characteristics of the ventral face network, the anatomical composition and connectivity of regions within the ventral face network, and their general relationship to perception. However, the specific computations of each region and how computations from multiple regions are coordinated to mediate perception remain elusive. In this section, we describe progress in understanding basic computations performed by pRFs across the ventral stream and how top-down attention affects these computations.

A logical starting point for developing computational models of the ventral face network is to determine receptive field (RF) properties reflective of ventral face-selective regions. Understanding RFs is important for three reasons. First, RFs are a fundamental aspect of the processing performed by neurons in the visual system (Gross et al. 1969, Hubel & Wiesel 1962). Because neurons with similar RFs are spatially clustered, we can measure the population receptive field (pRF)—the region of the visual field that drives the population of neurons within a voxel Dumoulin & Wandell 2008, Wandell & Winawer 2015)—with fMRI. Second, face recognition is thought to require spatial integration across facial features rather than the processing of isolated facial features (Tanaka & Farah 1993; Van Belle et al. 2010, 2011). Thus, determining the location and size of pRFs in the ventral face network may inform our understanding of which parts of the face are processed in different stages of the face network. Third, understanding pRFs may shed light on fixation/viewing behavior. For example, when asked to recognize a face, participants typically fixate on the center of the face and the eyes, but when asked to judge the emotion of the face, people also fixate on the mouth (Pelphrey et al. 2002). These fixation behaviors suggest that the spatial capacity of face processing is limited and may also be task-dependent.

5.1. Population Receptive Field Measurements Reveal a Hierarchical Organization of the Ventral Face Network, with Implications for Perception

Recently, we performed a series of experiments in which we built encoding models (Naselaris et al. 2011) that characterize pRFs of the ventral face network. Subjects were scanned while viewing faces at different positions and sizes that systematically tiled the central visual field (12.5°). From these data, we obtained the amplitude of fMRI responses of each voxel as a function of face position and size. Then, for each voxel, we fit a model that predicted the response by computing the overlap of the stimulus with a 2D Gaussian, followed by a compressive nonlinearity (Kay et al. 2013, 2015). This model-based analysis (a) provides estimates of pRF properties such as size and eccentricity (distance from fixation), (b) allows comparison of pRF properties across the ventral stream hierarchy, and (c) provides insight into how space is represented in the ventral face network.

Population-receptive-field mapping shows that voxels in the ventral face network are substantially modulated by the location and spatial extent of faces in the visual field, and our simple pRF model adequately explains these modulations. Population receptive fields in the ventral face network are characterized by four main features. First, pRF centers are located in the contralateral visual field (Hemond et al. 2007, Kay et al. 2015), which is consistent with retinotopic organization of visual cortex more generally (Holmes 1918). That is, pRF centers within voxels of the right hemisphere are centered in the left visual field and vice versa. Second, pRFs in the ventral face network tend to be located close to the center of gaze (Kay et al. 2015, Witthoft et al. 2016) rather than distributed across the visual field as in early and intermediate retinotopic areas (V1–hV4, Figures 6a and 7b). Third, the average pRF size progressively increases from V1 to subsequent retinotopic areas (V2–hV4), and into the face network (IOG-, pFus-, and mFus-faces) (Figure 6b) (Kay et al. 2015, Witthoft et al. 2016). This progressive increase of average pRF size is consistent with a hierarchical organization of visual cortex (Van Essen & Gallant 1994). Fourth, pRF size increases with eccentricity in the face network (Figure 6c) (Kay et al. 2015, Witthoft et al. 2016) as in earlier regions (Wandell & Winawer 2015). Additionally, the slope of this relationship increases across the visual hierarchy. Consequently, the size of pRFs in the ventral face network is larger than their eccentricity. Thus, pRFs in the ventral face network always cover the center of gaze (fovea) and extend to the ipsilateral visual field, processing information from both right and left visual fields.

Figure 6.

Population receptive fields (pRFs) reveal a hierarchical organization of the ventral face network. Population receptive fields were estimated from functional magnetic resonance imaging responses to faces presented at different visual field locations. For the data in this panel, pRFs were estimated in a task during which participants fixated on a central crosshair and reported when they detected a small dot superimposed on some of the faces. (a) Median eccentricity of pRF centers for areas constituting the ventral visual stream from V1 to mFus-faces. Error bars: 68% confidence intervals (CIs). (b) Median pRF size for each area from V1 to mFus-faces. Error bars: 68% CIs. Population-receptive-field size is defined as the standard deviation of a two-dimensional Gaussian that characterizes the response of the pRF to point stimuli. (c) Relationship between pRF size and eccentricity. Shaded area: 68% CI on a line fitted to the data. (d) Facial features processed by example pRFs across the ventral visual processing hierarchy. Circles: pRFs at 1^° eccentricity (derived from panel c). Each circle is drawn at ± 2 pRF sizes. The depicted face is sized to simulate a face seen from a conversational distance of 1 m, approximately 6.5^°, based on average male head sizes (Loftus & Harley 2005, McKone 2009). Ascending the hierarchy, spatial information is integrated across increasingly larger regions of the face until the latest stages where entire faces are processed by the neural population within a voxel. Adapted from Kay et al. (2015). Abbreviation: IOG, inferior occipital gyrus.

Figure 7.

Attention modulates population receptive field (pRF) properties in the ventral face network, enhancing spatial representations. Population receptive fields were measured under different tasks using the same stimulus. For the data in this panel, subjects performed either a one-back task on centrally presented digits (digit task) or a one-back task on the presented face (face task). (a) Task-induced changes in pRF properties. Bars: median across voxels; error bars: 68% confidence intervals (CIs). pRFs in IOG-, pFus-, and mFus-faces, and hV4 are larger (left), are more eccentric (middle), and have increased gain (right) during the face task (blue) compared to the digit task (red). (b) Tiling of visual field by 100 randomly selected pRFs from left pFus-faces. Dots: pRF centers; circles: pRFs drawn at ± 2 pRF sizes; intensity: pRF gain (see colorbar). An example face is shown at 5^° eccentricity. (c) Spatial uncertainty in discrimination of stimulus positions. Bars: amount of uncertainty for reference positions at 5^° eccentricity (median across angular positions ± 68% CI). During the digit task, uncertainty in IOG-, pFus-, and mFus-faces is large. However, during the face task, uncertainty is substantially reduced and is commensurate with the uncertainty in V1. Adapted from Kay et al. (2015). Abbreviation: IOG, inferior occipital gyrus.

An intuitive illustration of the visual information processed by pRFs across the ventral processing stream is shown in Figure 6d. Here, we show example pRFs at 1^° eccentricity across several regions spanning the ventral stream; these pRFs are superimposed on a face sized to approximate typical conversational distance (Loftus & Harley 2005, McKone 2009). The figure illustrates that, for a face in a typical viewing distance, a V1 pRF processes local information, such as the corner of the eye, whereas a pRF in hV4 may process an entire facial feature, such as an eye. However, the large and foveal pRFs in the face network spatially integrate information, which enables the processing of information across multiple facial features from both sides of the face. These data suggest a potential explanation for fixation behavior: When people fixate faces, they attempt to position pRFs in face-selective regions so as to optimally spatially integrate information across facial features.

Data comparing pRFs in the ventral face network in individuals with DP and typical individuals support this idea that spatial integration across facial features obtained by large and central pRFs is important for face perception (Witthoft et al. 2016). These data show that pRFs in the face network (and hV4) are smaller, and rarely extend to the peripheral or ipsilateral visual field, in individuals with DP compared to typical adults. Notably, across both typical individuals and those with DP, face recognition ability is positively correlated with pRF size in the ventral face network: Participants with larger pRFs perform better than those with smaller pRFs. In contrast, face recognition ability does not correlate with pRF size in early retinotopic areas. These data (Witthoft et al. 2016) provide empirical evidence suggesting that smaller pRF sizes in individuals with DP may reflect a deficit in spatial integration, which consequently affects face recognition.

5.2. Attention Modulates Population Receptive Field Properties, Enhancing Peripheral Representations Where Visual Acuity Is the Worst

Responses in the ventral face network are not just stimulus-driven but also modulated by internal top-down goals (Avidan et al. 2003, O’Craven et al. 1999, Summerfield et al. 2008, Yi et al. 2006). We characterized how top-down factors modulate pRF properties and spatial information by measuring pRFs under different attentional states (Kay et al. 2015). Population receptive fields were estimated separately for different tasks using identical visual stimulation, including a digit task, during which attention was directed toward digits presented at fixation, and a face task, during which attention was directed toward faces, which appeared at various locations tiling the visual field (subjects performed a one-back judgment on digits and the faces, respectively).

Fitting the pRF model separately to responses observed under different tasks, we found that pRFs in the ventral face network and hV4 are task-dependent. In these regions, pRFs are located more peripherally (Figure 7a, left), are larger (Figure 7a, center), and have a higher gain (Figure 7a, right) while participants attended to faces compared to when they attended to digits presented at fixation. In contrast, pRFs in early visual areas (V1–V3) were relatively stable and did not substantially change across tasks (Figure 7a).

To obtain an intuitive understanding of the effect of attentional modulation on pRFs in the ventral face network, we visualize the collection of pRFs from left pFus-faces measured in the digit (Figure 7b, left) and face tasks (Figure 7b, right). As pRFs are larger and more eccentric in the face compared to the digit task, there is extended coverage of the peripheral visual field during the face task compared to the digit task. For example, a face presented at 5^° eccentricity from fixation is processed by only a handful of pRFs during the digit task but by many pRFs during the face task.

To interpret the change in spatial representations across tasks, we used a decoding approach and quantified the spatial uncertainty of the location of a face presented at 5^° eccentricity from responses of a collection of pRFs spanning each visual area. Results show that the spatial uncertainty in decoding the location of the face from the collection of pRFs in the face network substantially decreases from the digit to face task (Figure 7c). Surprisingly, the spatial uncertainty in the face network during the face task is similar to that obtained by V1 pRFs, which are considerably smaller. These results illuminate another aspect of spatial coding: What determines the spatial resolution of processing by a collection of pRFs is not only their size but also their scatter. In other words, large and partially overlapping pRFs may provide similar spatial precision as small and nonoverlapping pRFs, consistent with the notion of coarse coding (Snippe & Koenderink 1992, Weiss et al. 1993). Overall, pRF measurements reveal that spatial information is systematically represented in the face network and that attention modifies and enhances these spatial representations.

Our research of pRF properties is only a first stepping-stone of building accurate encoding models of the face network, as we have implemented a rudimentary spatial filter that performs only spatial summation and applies the same type of spatial filtering throughout the ventral stream. Thus, important future elaborations of the pRF model would be to (a) add additional dimensions to the model that explain computations related to additional aspects of the stimulus [e.g., its shape and/or features (Kay & Yeatman 2017)], and (b) determine whether and how additional pRF properties vary within each cortical region and across regions. For example, implementing an array of orientation-selective filters for each V1 voxel, rather than just Gaussian filters, provides better predictive power in explaining V1 responses to natural images (Kay et al. 2008). In the domain of face processing, elaborating pRFs is necessary for explaining basic response properties of face-selective regions not explained by the present pRF model, such as preferential responses and tuning to individual faces (Gratton et al. 2013, Kriegeskorte et al. 2007, Nestor et al. 2011) and face parts (de Haas et al. 2016, Henriksson et al. 2015).

6. COMPUTATIONAL INSIGHTS FROM ANATOMICAL AND FUNCTIONAL FEATURES OF THE FACE NETWORK

The empirical findings reviewed here reveal an organized and reproducible implementation of a neural processing system for face perception in the human brain. Because generating an organized structure is more effortful than generating a disorganized structure, it is possible that certain optimization and computational principles generated this functional architecture over the course of evolution or development. Therefore, computational insights can be gleaned from the specific features of the physical implementation of the ventral face network in the brain. Here, we highlight some of the architectural and functional features that have not yet been implemented in computational models and consider putative computational aspects of these features that can be tested in future research.

With respect to encoding models such as pRFs, described above, an important aspect of these models that can be further developed is explicitly modeling how anatomical features may contribute to the formation of dynamic, task-dependent pRFs. The present pRF approach simply treats each task as distinct and estimates a model of the stimulus representation separately for each task. This provides useful insight, but further research is necessary to identify the neural mechanisms that underlie the source of the task modulations originating from other brain areas. To better understand how attention may modulate pRFs, one could consider how white matter connections of the face network—for example, top-down connections from cortical regions outside the face network (such as the connection from IPS-0 to pFus-faces), in addition to both bottom-up connections from early visual areas and earlier face-selective regions to higher face-selective regions—sculpt pRFs in the ventral face network. Thus, dynamic pRFs may be an outcome of an interaction between a static pRF generated by bottom-up connections and an attention field (Klein et al. 2014, Reynolds & Heeger 2009, Sprague & Serences 2013) mediated by top-down connections from IPS-0. Developing new encoding models that incorporate these anatomical features (e.g., Kay & Yeatman 2017) may reveal insight into the interplay between top-down and bottom-up processing in the face network—a topic that has been elusive thus far.

We believe that, in addition to macroscopic anatomical features such as white matter connections, there is also utility in incorporating microanatomical features into computational models. For example, cytoarchitectonic differences between pFus- and mFus-faces suggest that these regions have different neural hardware that may be optimized to perform different types of computations. These data therefore suggest that computational models should not necessarily implement a single generic neural computation or filter type that is duplicated across processing stages. Instead, there are likely different specialized computations occurring at different stages of processing. However, beyond initial computational models (Çukur et al. 2013a, Kay et al. 2015), a full comprehensive computational model of pFus- and mFus-faces is still lacking. Future experiments testing responses to a larger array of stimuli and modeling additional pRF dimensions (e.g., shape, viewpoint) could determine a complete model of computations in these regions and comprehensively test on what aspects they may differ. Furthermore, an ambitious and interesting direction for future work is to forge explicit links between anatomical properties such as cytoarchitecture, receptor architecture, myeloarchitecture, cell types, and microcircuit connections relative to the computational properties of neurons in the ventral face network.

Finally, a promising avenue of future research would be to relate recent empirical findings to recent developments in deep neural networks. Deep neural networks are capable of reaching human-like performance in complex object recognition tasks (Kriegeskorte 2015, Yamins &16), including face recognition (Taigman et al. 2014), and predict, to a noteworthy level of accuracy, experimentally measured responses in the primate and human ventral stream (Cadieu et al. 2014, Eickenberg et al. 2017, Guclu & van Gerven 2015, Jiang et al. 2006, Khaligh-Razavi & Kriegeskorte 2014, Kubilius et al. 2016, Yamins et al. 2014). Although these architectures are broadly “neurally inspired” by architectural features of the ventral visual stream (Fukushima 1982, Riesenhuber & Poggio 1999, Serre et al. 2007), there are a number of shortcomings. First, it remains to be seen how well these networks characterize and predict specific stimulus-tuning properties, such as pRFs reviewed above. Second, these networks also are purely stimulus-driven and, therefore, do not account for effects of task and attention on responses in the ventral face network. Finally, the computational architecture of most current deep neural networks is strictly serial and, thus, do not appear to respect the biological reality that the ventral face network contains bypass routes that skip stages of the processing hierarchy. Therefore, incorporating recent empirical findings described in this review into deep neural network architectures both may make them more accurate models of the human visual system and could advance our understanding of hypothesized benefits of specific architectural features. For example, the hypothesis that bypass connections provide resiliency to cortical damage (Weiner et al. 2016a) could be tested by comparing the effect of a virtual lesion of an intermediate layer of a strictly hierarchical neural network versus a neural network containing bypass connections.

7. CONCLUSIONS

As described in this review, neuroimaging research has advanced our understanding regarding the functional architecture of the ventral face network and how this architecture contributes to computations underlying human perception. For example, macroanatomically, the scale of our understanding has improved from centimeters to millimeters in which the spatial arrangement of regions within the ventral face network relative to cortical folding can be predicted with millimeter precision. Recent research has linked this consistent spatial arrangement of the ventral face network to underlying microanatomical features such as cytoarchitecture (Weiner et al. 2014, 2017), as well as long-range white matter connectivity (Gomez et al. 2015, Gschwind et al. 2012, Pyles et al. 2013, Saygin et al. 2012, Weiner et al. 2016a). Mechanistically, the development of new methods deriving pRFs (Dumoulin & Wandell 2008; Kay & Yeatman 2017; Kay et al. 2013, 2015) has begun to elucidate computational principles of the ventral face network. Although there are key questions that remain unanswered, these new research directions have opened exciting new opportunities for understanding how the functional neuroanatomical features of the face network contribute to computations underlying human face perception with unprecedented precision.

SUMMARY POINTS.

1. Face perception is mediated by a cortical network in ventral temporal cortex that has a reproducible organization constrained by underlying anatomical features such as the cellular architecture (cytoarchitecture) of the cortex and white matter connections.

2. Regions within the ventral face network have differential cytoarchitectonic structure, which suggests that the neural hardware is optimized for performing different computations within separate regions.

3. The functional topography and resiliency of the ventral face network is supported by both within-network connections among face-selective regions and between-network connections that interconnect face-selective regions with regions outside the face network.

4. White matter properties of the ventral face network are behaviorally relevant: They correlate with face recognition ability and demonstrate tissue abnormalities in individuals with face recognition deficits.

5. Computations within the ventral face network can be quantified with population receptive fields (pRFs) that both explain responses as a function of face location and size and form a basis for spatial integration across facial features.

6. Attention modulates pRF properties, consequently improving spatial representations in the periphery where visual acuity is the worst. These modulations provide critical insight into how attention changes the representation of information in the ventral face network.

7. Future work can incorporate empirical findings of pRF properties as well as anatomical information such as cytoarchitecture and white matter connections to generate neurobiologically accurate computational models of face perception.

FUTURE ISSUES.

1. What is the three-dimensional structure, cell type composition, and local connectivity of cells composing the cortical tissue of each face-selective region?

2. What are the basic neural circuits in face-selective regions, and what computations do these circuits perform?

3. What is the functional utility of white matter connections within the face network and to regions outside the network?

4. How do white matter connections contribute to the spatial segregation of face-selective regions of the ventral face network?

5. How does the interplay among the development of white matter, gray matter, and function of face-selective regions affect face perception and recognition?

6. What are additional dimensions of population receptive fields (pRFs) (e.g., shape, viewpoint) in the face network, and how do they vary within and across regions?

7. Why is there a multiplicity of face-selective regions? Do the three face-selective regions of the ventral face network contribute to different facets of face perception and recognition, or do they contain redundant information?

8. Will incorporating newly discovered implementational features of the face network into encoding models and deep neural networks improve their performance, resulting in better predictions of human behavior?

WHAT IS THE CAUSAL ROLE OF FACE-SELECTIVE REGIONS IN FACE PERCEPTION?

Neuroimaging is critical for revealing the functional architecture of the face network in awake, behaving humans. However, it is important to acknowledge that one main limitation of neuroimaging is that it cannot be used to determine the causal role of cortical regions for behavior. Thus, researchers use methods such as electrical brain stimulation (EBS), transcranial magnetic stimulation, and lesion studies to determine the causal role of face-selective regions in face perception. In EBS experiments, researchers first identify the selectivity of electrodes using electrocorticography (or fMRI). Then they apply EBS to test how disruption of normal brain activity under the electrode affects behavior.

EBS reveals three main findings: (a) stimulation of right inferior occipital gyrus (Jonas et al. 2012, 2014) or right pFus- and mFus-faces (Allison et al. 1994a, Parvizi et al. 2012, Puce et al. 1999, Rangarajan et al. 2014) results in perceptual deficits specific to faces, (b) stimulation of right, but not left, face-selective electrodes in the fusiform gyrus (FG) generates a deficit in face perception (Rangarajan et al. 2014), and (c) it is difficult to generate a hallucination of a face by stimulating face-selective electrodes in the FG when subjects have their eyes closed (Murphey et al. 2009).

Glossary

Population receptive field (pRF)

in functional magnetic resonance imaging, the region of visual space that stimulates a voxel

pFus-faces/FFA-1

a face-selective region typically overlapping the posterior-lateral tip of the mid-fusiform sulcus, located within cytoarchitectonic area FG2 and 1–1.5 cm posterior to mFus-faces/FFA-2

mFus-faces/FFA-2

a face-selective region overlapping the anterior-lateral tip of the mid-fusiform sulcus, located within cytoarchitectonic area FG4 and 1–1.5 cm anterior to pFus-faces/FFA-1

Fusiform face area (FFA)

once considered a homogenous face-selective area, it contains (at least) two cytoarchitectonically and functionally distinct components

Inferior occipital gyrus (IOG)

a gyrus in the occipital lobe that is posterior-lateral to the fusiform gyrus; considered the first processing stage of the ventral face network

Fusiform gyrus (FG)

a hominoid-specific macroanatomical structure in ventral temporal cortex that contains (at least) four cytoarchitectonic areas and multiple functional regions

Collateral sulcus (CoS)

a primary sulcus in human ventral temporal cortex; the medial boundary of the fusiform gyrus

Occipito-temporal sulcus (OTS)

a primary sulcus in human ventral temporal cortex; the lateral boundary of the fusiform gyrus

Mid-fusiform sulcus (MFS)

a shallow, longitudinal sulcus bisecting the fusiform gyrus; a landmark identifying cytoarchitectonic and functional boundaries

Cytoarchitecture

cellular organization across the six-layered cortical ribbon; a property used to parcellate brain areas from one another

FG1–4

labels for four cytoarchitectonic areas in the fusiform gyrus and neighboring sulci

Parahippocampal place area (PPA)

a place-selective region in the collateral sulcus and parahippocampal cortex

Developmental prosopagnosia (DP)

an impairment in recognizing faces despite normal vision, intelligence, and socio-cognitive abilities and no history of brain damage

Quantitative magnetic resonance imaging (qMRI)

a noninvasive imaging method that measures T₁ relaxation time and estimates macromolecular tissue volume

Receptive field (RF)

the region of the visual field within which a visual stimulus elicits a response from a neuron

Encoding model

computational model that explicitly identifies a set of features and computations that predicts evoked brain responses

Decoding

inferring information from distributed patterns of brain activity across a collection of neurons or voxels