A familiar face and person processing area in the human temporal pole

Significance

In order to socially interact with people we know, we must first recognize who they are. Familiar face perception has long been studied behaviorally and argued to rely on a distinct process from unfamiliar face perception. However, known face areas in the human brain respond to both unfamiliar and familiar faces, and it has remained unclear whether there are additional areas involved specifically in processing familiar faces. Here, we identify a brain region within the temporal pole that responds selectively to images of familiar faces over scenes, objects, and unfamiliar faces. The functional profile of this area suggests that it plays a role in connecting visual images of familiar faces with abstract concepts of familiar people.

Abstract

How does the brain process the faces of familiar people? Neuropsychological studies have argued for an area of the temporal pole (TP) linking faces with person identities, but magnetic susceptibility artifacts in this region have hampered its study with fMRI. Using data acquisition and analysis methods optimized to overcome this artifact, we identify a familiar face response in TP, reliably observed in individual brains. This area responds strongly to visual images of familiar faces over unfamiliar faces, objects, and scenes. However, TP did not just respond to images of faces, but also to a variety of high-level social cognitive tasks, including semantic, episodic, and theory of mind tasks. The response profile of TP contrasted with a nearby region of the perirhinal cortex that responded specifically to faces, but not to social cognition tasks. TP was functionally connected with a distributed network in the association cortex associated with social cognition, while PR was functionally connected with face-preferring areas of the ventral visual cortex. This work identifies a missing link in the human face processing system that specifically processes familiar faces, and is well placed to integrate visual information about faces with higher-order conceptual information about other people. The results suggest that separate streams for person and face processing reach anterior temporal areas positioned at the top of the cortical hierarchy.

As we interact with familiar people such as friends, family, and coworkers, their faces convey a wealth of information—from the essential question of who they are to subtle cues about their feelings and thoughts. Behavioral research has demonstrated that familiar faces are perceived more effectively than unfamiliar faces, with more robust recognition across changes in viewpoint, expression, and context, leading to arguments for a qualitatively distinct system (1–4). What processes in the brain underlie the perception of familiar faces?

Models of familiar face perception have long argued for the presence of a “person identity” node, involved in connecting faces with social identity representations stored in long-term memory [Fig. 1A, (5)]. Neuropsychological evidence suggests that such a node might exist in the temporal pole (TP, also termed area 38 or TG), a primate-specific brain area at the anterior tip of the temporal cortex. Patients with dysfunction of TP from brain damage, surgical resection, or frontotemporal dementia have been found to have difficulties recalling semantic information about familiar people, and/or connecting familiar faces with person identities [associative prosopagnosia, (6–8)].

Fig. 1.

Face-preferring regions within the human temporal pole (TP). (A) The classical model of cognitive operations supporting face recognition. (B) Human brain areas that are known to respond to generic face images. (C) Examples of face and scene images. (D) Examples of raw data obtained using a multiecho pulse sequence. (E) Substantially higher tSNR was obtained using multiecho data with multiecho ICA preprocessing, relative to single-echo data with no ICA. (F) Anatomical regions were hand-drawn to distinguish TP (area 38) from other neighboring regions of the anterior temporal cortex: perirhinal cortex (PR), anterior superior temporal sulcus (ASTS), and anterior inferotemporal cortex (AIT). (G) fMRI responses to faces versus scenes within TP (whole-brain general linear model-based analysis, thresholded at a false discovery rate of q < 0.01). (H) Region-of-interest (ROI)-based responses of face-preferring TP. Left: mean response across visual conditions. Right: contrast values by participant. *P < 0.05/4 = 0.0125, **P < 10⁻³, ***P < 10⁻⁴ (ROIs defined as the top 5% of face-preferring coordinates within anatomical search space; responses extracted from independent data; linear mixed model across runs, with participant included as random effect, error bars show SE across runs). (I) Responses to faces versus scenes within PR. (J) ROI-based responses within PR.

However, neuroimaging studies of brain responses to familiar faces have not reliably found responses in TP (9–13). Studies presenting images or videos of unfamiliar and familiar faces have most commonly found responses in regions of the posterior occipitotemporal cortex: the fusiform gyrus (fusiform face area, FFA), lateral occipital cortex (occipital face area, OFA), and posterior superior temporal sulcus (PSTS) [Fig. 1B, (14–16)]. A relatively weak response has been observed in the ventral anterior temporal cortex, ventral and inferior to the temporal pole [anterior temporal lobe face area, ATL, (17–19)]. Using images of personally familiar or famous faces, responses have additionally been observed in areas of the association cortex—including medial prefrontal cortex (MPFC), medial parietal cortex (MPC), and the temporo-parietal junction (TPJ)—thought to be more broadly involved in social cognition (11–13). The lack of a reliable TP response to familiar faces may result from magnetic susceptibility artifacts in the anterior temporal lobes caused by the nearby presence of the sphenoid sinus, leading to lower temporal signal to noise ratio (tSNR) and power to detect effects in this area.

Our recent work has identified two face areas in the anterior temporal cortex in the macaque: one in TP, and one in the perirhinal cortex (PR, or areas 35 to 36, 20). These zones of the cortex have well-characterized cytoarchitecture and anatomical connectivity. Both areas have direct, bidirectional connections with the hippocampal formation via the entorhinal cortex and are thus well positioned to support long-term memory for faces (21–23). The cytoarchitecture and anatomical organization of these regions are similar across humans and macaques (24). This raises the question: Do humans have face-selective brain areas within the temporal pole and perirhinal cortex?

To address the question of whether humans have regions of TP and PR specialized for processing faces and/or people, we use three methodological advances. First, we present images of close personally familiar faces, to maximize the strength of mnemonic responses (Fig. 1C). Second, we use multiecho pulse sequences optimized to maximize tSNR in the ventral anterior temporal lobes (Fig. 1D). We obtained substantially higher tSNR in the anterior temporal lobes using multiecho data combined with multiecho ICA preprocessing, relative to single-echo data without ICA (Fig. 1E). Third, in order to precisely characterize the anatomical organization of familiar face responses, areas TP, PR, AIT, and anterior superior temporal sulcus (ASTS) were hand-drawn on individual anatomical images based on previously established cytoarchitectonic and gross anatomical boundaries [Methods, Fig. 1D, (25, 26)]. These methodological advances enabled the identification and precise anatomical characterization of anterior face responses in individual human brains.

Comparing responses to faces versus scenes—both familiar and unfamiliar—revealed face-preferring areas across multiple zones of the anterior temporal lobe (whole-brain general linear model-based analysis, corrected for temporal autocorrelation using prewhitening with an ARMA(1,1) model, and corrected for multiple comparisons across coordinates using a false discovery rate of Q < 0.01; Fig. 1 G and I and SI Appendix, Figs. S2 and S3). Face-preferring responses were observed within TP in the left and right hemispheres of all participants (20/20 comparisons, Fig. 1 G and I and SI Appendix, Figs. S2A and S3A). Within a given brain and hemisphere, between one and three distinct regions were observed. The anatomical localization of these regions within TP varied, with responses commonly observed at the anterior-most tip of the temporal cortex, in lateral aspects of the TP adjacent to ASTS, and in ventromedial aspects adjacent to the ventral anterior insula. By situating face responses within subject-specific anatomical parcellations, we were able to distinguish face responses in TP from nearby responses within ASTS, AIT, and PR. This result establishes the presence of reliable face responses within the human temporal pole, which can be identified within individual brains using optimized fMRI methods.

Face responses within PR were also observed across participants and hemispheres (20/20 comparisons, Fig. 1 E and F and SI Appendix, Figs. S2B and S3B). These regions were typically localized to the anterior-most aspect of the collateral sulcus, with some participants showing a second, more posterior region. This demonstrates the presence of a reliable face response within the human perirhinal cortex, broadly consistent in anatomical location with the previously described ATL face patch. Based on this result, we propose that the ATL face area be more precisely named the PR face area. Brain regions within TP and PR preferring faces were similar in functional organization to face-responsive areas previously observed in the macaque (20). Given the similarity of cytoarchitectonic and anatomical organization of areas 38 and 35/36 across humans and macaques, this leads to the natural hypothesis that face-responsive regions of TP and PR are homologous across species. This contrasts with prior accounts, which have argued for a correspondence between anterior temporal face areas in humans with macaque area AM (17).

To what extent are neural responses within the anterior temporal cortex selective for faces, among other categories of visual image? We addressed this question using a region-of-interest (ROI) analysis. Functional ROIs were defined as the top 5% of face-versus-scene-preferring coordinates within subject-specific anatomical search spaces, and response magnitudes across multiple categories were extracted in independent data. Throughout the remainder of the paper, the terms TP, PR, ASTS, and AIT will refer to face-preferring ROIs within these broader anatomical regions. We found that TP was strongly selective for faces over other categories (Fig. 1H). Responses to faces were above a resting baseline, while responses to scenes and objects were below baseline. Both left and right TP responded significantly more strongly to images of familiar faces than familiar scenes and generic objects (all P’s < 10⁻⁸, individual values in SI Appendix, Tables S1 and S2; linear mixed effects model across runs, with participant included as a random effect). Selective responses to faces were also observed in bilateral PR, ASTS, and AIT (familiar faces versus familiar scenes and generic objects, all P’s < 10⁻⁴, Fig. 1J and SI Appendix, Fig. S4). Similar responses across conditions were observed when areas were using face versus scene or face versus object contrasts (SI Appendix, Fig. S5). Face responses were observed consistently across ROI size and participants in all areas (Fig. 1H and SI Appendix, Figs. S6 and S7). We defined a face selectivity index as $\mathrm{S}=\mathrm{U}\mathrm{F}-\mathrm{O}/\left(\left|\mathrm{U}\mathrm{F}\right|+\left|\mathrm{O}\right|\right)$, where UF is the response to unfamiliar faces, and O is the response to objects. Face selectivity was similar to or greater in anterior temporal face areas than in classic face areas OFA, FFA, and PSTS (SI Appendix, Fig. S8). The anterior temporal lobe therefore contains multiple regions that respond selectively to faces over objects and scenes.

Neuropsychological results indicate a specific role for the temporal pole in processing familiar faces (6, 8). To what extent are TP responses modulated by familiarity? We compared responses to familiar and unfamiliar faces, using identities that were matched on age, race, and gender. Across both hemispheres, TP responded substantially and significantly more to familiar than unfamiliar faces (Fig. 1H, both P’s < 0.001, mean response increase 76%). These familiarity effects were consistent across ROI size and participant (SI Appendix, Figs. S6 and S9). In contrast, PR responded similarly to familiar and unfamiliar faces, with a weak but significant difference observed only in the left hemisphere (Fig. 1J and SI Appendix, Figs. S6 and S9, Left P < 0.0125 = 0.05/4, Right P = 0.07, mean response increase 13%). This suggests a functional dissociation across anterior temporal face regions, in which PR contributes to processing all faces, while TP is specifically involved in processing familiar faces.

Is the functional role of TP limited to face processing, or does this area contribute more broadly to cognitive processes involved in understanding and remembering familiar people? To address this question, we scanned the same set of participants on an additional battery of tasks, eliciting visual, semantic, and episodic processing of other people, as well as theory of mind (ToM) reasoning. In the semantic task, participants rated personality traits of familiar people. In control conditions, they rated physical properties of generic objects, or spatial/navigational properties of familiar places. In the episodic task, participants imagined familiar people talking about specific conversation topics. In control conditions, they imagined generic objects engaged in physical interactions, or imagined navigating through familiar places. In the dynamic visual task, participants viewed images of unfamiliar faces, objects, and scenes (27). In the ToM task, participants read stories and answered true/false questions that either required understanding of a false belief or a false “photograph,” i.e. physical representation (28, 29).

Consistent with the face preference described above, TP bilaterally showed an increased response to videos of unfamiliar faces versus objects and scenes (Fig. 2, dynamic visual task, all P’s < 0.005). However, TP also responded strongly to familiar people over object and place controls in the semantic task. Responses during trait judgment about familiar people were substantially and significantly stronger than responses while judging properties of familiar places and generic objects (all P’s < 0.005). TP also responded strongly to familiar person over object and place controls in the episodic task (all P’s < 0.0125). Last, right TP responded more when evaluating unfamiliar characters’ mental states in the ToM task, relative to physical states of the world (P < 10⁻⁴, left TP trending at P = 0.02). TP thus shows a domain-specific response when processing familiar people across multiple tasks, which have a variety of stimulus features and behavioral demands. This suggests that TP is involved in social cognition and long-term memory – not just face perception—and is better characterized as person-selective than face-selective per se.

Fig. 2.

Region-of-interest (ROI)-based analysis reveals separate streams for person and face processing in the anterior temporal cortex. (A) Multiple tasks were used within individual participants, to determine whether areas responded during high-level social cognitive processing, in addition to face perception. Tasks included viewing images (visual task), trait judgment (semantic task), imagining events (episodic task), viewing videos (dynamic visual task), and reasoning about mental or physical representations (theory of mind task). (B) ROI responses from the face-preferring part of left temporal pole (TP). (C) Responses from right TP. (D) Responses left perirhinal cortex (PR). (E) Responses from right PR. (F) Responses from the right occipital face area (OFA). (G) Responses from the right fusiform face area (FFA). All ROIs were defined as the top 5% of face-preferring coordinates within anatomical search spaces. Responses (% signal change) were extracted from functionally defined ROIs across all task conditions. Error bars show SE across runs. *P < 0.05/4 = 0.0125, **P < 10⁻³, ***P < 10⁻⁴ (linear mixed model across runs, with participant included as random effect).

In contrast to the broad social responsiveness observed in TP, face-preferring regions of PR showed a strong response to faces but not other social conditions. PR responded bilaterally to videos of unfamiliar faces over scenes and objects (Fig. 2, all P’s < 0.001), but responded neither strongly nor reliably to people over places and objects in semantic and episodic tasks (left P’s ranging from 0.008 to 1; right P’s ranging from 0.09 to 1). A mixed effects two-way ANOVA across data from TP and PR identified an interaction between task condition and ROI [P < 10⁻³⁷, F(17,3524) = 17.1] as well as main effects of condition [P < 10⁻⁴⁸, F(17,3524) = 13.4] and ROI [P < 10⁻¹¹, F(1,3524) = 43.1]. Face-preferring regions of ASTS and AIT showed similar patterns of response to TP, with social preferences in both visual and cognitive tasks (SI Appendix, Fig. S4). Response profiles thus contrast starkly among different face-preferring regions of the anterior temporal cortex. This finding argues for two distinct streams of social information processing in the anterior temporal lobes: one for person processing and another for face processing.

How are face-preferring areas of TP and PR situated as components of broader neural systems involved in social cognition and face perception? To ask this question, we defined face-preferring functional ROIs within anatomical search spaces covering brain areas previously implicated in social cognition and face processing. Regions of the social network included MPC, MPFC, TPJ, superior frontal gyrus (SFG), and middle superior temporal sulcus (MSTS). Regions of the face network included the FFA, OFA, and PSTS (Fig. 3A).

Fig. 3.

Functional connectivity shows that face-preferring temporal pole (TP) and perirhinal cortex (PR) are associated with separate large-scale networks. (A) Functional ROIs shown in color, along with black outlines surrounding the anatomical search spaces used to define them. Social network regions include medial prefrontal cortex (MPFC), medial parietal cortex (MPC), superior frontal gyrus (SFG), temporo-parietal junction (TPJ), and medial superior temporal sulcus (MSTS). Face network regions include fusiform face area (FFA), occipital face area (OFA), and posterior superior temporal sulcus (PSTS). (B) Plot of “social versus visual” modulation indices, capturing the relative degree of person selectivity in semantic and episodic tasks compared to the visual task. Hyp: areas hypothesized from prior work to have social or visual responses. Exp: experimental areas tested in this study. (C) Whole-brain resting-state correlation maps from seeds in face-preferring right TP (Left) and right PR (Right). (D) Correlations from face-preferring TP, PR, anterior superior temporal sulcus (ASTS), and anterior inferotemporal cortex (AIT) with areas of previously described social cognition and face perception networks. (E) These results argue for separate streams for face and person processing within the anterior temporal lobe, providing separate bridges from the hippocampal formation to social cognition areas of the association cortex, and to face perception areas in the ventral visual stream.

We first compare functional response profiles of TP and PR with social and face networks. Based on prior research on these systems, we expected face areas to only have a domain preference, but for social network areas to have a person preference in conceptual tasks as well. Like TP, social network regions responded to people over places and objects across visual, semantic, and episodic tasks (all P’s < 0.0125, SI Appendix, Fig. S10 and Table S5 and S6). All social network regions except left MSTS also responded significantly more strongly to mental state versus physical reasoning in the ToM localizer (left MSTS P = 0.1, other P’s < 0.0125). In contrast, face network regions only responded strongly to visual face conditions over objects and scenes (all P’s < 0.0125, SI Appendix, Fig. S11 and Table S7). Among the other tasks, similar responses were observed to people, places, and objects, with only some small but significant differences observed in the semantic task (P’s ranging from 0.0002 to 1). Face network regions were not modulated by the ToM localizer, except for a small effect in right PSTS (right PSTS P < 0.001, other P’s > 0.2).

We computed a “social versus visual” modulation index, which captures the extent to which selectivity for people over object and place conditions is larger in cognitive tasks (social > visual), larger in the visual task (social < visual), or similar across cognitive and visual tasks (social = visual, Fig. 3B). As we expected, larger modulation values were observed for areas of the social relative to visual network regions. Like social network areas, TP showed stronger social selectivity in cognitive relative to visual tasks. In contrast, PR and face network areas had stronger modulation in visual relative to cognitive tasks. These results demonstrate that TP is similar in response profile to areas of the social network, while PR is similar to the face network.

To evaluate the functional network associations of TP and PR, we next measured synchronization of spontaneous fMRI signals in resting-state data. TP had reliable signal correlations with other social network regions within the association cortex, including in MPFC, MPC, SFG, TPJ, and STS (Fig. 3C and SI Appendix, Fig. S12). Face-preferring areas of TP—as well as ASTS and AIT—were significantly more correlated with social network areas than face network areas (Fig. 3D). In contrast, PR generally had low signal correlations with the rest of the cortex, showing weak but positive correlations with the fusiform gyrus (Fig. 3C and SI Appendix, Fig. S13), and significantly stronger correlations with face network versus social network areas (Fig. 3D). Taken together, these results indicate that TP constitutes a node in a broader network of the association cortex involved in social cognition, while PR is instead related to areas of the occipitotemporal cortex involved in face perception.

This study identifies a familiar face- and person-preferring region in the human temporal pole. These findings resolve a longstanding discrepancy between neuropsychological studies strongly indicating a role for the temporal pole in familiar face recognition and person understanding (6–8), and fMRI studies that have only rarely observed functional responses to familiar faces or people (9–13, 30–32). By presenting images of close personally familiar faces, and developing fMRI data acquisition and processing methods that boost signal quality in the anterior medial temporal lobes, we were able to reliably identify TP face responses in individual brains. By drawing anatomical regions based on established cytoarchitectonic criteria, face responses could be precisely localized to area 38. These results refine the view provided by neuropsychological studies, which typically involve alterations of large parts of the cortex, by identifying focal brain regions with domain-specific responses.

Prior models of familiar face perception have posited that posterior-to-middle ventral temporal areas, such as the OFA and FFA, support an initial generic face processing step, while anterior temporal cortex supports subsequent person recognition and retrieval of high-level social information from long-term memory (9, 16, 33, 34). Contrasting with this view, we present evidence for distinct systems for generic face and familiar person processing within the anterior temporal lobe, in regions of TP and PR that have direct anatomical connections with the hippocampal formation. This argues for distinct processing streams for people and faces that extend to the top of the cortical hierarchy.

Could the familiarity effect observed in TP result from increased attention to familiar faces? While we cannot rule out the possibility that familiar faces attracted participants’ attention, we consider this explanation unlikely. First, we note that familiar and unfamiliar face images were controlled for a range of low-level visual features including size, luminance, contrast, and saturation, suggesting against any differences in bottom–up attention. Second, familiar and unfamiliar images were equally task-relevant to the one-back task, with similar performance across conditions (Methods). This suggests against task-driven effects of attention, and demonstrates that participants paid attention to unfamiliar face stimuli. Third, we did not observe an increased response to familiar over unfamiliar scenes, which suggests against an additive effect of attention to familiar content. Last, the response to familiar relative to unfamiliar faces in TP was large: 112% (left TP) and 139% (right TP). This was substantially larger than familiarity effects observed in other face-responsive areas: for instance, 13% for the FFA and 22% for PR. The hypothesis that familiarity effects are driven by a multiplicative effect of attention does not explain this difference between TP and PR, which are similarly anatomically situated in the anterior temporal cortex. These arguments suggest that instead of attention, the increased TP response to familiar faces is driven largely by an effect of long-term memory.

Our study focused on face responses in TP and PR because these areas are 1) poorly understood in humans, due to susceptibility artifacts, 2) recently characterized in the macaque, and 3) well situated to contribute to long-term memory, given their connectivity with the hippocampal formation. We additionally characterized responses in neighboring areas ASTS and AIT and found that both areas exhibited response profiles similar to that of TP, showing a face response, familiarity effect, and person responses in semantic and episodic tasks. These responses are consistent with the previously reported component of the social network in the anterior STS and adjacent anterior lateral temporal cortex (29). This result suggests that TP shares its stimulus selectivity profile with a swath of surrounding anterior temporal cortex.

While prior fMRI studies have typically studied a specific cognitive or perceptual process and combined data across multiple participants in a standard template space, our approach assesses responses to a diverse set of tasks within individual human brains (35–37). This method enabled the identification of a strong functional dissociation in the response profiles of TP and PR. TP responded to a range of tasks involving familiar people, including semantic judgment and episodic simulation, while PR responded to images of faces but not social cognitive tasks. This dissociation fits well with prior neuropsychological results. The perirhinal cortex has been implicated in object memory and perception across humans, macaques, and rodents (38, 39). In contrast, damage localized to the human temporal pole typically does not impair the recognition of unfamiliar objects or faces but can impair familiar face recognition and person knowledge (6–8, 10). Thus, our results converge with neuropsychological evidence indicating separate streams for person and face processing within the anterior temporal lobe.

Face-preferring regions of TP and PR in humans had a similar functional organization to regions we have observed in macaques [Fig. 1G and (20)]. The similarity in cytoarchitectonic and gross anatomical organization of areas 38 and 35/36 across humans and macaques (24) suggests that TP and PR face responses may be homologous across species. Further work comparing the connectivity and response profiles of these areas across species will be needed to evaluate this potential homology.

The functional connectivity of TP and PR indicate distinct interactions with large-scale brain networks, with TP signal correlated with a social cognition network distributed across the association cortex, and PR more correlated with face processing areas in the ventral temporal cortex. These differences fit well with anatomical connectivity patterns of the ventral and temporopolar anterior temporal cortex established in the macaque. Cytoarchitectonic areas 38 and 35/36 both have anatomical connections with the entorhinal cortex, but their connectivity with other parts of the cortex differs. The temporal pole has strong connections with MPFC, orbitofrontal cortex, and dorsal STS (23), while perirhinal cortex shows strong connections with the nearby inferotemporal cortex (22). PR is thus anatomically positioned as an interface between the hippocampus and the ventral visual stream, while TP is positioned as an interface between the hippocampus and transmodal association cortex.

The hypothesis of corresponding areas across species indicates that insight into the function of the human anterior temporal cortex can be gained from invasive study of neuronal responses from these areas in macaques (40, 41). For instance, electrophysiological recordings from macaque TP have demonstrated that population responses encode the identity of familiar faces (40). These responses show a similar coding scheme to anterior parts of the inferotemporal cortex—representing face identity in a way that is invariant to view angle—but only coding for personally familiar faces. In combination with the present results, this suggests that TP—while broadly involved in person understanding—nevertheless contains a visual representation of face features. On this hypothesis, TP is well positioned to support the integration of visual face processing with higher cognitive aspects of social understanding, such as the representation of mental state and trait information about familiar people.

Methods

Participants.

Ten human participants (five male, five female; age 28 to 40) were scanned using fMRI. Participants were healthy with normal or corrected vision, right-handed, and native English speakers. The experimental protocol was approved by the Rockefeller University Institutional Review Board, and participants provided written, informed consent.

Tasks.

Participants performed multiple tasks across three scan sessions. They were asked to choose six of their top 10 most familiar people and places, and tasks involved processing these six people and places. Stimulus presentation scripts can be found at https://osf.io/5yjgh/ (42). Tasks, data acquisition, and preprocessing were identical to a previous report that used the same data but different analyses (43).

In the main visual perception task, participants viewed serially presented naturalistic images of faces, objects, and scenes. For each participant, we obtained 20 images each of six familiar faces and scenes. Familiar face images were obtained directly from friends or family members, without the participant seeing them. Control images were defined as six yoked unfamiliar faces and scenes. Familiar and unfamiliar faces were matched on age group (young adult, middle-aged, old), race, and gender. Familiar and unfamiliar scenes were matched on rough semantic category (e.g. outdoor street view; building interior). Object images were of six generic objects with varying physical properties—a banana, a baseball, a feather, a rock, a sponge, and a wrench. Face and object images contained no clear spatial structure within the image (e.g. corners), and had minimal contextual cues beyond the background. Scene images contained no people. All five image categories were matched, for each participant, on the mean and variance across images of luminance, rms contrast, and saturation (in CIE-Lch space, with a D65 illuminant): all P’s > 0.05, one-way ANOVA and Bartlett test. Images were presented at 768 × 768 resolution, 12.8 × 12.8 degrees of visual angle, for 1.85 s each with a 150 ms interstimulus interval, in 18 s blocks of images of a single identity. Participants performed a one-back task, pressing a button when an image was repeated. Task performance was high (mean hit rate 95.3%), indicating sustained attention throughout the scan. Performance did not differ substantially across categories: 98% for familiar faces, 95% unfamiliar faces, 93% objects, 96% unfamiliar scenes, and 94% familiar scenes. A postscan questionnaire verified that participants could recognize the person or place in the familiar images (mean: 100% for faces, 84% for scenes), but not the unfamiliar images (mean: 0% for faces, 2% for scenes).

In order to evaluate the broader task sensitivity of face-preferring parts of the cortex, four additional experiments were run. In the semantic task, participants rated traits of familiar people and places, and generic objects, on a 0 to 4 scale. This included personality traits of people (e.g. confident, angry, intelligent), spatial or navigational properties of places (e.g. cramped, large, has walls), and physical properties of objects (e.g. soft, heavy, rough). Participants rated by moving an icon left or right, over 18 s blocks of four questions for a given identity, for a total of 20 questions per condition and identity.

Prior to the episodic task, participants listed five common conversation topics for each familiar person, and five familiar subregions of each familiar place. In the scan, they were asked to imagine familiar people talking about common topics, navigating specific subregions of familiar places, and objects engaged in physical interactions (e.g. rolling or sliding down a hill, falling into water). Participants were specifically asked to generate a novel event, rather than remember a past event. Imagination blocks lasted 18 s, with a 3 s verbal prompt, and a 1 s hand icon at the end of the block, which participants responded to with a button press. After the scan, a free response description was collected, to ensure that participants could describe what they imagined. 9/10 participants were able to describe all events; the remaining one recalled 75% of events.

Across the visual, episodic, and semantic tasks, blocks were separated by 4 s of resting fixation, and presented in five 8 to 13 min runs per task, with palindromic block orders, counterbalanced across runs and participants. 18 s fixation blocks were included in the beginning, middle, and end of the experiment to estimate a resting baseline.

Two localizer tasks from the existing literature were also run: A localizer for areas involved in theory of mind (ToM; reading and answering questions about stories involving false beliefs or false physical representations, 29) and dynamic visual perception (watching videos of moving faces, objects, and scenes, 27). Localizer tasks were split into 4 to 5 min runs, with four runs for ToM and six for dynamic perception.

MRI Acquisition.

Participants were scanned on a Siemens 3T Prisma across three 2.5-h sessions, which included task acquisitions as well as 40 min of high-resolution anatomical images, 60 min of resting-state acquisitions, and spin echo acquisitions for distortion correction. Three each of T1- and T2-weighted anatomical images were acquired at 0.8mm resolution. Task and resting-state data were acquired using a multiband, multiecho EPI pulse sequence, optimized to boost temporal signal-to-noise ratio (tSNR) in the anterior medial temporal lobes (TR = 2 s, TE = 14.4, 33.9, 53.4, 72.9, and 92.4 ms, 2.4 × 2.4 × 2.5 mm resolution, 48 oblique axial slices with near whole brain coverage, multiband acceleration 3×, GRAPPA acceleration 2×, interleaved slice acquisition). 3 to 4 parameter-matched spin echo acquisitions were acquired per scan, between every four runs of task. Raw MRI data can be found at https://openneuro.org/datasets/ds003814 (44).

MRI Preprocessing.

Data were preprocessed and analyzed using a custom pipeline, integrating software elements from multiple software packages: FSL (6.0.3), Freesurfer (7.1.1), AFNI, Connectome Workbench 1.5, tedana 0.0.10, and Multimodal Surface Matching (MSM). The code is available at https://github.com/bmdeen/fmriPermPipe/releases/tag/v2.0.2 (45), with dataset-specific wrapper scripts at https://github.com/bmdeen/identAnalysis (46).

Anatomical images were preprocessed using an approach based on the Human Connectome Project (HCP) pipeline (47). The three images for each modality were linearly registered using FLIRT (48) and averaged; registered from T2- to T1- weighted; aligned to ACPC orientation using a rigid-body registration to MNI152NLin6Asym space; and bias-corrected using the sqrt(T1*T2) image (49). Cortical surface reconstructions and subcortical parcellations were generated using Freesurfer’s recon-all (50). Surface-based registration (MSMSulc) was used to register individual surfaces to fsLR average space (51). This registration was used to project the HCP multimodal cortical parcellation (52) onto individual surfaces.

Functional data were preprocessed using a pipeline tailored to multiecho data, aiming to optimize tSNR while maintaining high spatial resolution. Motion parameters were first estimated using MCFLIRT (53). Intensity outliers were removed using AFNI’s 3dDespike, and slice timing correction was performed using FSL’s slicetimer. Motion correction was then applied, in combination with topup-based distortion correction (54), and rigid registration to a functional template image, in a single-shot transformation with one linear interpolation, to minimize spatial blurring. Multiecho ICA was performed using tedana, with manual adjustments, to remove non-blood-oxygen-dependent (non-BOLD) noise components (55). Data were intensity normalized across participants. Data were subsequently analyzed in the volume (native functional space) and on the surface, after resampling to an individual-specific CIFTI space aligned with the anatomical template image, with 32k density fsLR surface coordinates, and 2mm volumetric subcortical coordinates. Registration between the functional and anatomical templates was computed using boundary-based registration (bbregister, 56). Surface and volumetric data were both smoothed with a 2 mm-FWHM Gaussian kernel. For resting-state data, the global mean signal was removed via linear regression, to diminish global respiratory artifacts not removed by multiecho ICA (57). Pairs of time points with excessive head movement between them (framewise displacement > 0.5 mm for task data, 0.25 mm for resting-state data) were excluded from subsequent analysis. For the sake of comparing tSNR across single- and multiecho datasets, single-echo data were defined by selecting the third echo, and preprocessed in the same way but without multiecho ICA.

Whole-Brain Analysis.

Whole-brain statistical analyses in volumetric and CIFTI space were performed in individuals using AFNI’s 3dREMLfit, modeling temporal autocorrelation with a coordinate-wise ARMA(1,1) model (58). Results were thresholded using a false discovery rate of q < 0.01 (two-tailed) to correct for multiple comparisons. We compared responses to faces versus scenes in the main visual perception task (boxcar regressors convolved with canonical double gamma hemodynamic response function).

Definition of Anatomical Regions.

To precisely localize functional responses to anatomical regions, the left and right TP, PR, ASTS, and AIT were hand-drawn on individual brains. Regions were drawn on coronal slices of MNI-aligned images at 2 mm resolution, using 0.8 mm T1 images as a reference. They were subsequently resampled to 32k-vertex surface hemispheres and then modified in the volume to remove any discontinuities observed on the surface.

Regions of TP and PR were defined based on previously established anatomical boundaries of cytoarchitectonic areas 38 and 35/36 [Fig. 1D, (25, 26)]. TP extended anteriorly to the tip of the temporal cortex, and posteriorly to its border with PR. On its dorsomedial surface, the TP extended laterally to the lateral bank of the lateral-most temporopolar sulcus. On anterior-most slices in which the temporopolar sulcus was not visible, there was no lateral border. On its ventrolateral surface, the TP extended laterally to the medial lip of the inferior temporal sulcus, or if that was not visible, the superior temporal sulcus. On the 3 to 4 anterior-most slices in which neither sulcus was visible, there was no lateral border. Anterior to the fronto-temporal junction (FTP, the first slice containing white matter connecting the frontal and temporal lobes, also termed the limen insulae), TP had no medial boundary. Posterior to the FTP, its medial boundary was the medial edge of the parahippocampal gyrus (PHG).

The border between TP and PR was placed at whichever of two landmarks was farther anterior: the anterior tip of the collateral sulcus, or one slice (2 mm) anterior to the FTP. The posterior border was two slices (4 mm) posterior to the last slices containing the head of the hippocampus, determined by the presence of the gyrus intralimbicus. The lateral border of PR varied from the midpoint between the fundus and lateral edge of the collateral sulcus, to the midpoint of the lateral edge of the collateral sulcus and medial edge of the occipitotemporal sulcus, depending on the depth of the collateral sulcus, based on previously established criteria (25). The medial border of PR in its first and last slices (2 mm) was the medial edge of the PHG. In all other slices, the medial border of PR was positioned at the midpoint of the lateral bank of the collateral sulcus, defining the boundary between PR and entorhinal cortex.

ASTS and AIT were defined based on gross anatomical features, rather than previously described cytoarchitectonic boundaries. The posterior border of both areas was arbitrarily chosen as two slices (4 mm) anterior to the posterior border of PR. This position was chosen so that the ASTS typically comprised roughly the anterior-most third of the main branch of the STS, not including parts of the STS after its posterior split and branching. The anterior border of each region was defined by the posterior borders of TP and PR. ASTS included the cortex within the STS, extending to the dorsal and ventral lips of the sulcus. AIT included cortex between the ventral lip of the STS and the lateral edge of TP or PR. Summary statistics of the size of drawn regions are given in SI Appendix, Table S8. TP and PR were similar in size, while ASTS and AIT were roughly double in size.

In addition to TP, PR, ASTS, and AIT – the experimental focus of this study—we defined a set of anatomical areas intended to capture zones previously implicated in face perception and higher-order social cognition. These areas were defined as combinations of parcels from the HCP multimodal parcellation (SI Appendix, Table S3, 52). Anatomical zones of the face network were defined by choosing regions of the multimodal parcellation corresponding to the location of previously described face responses in the FFA, OFA, and PSTS, using parcels defined by Julian et al. as a ref. 59. Anatomical zones of the social network covered regions associated with the default mode network, including MPFC, MPC, SFG, TPJ, and MSTS. The hand-drawn ASTS ROI was subtracted from the MSTS ROI to avoid potential overlap.

Region-of-Interest (ROI) Analysis.

ROI-based analyses were conducted to assess how face-preferring brain areas respond across a range of task conditions. ROIs were defined as the top 5% of face-versus-scene-preferring CIFTI coordinates (main visual task) within search spaces corresponding to the anatomical regions described above. Analyses were also performed for a range of ROI sizes from 5 to 40%. To extract responses in the main visual task, a leave-one-run-out analysis was performed, in which ROIs were defined in all but one run of data, and responses extracted in the left-out run. Statistics were performed on percent signal change values across runs, using a linear mixed effects model (MATLAB’s fitlme), with participant included as a random effect.

Responses to people were compared with places and objects, across visual, semantic, episodic, and dynamic tasks. Responses to familiar and unfamiliar faces were compared, as were responses to the belief versus false photo conditions of the ToM localizer. All comparisons were thresholded at P < 0.0125 < 0.05/4, applying Bonferroni correction across the four experimental ROIs TP, PR, ASTS, and AIT. Multiple comparison correction was not applied across left and right hemispheres, because separation by hemisphere was intended as an internal consistency check. Correction was also not applied across areas of the social and face networks, which are treated as previously described areas whose response profiles are tested only to confirm expected responses and compare to the experimental ROIs.

To evaluate differences between the response profiles of TP and PR, a condition-by-ROI mixed effects ANOVA was performed on data across all task conditions (MATLAB’s fitlme). Effects of condition, ROI, and intercept were included as random effects by participant. The condition-by-ROI interaction was not included as a random effect, because doing so led to model convergence issues.

ROI-Based Functional Connectivity Analysis.

Resting-state correlations were computed between the face-preferring ROIs defined above, including the experimental ROIs TP, PR, ASTS, and AIT, along with areas of the social and face networks. For each of the experimental ROIs, correlations to social versus face network areas were compared using a permutation test, permuting regions (10,000 iterations, two-tailed test, P < 0.0125 threshold). CIFTI-based whole-brain correlation maps, with functional ROIs as seeds, were computed for face-preferring regions of right TP and PR within each individual participant.

Social versus Visual Response Analysis.

To compare regions based on the extent to which they respond specifically to person conditions, rather than just face images, we computed a “social versus visual” modulation index. The mean response (percent signal change) to people over objects and scenes (“person contrast”) was first computed for visual, semantic, and episodic tasks. The person contrast for the visual task was then subtracted from the mean person contrast across semantic and episodic tasks and normalized by response range for each participant. This measure is positive when effects of social content are larger for semantic and episodic than for visual tasks.

Supplementary Material

Appendix 01 (PDF)

Data, Materials, and Software Availability

Raw data are available at https://openneuro.org/datasets/ds003814 (44). Stimulus materials are available at https://osf.io/5yjgh/ (42). Analysis code is available at https://github.com/bmdeen/fmriPermPipe/releases/tag/v2.0.2 (generic analysis tools) (45) and https://github.com/bmdeen/identAnalysis (dataset-specific wrapper scripts) (46).