Patches With Links: A Unified System for Processing Faces in the Macaque Temporal Lobe

Abstract

The brain processes objects through a series of regions along the ventral visual pathway, but the circuitry subserving the analysis of specific complex forms remains unknown. One complex form category, faces, selectively activates six patches of cortex in the macaque ventral pathway. To identify the connectivity of these face patches, we used electrical microstimulation combined with simultaneous functional magnetic resonance imaging (fMRI). Stimulation of each of four targeted face patches produced strong activation specifically within a subset of the other face patches. Stimulation outside the face patches produced an activation pattern that spared the face patches. These results suggest that the face patches form a strongly and specifically interconnected hierarchical network.

One sentence summary:

fMRI combined with electrical microstimulation reveals that anatomically-distinct face-selective regions in macaque inferotemporal cortex are strongly and specifically interconnected.

An essential step to understand the neural mechanism underlying any percept is to identify its anatomical substrate. For example, many theoretical models of object recognition propose a hierarchical architecture (1, 2), but it remains unclear if and how such hierarchical models are actually implemented by the brain.

The face processing system of macaque monkeys provides an ideal preparation for dissecting the large-scale functional anatomy of object recognition. Almost all macaques have a set of face-selective regions that can be easily identified by fMRI (3, 4) and readily targeted for anatomical experiments (5). Understanding their connectivity should provide important insights into the large-scale circuitry used by the brain to perceive a complex form.

It is debated whether face processing relies on a sequence of dedicated processing stages (6) or whether it relies on distributed representations (7). The former model predicts that face-selective regions show strong connections to each other but not to surrounding non-face-selective temporal cortex, while the latter predicts strong connections between face-selective regions and surrounding non-face-selective temporal cortex (8).

Tracer injections made into the macaque temporal lobe reveal a patchy connectivity pattern (9-14). For example, Saleem et al. found that injections into TEO produce labeling in TE restricted to between two to five discrete foci (10). However, the functional properties of cells at injection and termination sites were not identified in these studies. In general, to dissect functional anatomy, it is necessary to combine connectivity maps with functional topography (15, 16). Specifically, the connections of the macaque face patches cannot be deduced from previous studies. To identify the anatomical connections of the macaque face patches, we used fMRI-guided electrical microstimulation combined with simultaneous fMRI (17-19).

Macaque monkeys typically have six discrete, bilateral patches of face-selective cortex (Fig. 1, Fig. S1). These patches are organized into: one posterior patch on the lateral surface of area TEO (which we will refer to as “PL”, for posterior lateral), two middle face patches in posterior area TE, one located in the fundus of the superior temporal sulcus (STS) (“MF”, for middle fundus) and one on the lower lip of the STS (“ML”, for middle lateral), and three patches in anterior area TE, one located near the fundus of the STS (“AF”, for anterior fundus), one on the lower lip of the STS and adjacent gyrus, in area TEad (“AL”, for anterior lateral), and one more medially on the ventral surface, just lateral and anterior to the anterior middle temporal sulcus (AMTS), in area TEav (“AM”, for anterior medial) (20).

Fig. 1.

Face-selective patches in monkey M1. (A) The flattened cortical surfaces (“flatmaps”) from both hemispheres show regions significantly more activated by faces than by other objects. Computational flattening involves distorting the spatial arrangement of the data, and under-estimates the size of the sulci (shown in dark grey). Anatomical labels: sts: superior temporal sulcus, sf: Sylvan fissure, ips: intraparietal sulcus, ls: lunate sulcus, ios: inferior occipital sulcus, ots: occipitotemporal sulcus. (B) The same contrast overlaid on high resolution coronal slices from monkey M1. The anterior-posterior position of each slice in mm relative to the interaural line is given in the top right corner; the left hemisphere is shown on the left. The face patches are labeled as in (A). (C) Mean time courses extracted from the six face patches of the right hemisphere. Three different visual stimulation conditions were presented: faces (green epochs, F: human faces, M: monkey faces), objects (orange epochs, H: hands, G: gadgets, V: vegetables and fruits, B: monkey bodies), and scrambled versions of the same images (white epochs).

We identified the locations of face patches in four monkeys (M1 – M4) by scanning them with a standard face localizer stimulus (3). Individual animals and hemispheres exhibited slight variations on the prototypical pattern just described. Figures 1A, B show face patches in the left and right hemispheres of monkey M1 on flattened maps of the posterior 2/3 of the brain excluding prefrontal cortex and in coronal slices; this animal had five discrete face regions in the right hemisphere (PL and ML were confluent). Time courses from the face patches confirm the face selectivity of each patch (Fig. 1C). Figure S1 shows the face patches of the three other animals (M2 – M4) used in this study.

We then targeted a subset of the face patches for microstimulation combined with simultaneous fMRI. We first verified that the electrode correctly targeted each face patch by recording spiking activity. We then transferred the animal to the scanner for microstimulation. We stimulated a total of four different face patches (ML, AL, AM, and AF), several inferotemporal (IT) sites neighboring the face patches, and a site in the upper bank of the STS (Table S1).

We first targeted ML in monkey M1. Figure 2A shows MR images of the electrode descending into ML, in sagittal and coronal planes. Figure 2B shows the response profile of the last cell recorded from this patch prior to microstimulation; this cell was highly face selective (as were neighboring ones above it). The location of the electrode tip, marked on the flat map, confirms that stimulation was within ML (Fig. 2C). Comparing activation with and without microstimulation revealed five discrete regions in the temporal lobe (Fig. 2C). Stimulation resulted in a large spread around the electrode tip, a stretch of 4 mm with little activity, and then three discrete anterior patches located 6 - 11 mm anterior to the stimulation site. These patches coincided with the three anterior face patches of this monkey (compare Fig. 1A with Fig. 2C, and Fig. 1B with Fig. 2D). This activation pattern was reproducible across scan sessions, and was not sensitive to the choice of significance threshold (Fig. S2). Time courses from the six face patches confirm strong activation during microstimulation epochs (Fig. 2E).

Fig. 2.

Brain regions activated by microstimulation in the lateral middle face patch (ML) of monkey M1. (A) The position of the electrode in relation to the face patches (indicated by green outlines) in sagittal and coronal MRI slices; the tip of the electrode was located 3.5 mm anterior to the inter-aural line. (B) Selectivity profile of the last neuron recorded before stimulation. The bars show the mean response of this unit to images from eight different image categories (faces, fruits, gadgets, hands, bodies, monkey body parts, monkey bodies, and scrambles), error bars 95% confidence intervals. (C) Areas significantly activated by microstimulation versus no microstimulation overlaid on the flatmap. The face patches (cf. Fig. 1) are indicated by the green outlines. The stimulation site inside ML is marked by an “x”. (D) The same functional contrast overlaid on coronal slices. The face patches are indicated by green outlines. The “+” indicates the approximate stimulation site (the slice containing the actual stimulation site, at +3.5, is not included in this mosaic). (E) Mean time courses from the six face patches in the right hemisphere. Microstimulation blocks (gray epochs) were interleaved with fixation only blocks (white epochs).

Figure S3 shows the results of stimulating ML in two additional animals (M2 and M3). In monkey M2, stimulation in ML elicited activation in three other face patches: MF, PL, and AL. In monkey M3, stimulation in ML elicited activation in two other face patches: PL and AL. Results from the three animals show that ML is strongly connected to PL and AL, and more weakly/variably to the remaining face patches.

If fMRI activity is driven principally by the energy demands of transmitter release during synaptic activity (21, 22) then these electrical stimulation-induced activations likely reflect orthodromic (rather than antidromic) spike propagation. Activations could result from direct or indirect connections (23).

Microstimulation-induced activations were usually much stronger in the hemisphere ipsilateral to the stimulation site, but we sometimes observed activation in contralateral face patches as well. Figure S4 shows contralateral activations induced by stimulation in the right hemisphere ML of monkeys M1 and M2. These contralateral activations were confined to the face patches PL, ML, and MF.

ML was strongly connected to AL in all three animals. To follow the circuit, we next targeted AL (Fig. 3A). Figure 3B shows that the last cell recorded before moving the animal to the scanner was face selective. In addition to spread around the stimulation site, stimulation in AL produced strong activation in ML, MF, and AF, and weak activation in AM (Fig. 3C-E). In contrast to stimulation in ML, stimulation in AL produced no activation in PL. In a second animal (M4), stimulation of AL elicited activation in ML, MF and AM, but not AF or PL (Fig. S5). Thus AL appears to be robustly connected to ML, MF, and AM, and more variably to AF.

Fig. 3.

Brain regions activated by microstimulation in the anterior lateral face patch (AL) of monkey M1. Same conventions as Fig. 2. (A) Electrode position in AL. (B) Example of neuronal selectivity. (C) The contrast microstimulation versus no microstimulation revealed microstimulation-induced activity in four discrete patches in the temporal lobe coinciding with AL (the stimulation site), AF, ML, and MF, as well as a fifth patch of faint activation coinciding with AM. PL was the only face patch not activated. (D) The same contrast overlaid on coronal slices. (E) Time courses from the six face patches of the right hemisphere.

We next targeted AM, the most anterior of the six face patches. In monkey M1, stimulation in AM induced activity in ML, AL, and AF (Fig. S6A-E). In monkey M2, stimulation in AM induced activity in ML and AL (Fig. S6F-J; this animal lacked an AF in the stimulated hemisphere, see Fig. S1A). Thus AM appears to be robustly connected to AL and ML.

Finally, we stimulated the third of the anterior face patches, AF, in one animal (M1, Fig. S7). This experiment showed that AF is strongly connected to MF, and more weakly to ML.

In the experiments described so far, the animal fixated a blank gray screen during microstimulation. To test for interactions resulting from combining microstimulation with visual stimulation (24, 25), we combined electrical and visual stimulation in a stimulus sequence composed of six conditions: faces, faces + microstimulation, objects, objects + microstimulation, blank, and blank + microstimulation. We ran two monkeys (M1 and M2) on this stimulus sequence, microstimulating in ML. This experiment revealed that: 1) the strength of activation elicited in the face patches by microstimulation in ML was comparable to that elicited by viewing faces, and 2) there was only a weak interaction between responses to microstimulation and those to visual stimulation (Text S1).

If the circuitry of IT cortex follows a single scheme, then stimulating just outside a face patch should yield an activation pattern similar to that obtained by stimulating inside a face patch. If, on the other hand, the face patches constitute a unique system within IT cortex, then stimulating outside the face patches may yield a qualitatively different activation pattern. In particular, a distributed mechanism for coding non-face objects might predict that stimulation outside a face patch should lead to widespread activity throughout IT cortex.

We therefore stimulated three different sites just outside the face patches. First, we stimulated a site just posterior to ML, on the lower lip of the STS, in monkey M1 (Fig. 4A). The last cell recorded from this site responded only to scrambled patterns (Fig. 4B). Stimulation at this site produced a large spread of activity within the STS around the stimulation site (which spared both PL and ML), as well as a discrete patch of activity in the lower bank of the STS, anterior to ML and 7 mm anterior to the stimulation site (Fig. 4C-E). This result suggests that IT cortex outside the face patches also exhibits discrete patchy connectivity, consistent with previous anatomical tracer studies (10, 13, 26). Figure S8 shows the result of microstimulating outside ML in monkey M2. In addition to a large spread around the stimulation site (which included PL but largely spared ML), there was activity in a discrete region just posterior to AL, on the outer surface of the inferotemporal gyrus. Finally, Figure S9 shows the result of stimulating just posterior to AM in monkey M2. This produced spread around the stimulation site as well as activity in a discrete posterior projection site within the ventral bank of the STS and a discrete anterior projection site within the fundus of the STS.

Fig. 4.

Brain regions activated by microstimulation outside the lateral middle face patch (ML) in monkey M1. Same conventions as Fig. 2. (A) Electrode position just posterior to ML. (B) Example of neuronal selectivity. (C) The contrast microstimulation versus no microstimulation revealed microstimulation-induced activity around the stimulation site as well as in a distinct patch anterior to ML. (D) The same contrast overlaid on coronal slices. Note how the activation spared most of PL and the other face patches. (E) Time courses from the patch just anterior to ML and from ML.

Face cells have been found in both the upper and lower banks of the STS (27). To compare connections of the upper versus lower banks of the STS, we stimulated a site in the upper bank of the STS in monkey M1 at approximately the same AP position as ML (Fig. S10A-E). Stimulation at this site produced a different activation pattern from stimulation in the lower bank: instead of discrete clusters of activation, we observed a large swath of activity extending from −5 to +20 within the upper bank and fundus of the STS, as well as activation in cingulate and somatomotor cortex (Fig. S10F-J, second monkey). This experiment shows that the technique of fMRI combined with microstimulation is capable of detecting highly distributed connectivity patterns. Furthermore, the result suggests that microstimulation is capable of activating cells more than one synapse away (it seems unlikely that all of the areas activated in Fig. S10 were directly connected to the stimulation site). If true, this strengthens our main finding concerning the specificity of connections between face patches, since it implies that cells two synapses or more away from the stimulation site were also located within the face patch system.

We did not observe any other cortical regions consistently activated besides those described (28). However, we did consistently observe stimulation-induced activation in three subcortical regions: the amygdala, claustrum, and pulvinar. Stimulation in AM elicited activation in the lateral nucleus of the amygdala and laterally adjacent claustrum in both monkeys tested (Fig. S11A, B). Stimulation outside but close to AM also activated the lateral amygdala and claustrum (Fig. S9C, slice at +20.5), suggesting that these two structures receive inputs from a larger region within ventral IT (29).

We observed activation of the inferior pulvinar in response to microstimulation of both ML (Fig. S11C) and AL (Fig. S11D), as well as IT foci outside the face patches (data not shown). Thus, the inferior pulvinar appears to be strongly connected to a large portion of IT cortex (9). Because the pulvinar, claustrum, and amygdala were the only non-face-selective brain structures consistently activated by stimulation of the face patches, we hypothesize that these structures may constitute three bottlenecks for communication between the face patches and other regions of the brain (30).

In this study, we imaged brain regions activated by microstimulation of four different macaque face patches (ML, AL, AM, and AF). The results suggest that the six macaque face patches form a tightly and specifically interconnected system, as summarized in Fig. S12 and Table S2. The existence of an interconnected circuit consisting of six nodes, likely dedicated to coding the same Gestalt form, shows that functional specialization in IT exists not only at the level of isolated columns (31) or patches (3), but extends to the level of connected, distributed networks.

Our results extend previous findings of functional specialization in early visual cortex, where modules processing the same visual attribute have been shown to be specifically connected (32), e.g., color-specific blobs in area V1 with color-specific thin stripes in area V2 (33). The face patch network likely represents a stable structural network, since the connections were apparent across three very different functional states: without visual stimulation, with visual stimulation by faces, and with visual stimulation by non-face objects (Text S1). However, functional connectivity within the face patch system and between the face patches and other brain areas may depend on behavioral and perceptual state (24, 34).

Of all the face patches, we know most about the functional properties of the two middle ones, ML and MF. Single-unit recordings show that almost all visually responsive cells in this region are face selective (5), implying that the step of face detection has been accomplished. Because microstimulation of ML activated three anterior face patches (Fig. 2C), this suggests the existence of a sequence of dedicated face processing stages following face detection. Such an architecture would be consistent with computational models of object recognition in which a detection stage precedes individual recognition (35). However, our results suggest an important role for feedback in face processing. Stimulation of AL led to feedback activation of ML (Fig. 3) with a strength and spatial precision comparable to the feedforward activation of AL by ML stimulation (Fig. 2).

Is the circuitry of the face patches unique, or a theme that is recapitulated in surrounding non-face-selective cortex? The results of microstimulating outside the face patches (Figs. 4, S8, and S9) suggest that there do exist largely self-contained networks of modules, outside the face patches, for processing shape. Since non-face objects elicit highly distributed fMRI response patterns in IT cortex (7), these networks of non-face modules likely represent aspects of form present in almost all objects, such that any object would activate multiple networks. But given their size, there may exist only a finite set of macroscopic shape processing networks.