Head to toe, in the head

Faces, Bodies, and Communication

Sometime about 250,000 y ago, primates started talking to each other (1). Before that time facial expressions and body language were the main modes of communication among primates. Even today in the presence of our sophisticated language system, face and body gestures play a major role in human communication. If someone tells you that she is not bored with a conversation but her half-open eyelids, raised eyebrows, dropped shoulders, and the way she puts her hand under her chin “tell” you the opposite, you would probably trust the ancient signal more than the modern sounds that we call words. In a recent PNAS article, Fisher and Freiwald (2) might have unveiled where in the brain such signals are encoded.

Understanding the basis of face–body representation in the primate brain is crucial for developing a mechanistic theory of human behavior because faces and bodies are the main objects of interest in a wide range of primate behaviors, including identification, nonverbal communication, coaction, and social cognition. Past studies have revealed a hierarchy of neural clusters in the inferior temporal (IT) cortex of humans and macaque monkeys that selectively responds to the images of faces (3–6). This hierarchy is thought to causally support face-recognition behavior (7–9). A separate constellation of neural clusters was also found to represent images of bodies and body parts (10). Now, using functional MRI in macaque monkeys, Fisher and Freiwald (2) successfully demonstrate a systematic development of whole-agent (a face and its body together) selectivity through the processing hierarchy of the IT cortex. Specifically, the authors show that although the responses of the posterior parts of the face-selective network (lower in the hierarchy) are mainly driven by isolated images of faces (and not bodies), some anterior parts of this network (higher in the hierarchy) prefer images of a whole-agent: a face mounted on a body. This is the first time that a synergic interaction in the neural responses to faces and bodies is documented. This finding suggests increasing intercommunication of face and body signals through the visual information-processing hierarchy in the brain.

Doesn’t Have to Be Like This, but It Is

One notable point about Fisher and Freiwald’s (2) finding is that the brain doesn’t have to be wired like this to support whole-agent recognition at the behavioral level. In other words, the fact that we can behaviorally combine the visual information of bodies and faces and recognize people as whole-agents does not necessitate existence of neural clusters (at the spatial scale of millimeters, detectable by functional MRI) that respond to whole-agents more than their parts. A neural system can separate and recognize different classes of objects by giving proper weights to their elements (11, 12) without necessarily having to represent them as whole stimuli, either at the single neuron level or at such large spatial scales. For example, we can use different features of cars (shape of the headlights, curvature of the hood, and so forth) to identify different types of cars, but this behavioral ability does not imply that the neural networks in our heads have neurons that respond to images of whole cars. This logic makes Fisher and Freiwald’s (2) results not necessarily expected, and thus informative and important. This also drives the need for learning more about this system to understand what computational (e.g., wiring length constraints) and evolutionary forces (need for faster reaction times, and so forth) have shaped the processing schema in this particular way.

Detection vs. Discrimination Trade-Off

One possible misunderstanding about the role of categorical neural representations in behavior (high level or low level) is the confusion between “detection” and “discrimination” behavioral tasks. To remain careful with interpretation of the current findings about whole-agent representation in the IT cortex, this confusion should be carefully addressed.

The case of “face neurons” provides a great example. Face neurons are traditionally defined as the neurons whose responses discriminate images of faces from images of nonface stimuli (13). Based on this definition, an ideal face neuron is a neuron that responds invariantly to all instances of face images and does not respond to images of nonface objects. This hypothetical face neuron could directly mediate face detection behavior (detecting faces with various sizes, identities, and so forth among other objects). However, such a neuron does not necessarily support other face-related tasks, such as face discrimination (distinguishing different faces from each other). In fact, tuning a neuron as a good detector for any given object class (e.g., faces) will take a toll on the neuron’s ability to discriminate members of that object class (e.g., different faces) from each other. This is because in the presence of natural noise, a good detector should have minimal response variation to different instances of the category it detects (14). On the other hand, a neuron tuned to discriminate members of a given category from each other has to maximize its response variation to them (e.g., different faces); this will consequently take a toll on the neuron’s detection performance for the encoded category because it will respond low to some of the members of the object category. Fig. 1 provides a schematic summary of this theoretical trade-off. Surprisingly for the case of faces, inactivation of chunks of the brain that contain “face-detector neurons” reduces the face-discrimination behavioral performance (9, 15), but this can easily result from the fine-grained spatial mix up of the detector and discriminator neurons (15).

Fig. 1.

Detection vs. discrimination trade-off. A perfect “category detector” neuron (a neuron that distinguishes an object category from all other objects; e.g., a “face neuron” or a “whole-agent” neuron) cannot encode within-category differences because it has to minimize its within-category response variation to provide the strongest (most noise-tolerant) signal to contrast the category of interest against other categories. Alternatively, a “within-category discriminator” neuron (a neuron that distinguishes different members of a category; e.g., different faces or different whole-agents) should maximize its response variation across the members of category of interest; thus, it is not a good category detector. Each subplot schematically depicts the neuron’s response (ordinate) for various visual stimuli. The abscissae indicate the visual stimuli ranked based on their category and the neural responses they can drive. The pink bar spans the stimuli that belong to the category of interest (e.g., faces or whole-agents) and the green bar spans visual stimuli that belong to other categories. The Left subplot shows a hypothetical ideal category detector (face detector in this case) that responds invariantly to all faces but not to any other object. This neuron cannot signal the differences between faces (for face-discrimination tasks). The Center subplot shows the same for a neuron that conveys some category detection signal and some within-category discrimination signal. The Right subplot represents a neuron that uses its maximum dynamic range for within-category discrimination, thus is a great discriminator but a poor category detector. The dashed blue lines indicate alternative scenarios for the neurons' responses to “other” categories. Note that a neuron can respond to other categories (thus convey no detection signal) but remain a good within-category discriminator.

The same logic applies to any other categorical neural representation that is defined by a “detection contrast”: contrasting the average neural responses to a given object class to that of other object classes. This includes whole-agent neural responses in the IT cortex as well. The fact that an IT neural cluster (cluster AF) responds more to the images of whole-agents compared with other image categories (a detection contrast), directly suggests its involvement in whole-agent detection behaviors, such as noticing a person in the room, noticing a predator

Fisher and Freiwald reveal for the first time an area of the macaque brain that responds to the images of whole-agents more than its sum response to isolated images of faces and bodies.

in the dark, and so forth. Involvement of this neural cluster in discrimination tasks, such as reading the face–body language, discriminating people from each other, and other instances of social cognition is still possible (because no neuron in a real brain is an ideal detector and because detector and discriminator neurons may be spatially mixed); however, it is currently less supported by direct evidence. This aspect is particularly important because using a detection contrast here (whole-agents vs. other stimuli) might have washed up potential neural responses in other cortical areas that may respond differentially to different whole-agents. I expect that Fisher and Freiwald’s (2) important finding opens the door to further explorations such as using multivoxel pattern analysis, single-unit neural recordings, and causal studies (e.g., measuring the direct effect of inactivation of this area on whole-agent discrimination behavior) to determine how whole-agents are encoded in the primate brain.

Forward

Fisher and Freiwald (2) reveal for the first time an area of the macaque brain that responds to the images of whole-agents more than its sum response to isolated images of faces and bodies. Given the importance of whole-agent perception in human behavior and the potential link between this brain area and a wide range of whole-agent related behaviors (such as social cognition), this is a remarkable finding. It is now time to systematically answer a series of questions that naturally rise from the study: how do face- and body-selective areas of the brain interact at the neural circuit level? How does this area encode various face–body configurations? How does this area represent the differences between different people (whole-agents)? Does this area causally support a set of whole-agent related (detection and discrimination) behaviors? Hopefully future studies will shed more light on this interesting and exciting area of science.