Social Attention and the Brain

Abstract

Humans and other animals pay attention to other members of their groups to acquire valuable social information about them, including information about their identity, dominance, fertility, emotions, and likely intent. In primates, attention to other group members and the objects of their attention is mediated by neural circuits that transduce sensory information about others and translate that information into value signals that bias orienting. This process likely proceeds via two distinct but integrated pathways: an ancestral, subcortical route that mediates crude but fast orienting to animate objects and faces; and a more derived route involving cortical orienting circuits that mediate nuanced and context-dependent social attention.

When Barack Obama walks into a room, all eyes are on him. So too for Angelina Jolie and Brad Pitt, but no less so for the chair of a biology department at faculty meeting or the captain of the basketball team at a pep rally. Looking at others is an important and natural feature of our everyday life — so much so that when others fail to meet our gaze we immediately sense that something is wrong. Clearly, at whom we look — and how — betrays much about our own identity: who we are, where we come from, our gender, age, and social status. Although these aspects of social attention seem, at first glance, fundamentally human, they are also biological features with deep evolutionary roots. Indeed, a hallmark of primate evolution is an increasing reliance on vision to guide behavior. Such visually-guided behaviors include the discovery and selection of high-quality foods, recognition and pursuit of receptive mates, identification and solicitation of potential allies, the detection of predators, and avoidance of social threats [1].

As a direct result, how biologically alive an object appears (its ‘animacy’) strongly predicts how much it attracts visual inspection [2,3]. When confronted with a visual scene in the laboratory while lacking a predefined task or instruction, we generally look toward objects with social importance, namely individuals, their faces, and especially their eyes (humans [2]; monkeys [4]). This orienting bias arises almost immediately after the stimuli appear, within a lag as short as 120 ms, and is evident in the first glance [5]. Two cues seem to drive fast identification of animate objects: faces, particularly the paired concentric circles comprising the eyes [6]; and irregular motion [7]. Eyes may be especially important not just as a potent indicator of animacy, but also as an indicator of affective, attentional and intentional state [6].

While no aspect of primate social orienting behavior is fully understood at a mechanistic level, both ancestral subcortical and more derived neocortical circuits likely play important roles (Figure 1). Specifically, subcortical circuits — believed to run from the superior colliculus through the pulvinar to the amygdala, though direct anatomical confirmation is lacking — appear to serve as an ‘early warning’ system providing a crude but fast description of animate objects [8] and the foci of their attention [9]. Complementing this system, neocortical circuits, perhaps developing under the guidance of subcortical systems [10], further facilitate social attention. These include processing of social identity and expression in the fusiform gyrus and superior temporal sulcus [11], and observed gaze in the superior temporal sulcus [12–14] and posterior parietal lobe [12,15].

Figure 1. Connectivity of social (red), reward (blue), and orienting (green) cortical areas governing social attention.

In addition to the cortical pathway, a fast subcortical pathway connects superior colliculus to amygdala via the thalamus (not shown here). Note that multiple social processing areas lie along superior temporal sulcus, occupying both posterior and anterior temporal lobe, and that functional activity in imaging tasks has not been conclusively integrated with past anatomical or electrophysiological studies. PPC, posterior parietal cortex, including macaque areas 7A and LIP; STS, superior temporal sulcus regions; SEF, supplementary eye fields; FEF, frontal eye fields; OFC, orbitofrontal cortex; AMYG, amygdala.

Social Information Reinforces Attention

For vision to guide behavior effectively, an animal must decide where to look, how long to look there, and whether to spend time in continued observation or instead devote visual processing to online guidance of other behaviors (for example [3]). The rich literature on neural mechanisms of visual orienting decisions sheds light on how the brain accomplishes this task.

Neurons in several brain areas, including the lateral intra-parietal area (LIP), prefrontal cortex, superior colliculus, basal ganglia and posterior cingulate cortex, signal the predicted value of orienting toward a particular object for fluid rewards (reviewed in [16]). Modulation of neural responses to stimuli by their value likely serves to bias the visual orienting system toward shifting gaze to the most important among objects in the visual field [17]. While these types of study have yielded useful insight into the neural mechanisms underlying decision-making more generally, they are limited in that the expected rewards in the laboratory are typically food or juice, while in the natural world, the ‘reward’ for orienting is useful visual information. Indeed, one of the most fundamental, and often overlooked, aspects of social behavior is the intrinsic reinforcement that motivates attention to others.

To explore the contribution of social information value to visual orienting, we took advantage of the natural tendencies of monkeys and other primates, including humans, to look toward other individuals and the objects of their attention [6,18]. We specifically tested the idea that value-based scaling of neural target signals extends spontaneously, in the absence of training, to socially-informative stimuli [19]. The orienting behavior of male rhesus macaques and the responses of neurons in LIP were simultaneously studied using a ‘pay-per-view’ orienting choice task. Monkeys first fixated on a central square on a dark computer monitor, then two yellow squares were illuminated, one in the response field of an LIP neuron being recorded and the other directly opposite. Choosing the target in the response field was rewarded with a drop of juice and a picture of a familiar monkey, while choosing the other target was rewarded only with juice. The relative amount of juice reward associated with the two targets and identity of the monkeys in the pictures were varied across conditions.

We estimated the reinforcement value of social images in terms of how much they augmented or decreased the desirability of a juice reward (compare [20]). Generally, male monkeys valued orienting to images of high-ranking males and female sexual signals (these images substitute for fluid rewards) but did not value orienting to images of subordinate males (monkeys must be paid extra juice to view these image). Importantly, LIP neurons responded most strongly when monkeys chose to view images of dominant male faces or female sexual signals, but responded weakly when the expected outcome was the face of a subordinate male (Figure 2). This scaling of attention-related neuronal activity by social reward value precisely paralleled value-based scaling of neuronal activity by fluid rewards.

Figure 2. The LIP population response simultaneously encodes social image value and fluid value during a ‘pay-per-view’ choice task.

(A) Average firing rate for 34 neurons plotted against time for all trials in which the subject chose to view the image (T2) in the neuron’s response field, separated by image class. Inset: values determined for different image classes for two male monkey subjects (open and closed bars), in ms of fluid delivery time. Positive deflections indicate the subject was willing to forgo fluid to view that image class. Negative deflections indicate the subject required fluid overpayment to choose that image class. Hindquarters refers to the perineal sexual signals of familiar females. Dominant and subordinate refer to the faces of familiar dominant and subordinate males. Gray refers to a plain gray square matched for size and luminance to the other image classes. (B) Average firing rate of the population for all trials in which the subject chose to view the image (T2) in the neuron’s response field, separated by fluid value relative to the non-chosen target (T1). (C) Firing rates plotted as a function of image value in four 200 ms epochs. Black symbols represent regressions performed on all data in which the subject chose to view the image, and gray symbols represent the same analysis restricted to trials in which the fluid payoff for choosing T1 was equal toT2. (D) Firing rates plotted as a function of the difference in fluid payoff between T2 and T1. Black symbols represent regression performed on all data in which the subject chose to view the image, and gray symbols represent the same analysis restricted to trials in which the image value calculated for that block was greater than -5 and less than 5 ms. Error bars represent SEM. The data in (C,D) were binned for display, but all regressions were performed on raw data. *p < 0.05, **p < 10 −3. (Adapted with permission from [19].)

There are at least two important theoretical implications of this finding. First, it indicates that the sensitivity of neurons in the visual orienting system to conditioned juice rewards generalizes to more naturalistic, socially-relevant outcomes. Second, juice value and social value are encoded simultaneously and in the same manner by LIP neurons, suggesting that sundry information about the value of attending to different objects and events in the environment is collapsed into a common currency before it reaches LIP. Furthermore, it seems clear that this common currency represents importance rather than valence (attractiveness). In our pay-per-view task, monkeys preferred to orient to both female sexual signals and the faces of dominant males. After choosing to orient to dominant male faces, however, they rapidly looked away — likely because sustained eye contact between macaques is a threat and provocation [21]. This finding suggests different motivations for looking at sexual signals and high-status faces. While sexual signals initiate approach behavior for obvious reasons, orienting to the faces of dominant males is partially motivated by threat assessment that may instigate retreat. LIP neurons nonetheless responded strongly to both types of images, suggesting a role in orienting to socially-relevant stimuli rather than signaling the pleasure derived from doing so. Thus, the common currency of orienting value, as observed in LIP, integrates factors spanning both outcome modality (in this case social and nutritive) and outcome valence (positive or negative).

We speculate that the common currency of target value observed in LIP will be observed in other areas associated with visual orienting. This supposition is based on two arguments. First, the similarities in fluid reward coding discovered in the brain areas mentioned above implies a redundant representation of orienting value across the visual orienting system. The second argument is an appeal to parsimony: primates can only overtly orient toward one location at a time, and attention control mechanisms must unambiguously select a single target location to prevent conflict between alternative motor plans. Thus, a representation of orienting value that generalizes across affective valence and outcome modality would most efficiently guide attention and orienting within natural environments, in which energetic, social, sexual, and other factors must be balanced to adaptively guide behavior.

Computing Social Orienting Value

Despite a dearth of research directly addressing how social information is incorporated in a common currency to guide attention, several studies using a variety of non-social outcomes suggest the importance of the orbitofrontal cortex (OFC) and striatum in the process. These reports have suggested that the OFC transforms reward and punishment information into a common currency of subjective value, in which arbitrary options can be compared. Indeed, electrophysiological studies have demonstrated that neurons in OFC signal several types of information essential to computinga common currency for decision, including information related to subjective preference among different fluid rewards, flavor-specific satiety, and aversive outcomes (reviewed in [22]).

These observations suggest that OFC neurons link predicted rewards to state variables, including both internal (motivation and satiety) and external (alternatives, opportunity costs) factors that impact the subjective value of an action. In contrast to the value-related modulations of neuronal activity in LIP and the superior colliculus (reviewed in [16]), primate OFC neurons encode the abstract value of available options independently of the visuospatial and motor contingencies of the task [22] (or at least do so while choices are made [23]).

The OFC is well-situated anatomically to pass abstract value information to executive systems which, in turn, translate this information into the spatially specific representation of target value observed in LIP, the superior colliculus and other areas, doing so both directly and via projections to the ventral striatum(VS), which in turn projects via the thalamus to other areas of cortex [24]. Several lines of evidence demonstrate that this circuitry encodes behaviorally-relevant social information, in addition to other types of rewards. For example, neuroimaging studies have demonstrated the OFC and VS respond to socially reinforcing stimuli such as beautiful or smiling faces [25], while OFC lesions disrupt interpersonal behavior [26]. Similarly, neuroimaging studies have demonstrated that both the ventral and dorsal striatum respond to more complex social information, such as cooperation with a human partner or the opportunity to punish a traitor (see [27] for a review of social economic games and their neural correlates).

Additionally, the amygdala likely plays an important role in calculating and updating social orienting value. Recent human imaging work has demonstrated a complex, regionally- and temporally-specific amygdala response to social conditioning. The medial ventral amygdala has an initially strong, but quickly habituating, response to faces which predicts social outcome regardless of valence (positive, negative or neutral); the lateral ventral amygdala responds strongly to negatively valenced outcomes without apparent habituation; and the dorsal amygdala responds to both positive and negative outcomes with response magnitude decreasing over time [28]. The amygdala likely interacts with the striatumand OFC in creating and monitoring social value, as it shares dense connections with both structures [29,30].

Joint Attention and Gaze Following

Attending to another animal can potentially reveal several types of useful information, such as its species, gender, age, health, and affective state. One of these signals, gaze direction, is remarkable in that it redirects attention away from the observed individual and toward the locus of their attention. Even for minimally social animals, observed gaze direction may usefully predict the movement trajectories of both predators and prey. Moreover, because group-living animals share an overlapping goal set — eating similar foods, avoiding similar predators — the behavioral states of other individuals can provide useful information. Reading the intentions of other individuals can help localize food sources and coordinate group movements, while reading their attention can speed threat-detection and anti-predator behavior. Finally, group-living animals in complex societies may learn about the intentions and dispositions of others by relating observed gaze and emotional expressions.

For all of these reasons, group-living animals should both attend and often ‘mirror’ the attentional state of others, and their attention systems should be likewise influenced by social cues that transmit information about the attention and intentions of others [6]. Among humans, for example, our tendency to reflexively follow gaze may play an especially important role in language development [31].

Our sensitivity to others’ gaze is two-fold: more urgently, we sense when we are being watched; more subtly, we sense the referent of observed gaze within our shared environment, discriminating between distal regions which are, or are not, the focus of another’s attention. There is overwhelming evidence that the first manner of sensitivity to gaze direction — sensitivity to being watched — is both innate [32] and shared by most vertebrates [33]. The second manner of sensitivity to gaze direction, however — the use of gaze as a referential cue—remains somewhat mysterious. It is clear that both humans and many animals [6] follow the gaze of others with their own. Furthermore, the shared psychophysical features of gaze following in humans and macaques indicate that the behaviors share similar neural mechanisms [18].

While gaze following appears fairly reflexive in humans and other primates (for example [18]), its sensitivity to social context suggests that the underlying mechanisms are not strictly modular but rather deeply enmeshed with other aspects of social information processing (reviewed in [34]). For example, just as they will sacrifice more juice to look at dominant males [19,20], rhesus monkeys are also more likely to follow their gaze [35]. There also appears to be important individual variation in socially-cued orienting. For example, men typically follow gaze less than women do, suggesting that sex hormones may influence development of this system [36]. Together with decreased social orienting by high-status monkeys relative to low-status monkeys, these findings may suggest a suppressive role for testosterone [35]. Notably, a failure to seek or respond to joint attention is a crucial diagnostic feature of autism, which is much more prevalent in males than females [37].

Several brain areas have been identified as especially important for attention to the gaze of others. First, neurons near the superior temporal sulcus in monkeys and humans are selective for dynamic features of facial expression, including gaze direction [14], and the most anterior of these neuronal populations appears to be sensitive to the explicit direction in which individuals look [13]. Second, neurons in the amygdala are sensitive to gaze direction [9] and may mediate attention to the eye region [38]. Finally, the posterior parietal cortex may contribute to the redirection of attention by observed gaze: Neurons in posterior parietal cortex are sensitive to the direction of observed gaze [12], and, in monkey LIP, respond both when the monkey looks toward a region of space and when the monkey observes another monkey doing the same [15] (Figure 3).

Figure 3. LIP neurons mirror observed gaze.

(A) Gaze following in macaques. An image of a monkey face with averted gaze (cue) was displayed centrally for 100, 200, 400 or 800 ms. Next, the cue was extinguished simultaneously with the appearance of a peripheral target located in the direction the of gaze of the cue (congruent condition) or directly opposite (incongruent condition). Monkeys then immediately shifted gaze to the peripheral target to receive a fluid reward. For cue durations≤400 ms the congruent condition elicited reaction time savings compared to the incongruent condition, indicating a shift of attention. Error bars represent SEM across sessions. (B) Neural cue responses in LIP. Significant neuronal responses to observed gaze direction in 10 ms bins. Neurons with firing rates enhanced by cues with gaze directed at their response fields (red) are temporally clustered in the time windows for which gaze-following behavior is strongest. Those neurons with firing rates suppressed by cues with gaze directed toward their response fields maintain tonic decreases in activity throughout the fixation period. (Adapted with permission from [15].)

Deception and Covert Attention

Because many social animals follow the gaze of others, overtly attending to desired objects may impose substantial costs in competitive interactions, for example by revealing rare resources or predicting imminent behaviors. Though humans may have evolved especially readable eyes [39], primate gaze comprehension is generally more pronounced in competitive than cooperative contexts [40]. The costs and benefits of social signaling with the eyes may have led to an evolutionary arms race favoring deceptive orienting behavior. In particular, the ability to covertly attend objects and events in the environment, while overtly fixating elsewhere, could help to obscure sensitive information.

Humans clearly can, with some difficulty, divide their attention amongst multiple locations in a scene [41]. Macaques are often trained in the laboratory to attend to peripheral targets without moving their eyes; but to our knowledge, covert attention has never been systematically studied in other animals. The ability to divide attention may have evolved, in part, to solve the social problem of hiding intent. Neurophysiological studies demonstrate that covert and overt attention depend on overlapping brain systems, thus reinforcing the notion that covert attention evolved in some species from more ubiquitous mechanisms underlying overt orienting [42]. Further comparative studies of covert and overt attention, specifically contrasting species that differ in social competition, will be needed to test whether the ability to read gaze promotes the evolution of covert attention.

Conclusion

Social interaction is a foundation to our way of life and a yardstick by which we measure its quality, but the fundamental mechanisms that guide our social relationships have deep evolutionary roots. We extract key information from the social environment by orienting toward important individuals and subsequently to the objects of their attention. This orienting bias arises both through reflexive subcortical pathways shared by most vertebrates, and through more subtle, nuanced, and context-dependent pathways in primates and presumably other mammals and birds. These orienting biases play an important role in human cognition and behavior, including language acquisition, and are vulnerable to disruption in disorders such as autism and social anxiety.

Understanding individual and species differences in the neural mechanisms that mediate social attention, the genetic origins of these differences, and their implications for differences in social behavior and social structure will require further comparative study using naturalistic, ecologically-valid social contexts (see [43] for hurdles in investigating social interaction in the laboratory). Doing so will further demand deeper understanding of orienting behavior in the real world, outside the confines of the laboratory [43]. The neuroethological approach advocated here, involving active collaboration between psychologists, geneticists, ecologists and neurobiologists, provides a strong rationale for optimism that the complex and dynamic interactions that characterize social behavior — and which characterized our species’ evolution—will become more tractable targets for experimental and mechanistic inquiry.