Social cognition and the anterior temporal lobes: a review and theoretical framework

Abstract

Memory for people and their relationships, along with memory for social language and social behaviors, constitutes a specific type of semantic memory termed social knowledge. This review focuses on how and where social knowledge is represented in the brain. We propose that portions of the anterior temporal lobe (ATL) play a critical role in representing and retrieving social knowledge. This includes memory about people, their names and biographies and more abstract forms of social memory such as memory for traits and social concepts. This hypothesis is based on the convergence of several lines of research including anatomical findings, lesion evidence from both humans and non-human primates and neuroimaging evidence. Moreover, the ATL is closely interconnected with cortical nuclei of the amygdala and orbitofrontal cortex via the uncinate fasciculus. We propose that this pattern of connectivity underlies the function of the ATL in encoding and storing emotionally tagged knowledge that is used to guide orbitofrontal-based decision processes.

INTRODUCTION

There is little doubt that the special status of human beings among living creatures is strongly related to our outstanding ability to organize our behavior with other members of our own species and that this ability is afforded to us by unique attributes of our brains. One of the central questions of social neuroscience is whether our brains are specifically adapted to process social information or whether mechanisms involved in non-social processes are sufficient to explain this ability. Social processes are often complex and can involve a wide variety of domains, such as face perception and emotional regulation, all of which have been intensively studied within the nascent field of social neuroscience over the past two decades. However, little is known about the database of stored information about social words, social behaviors and social entities, or in other words, that which constitutes social knowledge. Social knowledge can be considered a type of semantic memory, gained by experience and instruction, that is used to interpret others’ social behavior and to express and comport ourselves socially.

To understand how the brain represents social knowledge, it is important to consider whether this represents a special, or privileged, category of semantic information. It has been proposed that evolutionary pressures have led to the categorical organization in the brain of a small number of knowledge types, such as living things (animals and plants) and artifacts (Caramazza and Shelton, 1998).This argument is based on the idea that survival and reproductive fitness are improved by the ability to recognize, understand and communicate to others information about some categories of information. One can assume that the most basic knowledge categories that would improve fitness would be food and predator information. Evidence for this view is found in cases of patients with focal lesions who show specific performance dissociations between living things and artifacts (Caramazza and Shelton, 1998). The ability to distinguish close exemplars of a particular type and assign meaning to them may have its roots in evolutionary pressures to distinguish similar items that enhance (or diminish) survival and reproduction.

For social animals, information about other people and our relationships with them is of tantamount importance for mating and survival. The processing requirements in this case are quite different than that required to tell one animal apart from another since social knowledge is highly multimodal (e.g. what someone looks and sounds like), emotional (e.g. whether or not you like the person and what they mean to you), and is associated with both episodic and semantic memories (e.g. memories of shared experiences and biographic information). This review focuses on one part of the brain that has an important role in representing social knowledge: the anterior temporal lobe (ATL). This hypothesis is based on the convergence of several lines of research, reviewed later.

EVIDENCE THAT THE ATLS HAVE A SOCIAL FUNCTION

Anatomy of the ATL

The first piece of indirect evidence linking the ATL to social cognition is anatomical. As we will see, its location and connectivity are suitable for processing social and mnemonic information.

The ATL includes the temporal pole and anterior extents of the perirhinal (BA 35 and 36) and entorhinal cortices, along with the anterior portion of the fusiform, inferior, middle and superior temporal gyri. As such, it is very unlikely that the entire region has one unified function (Martin, 2009; Simmons and Martin, 2009). Superior aspects of the ATL are adjacent to the insula and inferior frontal gyrus. Medial aspects are contiguous with the amygdala and hippocampal input structures, entorhinal and perirhinal cortices. Moreover, the temporal portion of the human olfactory cortex, pyriform cortex, is found on the medial surface of the temporal pole (Allison, 1954; Price, 1990; Ding et al., 2009). However, the precise demarcation of the temporal pole and its extent is ambiguous with regard to most temporal lobe structures. von Economo (1929) remarked that its boundary is ‘always a gradual one, and not at all distinctly marked’; cited in Ding et al. (2009). The ATL is a transitional region where many different cortical regions meet.

Insight into the functionality of the ATL, and more specifically, the temporal pole, can be gleaned by examining the connectivity of the major subdivisions: dorsal and ventral ATL (see Figure 1). The dorsal ATL is mainly interconnected with the medial frontal cortex with some connectivity to orbital regions (BAs 10m, 10o, 11m, 13a, 14c, 14r, 25 and 32) (Kondo et al., 2003). It receives it’s primary input from auditory processing centers in posterior superior temporal sulcus (STS) (Ding et al., 2009; Blaizot et al., 2010). In contrast, the ventral ATL is interconnected with the orbitofrontal cortex (BAs 11l, 13b, 13l and 13m) (Morecraft et al., 1992). The ventral pathway receives it’s input from visual processing regions of the inferior temporal cortex (Ding et al., 2009; Blaizot et al., 2010).

Fig. 2.

(A) Results of an ALE meta-analysis of activations to famous faces (blue), personally familiar faces (red), and their overlap (green; also indicated by arrows). (B) Results of our empirical study of famous and personally familiar faces. In both analyses, the baseline stimulus category was unfamiliar faces (R.J. Von der Heide, L.M. Skipper and I.R. Olson, accepted pending revisions).

Fig. 1.

(A) An illustration of a major association fiber pathway, the uncinate fasiculus, which connects the ATL with orbital and medial aspects of the frontal lobe (used with permission from de Schotten et al., 2012). Below this is a depiction of the sensory sensitivity of the ATL. Yellow = auditory sensitive cortex; blue = visual sensitive cortex (which continues on the inferior surface); green = multisensory sensitive cortex. (B) Functional connectivity of the ATL during processing of abstract person information in the form of sentences about people (depicted based on the data of (Simmons et al., 2010). mPFC, medial prefrontal cortex; amyg, amygdala.

The ATL also projects to entorhinal cortex, perirhinal cortex and parahippocampal cortex, which are closely aligned with episodic memory functions of the hippocampus (Insausti et al., 1987; Moran et al., 1987; Suzuki and Amaral, 1994) reviewed by Blaizot et al. (2010). It has been noted that in human primates, the pattern of connectivity of the ATL bears striking similarity to that of the amygdala (Mesulam, 2000; Kondo et al., 2003, 2005). Indeed, the ATL is highly interconnected with both the amygdala and orbital frontal cortex and is therefore often referred to as a paralimbic region (Mesulam, 2000). Ghashghaei and Barbas (2002) investigated the anatomical connectivity of the ATL, amygdala and orbitofrontal cortex. They reported that the orbitofrontal cortex has bidirectional connections with the amygdala and that these connections were found in a dense U-shaped pattern in the basal lateral nucleus of the amygdala. Auditory and visual association regions in the ATL (e.g. dorsal and ventral ATL) also sent projections to the amygdala. Interesting, the ATL projection sites were in the same territory of the amygdala as the axons from orbitofrontal cortex; significant overlap was found in both the basal lateral and basal medial nuclei (Ghashghaei and Barbas, 2002). These nuclei are known to have a role in emotional memory such as fear conditioning (Pickens et al., 2003).

A large hook-shaped white matter tract, the uncinate fasciculus, links the ATL (anterior BA 20 and 38) and nearby BA 28 (entorhinal cortex) and BA 36 (parahippocampal cortex) to the cortical nuclei of the amygdala and to orbitofrontal cortex. The uncinate is one of the long-fiber association pathways. The function of the uncinate is not known, although it is affected in some psychiatric and neurological diseases (McIntosh et al., 2008; Craig et al., 2009). This white matter tract has an unusually protracted development, being one of the last to finish myelinating (Craig et al., 2009).

The ATL’s proximity to orbital regions causes it to be damaged in the same traumatic accidents that affect ventral prefrontal cortex. Its proximity to the amygdala causes it to be affected by some of the same incidents that affect this region and the neighboring hippocampus, such as herpes encephalitis. Moreover, temporal lobe resection surgery, used to treat intractable epilepsy, typically entails removal of the amygdala and anterior hippocampus, along with the ATL. As such, structure–function relationships detailed in many neuropsychological reports on the effects of orbitofrontal or amygdala lesions on social cognition must be viewed cautiously as the ATL is frequently damaged as well.

Damage to the ATLs causes changes in social behavior

The second line of evidence linking the ATL to social cognition comes from ablation studies. Monkeys with surgical bilateral lesions of the ATL, excluding the amygdala, exhibit grossly abnormal social behavior. They do not produce appropriate social signals (vocal or facial), nor do they appear to recognize the social signals of peer monkeys. They show little social interest in their peers, and at times are rejected from their social group. They are tame and show little aggression toward peer monkeys when provoked. Those with babies were neglectful and often violent toward them, causing consternation among other female monkeys. Some, but not all, of these social problems improved over time (Bucher et al., 1970; Myers and Swett, 1970; Franzen and Myers, 1973; Myers, 1975; Kling and Steklis, 1976; Kling et al., 1993). Orbitofrontal and amygdala lesions lead to a similar pattern of behavior, underscoring the tight coupling of the ATL to these regions. In contrast, lesions to posterior temporal cortex (area IT) and the anterior cingulate cause no social problems. These findings have been replicated in several species of monkeys, suggesting that there is evolutionary conservation of structure-function in the ATL across non-human primate species (reviewed in Olson et al., 2007). It is difficult to understand the functionality of the ATL cortex, apart from its connections, based on this evidence alone, however. Surgical removal of the ATLs would necessarily damage the fibers connecting it to limbic regions of the brain, and thus, might damage the function of those regions as well.

Focal bilateral ATL lesions are rare in humans, although there are two instances of somewhat focal bilateral lesions. One rare disorder is Kluver-Bucy Syndrome, first discovered in monkeys, which is characterized by at least three of these symptoms: docility, hyperphagia, hyperorality, hypersexuality and visual agnosia with hypoemotionality (Kluver and Bucy, 1937). It was first reported in humans in the 1950s following bilateral anterior and medial temporal lobectomy surgery for epilepsy (Terzian and Dalle Ore, 1955). Since that time, case reports indicate that partial or complete Kluver-Bucy arises from a diverse set of disorders—surgical brain lesions, encephalitis, degenerative diseases of the brain and trauma (Gerstenbrand et al., 1983) all of which commonly damage the anterior and medial temporal lobes. At times, partial Kluver-Bucy has been reported to follow focal unilateral ATL damage (Ghika-Schmid et al., 1995) although some of the symptoms such as hypersexuality should be attributed to ATL disconnection from the amygdala and hypothalamus (Kling et al., 1993). The one symptom that we will return to later in the paper is hypoemotionality to visual stimuli.

A second disorder that can affect bilateral ATLs is called frontotemporal dementia (FTD). FTD is a progressive disease characterized by a somewhat rapid degeneration of frontal and/or ATL tissue that, in its early stages, is left or right lateralized, frontal or temporal localized. Temporal damage is most evident in anterior regions, while more medial regions, such as the hippocampus, remain intact at early stages of the disease. Patients with the ‘behavioral variant’ of FTD (bv-FTD), usually associated with right ATL and/or frontal atrophy, exhibit striking social and emotional problems. These patients exhibit fixed facial expressions and dramatic changes in personality, making social interactions uncomfortable for both strangers and family members (Mychack et al., 2001). Social problems include bizarre changes in dress and preferences and loss of manners and social graces (reviewed in Olson et al., 2007). These changes in personality and social conduct relative to premorbid state are core diagnostic features of FTD (Neary et al., 1998). Mendez et al. (2006) described an illustrative case:

A 71-year-old right-handed woman presented with a 2-year history of gradual and progressive personality change characterized by increasingly outgoing and disinhibited behavior. Her family complained that she approached strangers and initiated conversations, sometimes kissing them in greeting, and rubbing their arms. At one point, she was found sitting on a stranger’s lap. She became flip, lighthearted, and silly with frequent laughter and an increased preference for cartoons … she commented on her sex life and that of her adolescent granddaughter, raising her blouse, touching her granddaughter’s breasts, using foul language, and burping or spitting in front of others.

A positron emission tomography (PET) scan showed bilateral ATL hypoperfusion, worse on the left (Mendez et al., 2006). This detailed case report is reminiscent of Franzen and Myers’ (1973) study of rhesus monkeys with bilateral ATL lesions who indiscriminately approached and initiated contact with other monkeys, regardless of rank.

Another variant of FTD, termed semantic dementia, is associated with somewhat left-lateralized anterior, lateral and ventral temporal lobe damage. Patients with this disorder have progressive semantic deficits that are characterized by amodal receptive and expressive semantic deficits, usually assessed by confrontation naming. These deficits are observed in response to a wide variety of stimuli and across a variety of domains (Rogers et al., 2006; Patterson et al., 2007). Behavioral problems are also common in semantic dementia (Bozeat et al., 2000).

One must bear in mind that in humans with bv-FTD or semantic dementia, it is difficult to attribute social deficits strictly to the ATL. FTD is caused by widespread cellular loss, most prominently in orbitofrontal cortex and the ATLs (Hodges and Patterson, 2007), and also extending into lateral, medial and interior portions of the temporal lobe (Mion et al., 2010). Many of the changes in personality experienced by patients with bv-FTS have been correlated with frontal lobe pathology (Edwards-Lee et al., 1997; Rankin et al., 2003; Sollberger et al., 2009) although interestingly, changes in affiliative traits, such as interpersonal warmth, empathy and extraversion, are associated with right ATL pathology (Edwards-Lee et al., 1997; Rankin et al., 2003; Sollberger et al., 2009). Related to this, a recent study reported that the personality trait of seeking and enjoying social relationships correlated with increased gray matter in the inferior ATL (Lebreton et al., 2009). Also, a recent study investigated regions of atrophy related to social impairments in bv-FTD and found that impairments in the identification of sarcasm and emotion in video vignettes was associated with atrophy in the right temporal pole, as well as lateral orbitofrontal cortex, insula and the amygdala (Kipps et al., 2009). The social deficits that accompany FTD go beyond personality changes to include changes that can plausibly linked to social knowledge deficits: inappropriate dress, loss of social graces and loss of knowledge about context-appropriate behavior, including appropriate emotional responses.

The ATLs are critical for person memory

The third line of evidence linking the ATL to social cognition comes from studies of person memory. Memory for who people are, and their social and emotional significance is essential for delineating the social environment and for forming and maintaining group cohesion. It is the glue that binds people together in a social group. A large number of findings converge to show that a region of the ventral ATL plays a critical role in person memory.

One of the earliest neuroimaging studies of face processing reported PET activations in the ATL to personally familiar and famous faces (Sergent et al., 1992). Since that time dozens of functional magnetic resonance imaging (fMRI) studies have shown that the human ATL is particularly sensitive to personally familiar or famous faces as compared with unfamiliar faces (see Table 1). The ATL is acutely sensitive to different types of familiarity manipulations: responsiveness is enhanced by knowledge-base familiarity in the form of semantic knowledge (Nieuwenhuis et al., 2011; Ross and Olson, 2012) but decreased by perceptual familiarity in the form of stimulus repetition (Sugiura et al., 2001; Sugiura et al., 2011).

Table 1.

fMRI studies of person knowledge used in the ALE analysis depicted in Figure 2

Reference	Face stimuli		ATL
	Personally familiar	Famous
Arsalidou et al. (2010)	x		N
Bai et al. (2011)		x	Y
Barense et al. (2011)		x	Y
Bernard et al. (2004)		x	N
Brambati et al. (2010)		x	Y
Donix et al. (2010)	x		N
Eger et al. (2005)		x	Y
Elfgren et al. (2006)		x	Y
Gesierich et al. (2011)		x	Y
Gobbini et al. (2004)	x	x	Y
Henson et al. (2003)		x	Y
Ishai et al. (2002)		x	N
Leveroni et al. (2000)		x	Y
Nielson et al. (2010)		x	Y
Pourtois et al. (2005)		x	Y
Ramon et al. (2010)	x		Y
Ross and Olson (2011)		x	Y
Rothstein et al. (2005)		x	Y
Sugiura et al. (2005)	x		Y
Sugiura et al. (2011)	x	x	Y
Trinkler et al. (2009)	x	x	Y
Turk et al. (2005)		x	N
N	7	18	17/22

Most studies used unfamiliar faces as the baseline task although a few used scrambled or morphed faces (Ishai et al., 2002; Rothstein et al., 2005; Brambati et al., 2010; Gesierich, et al., 2011). The rightmost column indicates whether ATL activations were observed. Note that at least one study (Turk et al., 2005) probably did not have signal coverage in the ATLs.

The results of our meta-analysis of fMRI studies of person memory using the activation likelihood estimation (ALE; Turkeltaub et al., 2002) are depicted in Figure 1 and studies are listed in Table 1 (for full details, see R.J. Von der Heide, L.M. Skipper and I.R. Olson, submitted for publication). We also performed an empirical study of person memory using famous and personally familiar faces (R.J. Von der Heide, L.M. Skipper and I.R. Olson, submitted for publication). In Figure 1, it can be seen that the activation patterns in the ATL to famous and personally familiar faces are remarkable similar across the ALE brain and the empirical study except that the activations in our empirical study are more ventral. This discrepancy probably reflects the use of conventional EPI imaging techniques in studies included in the ALE analysis that provide little to no signal on the inferior surface of the ATL due to susceptibility artifacts and signal distortion (Devlin et al., 2000). Optimized imaging parameters, such as using a short echo time, must be employed to overcome this problem (Visser et al., 2009).

In macaques, face patches in the ATL are found in the middle temporal gyrus and inferior temporal gyrus extending into the ventral surface (Hadj-Bouziane et al., 2008; Moeller et al., 2008; Ku et al., 2011). In humans, Allison et al. recorded from the inferior surface of the human temporal lobe in patients undergoing resection surgery and found that a late event related potential (ERP) termed the P350 localized to the inferior ATL was preferentially sensitive to faces and face priming (Allison et al., 1999; Puce et al., 1999); see also Barbeau et al. (2008). Two recent fMRI studies used multivariate techniques to identify a region in the ventral ATL sensitive to face identity changes (Kriegeskorte et al., 2007; Nestor et al., 2011).

These findings predict that ATL damage from resection or disease should impair person memory. In accordance with this, ATL damage from focal injury, epilepsy resection surgery, or from cell loss in the temporal variant of FTD frequently causes person memory deficits that have been termed ‘associative prosopagnosia’ (Damasio et al., 1990; Evans et al., 1995; Gentileschi et al., 1999; Gainotti et al., 2003; Joubert et al., 2003; Snowden et al., 2004). Associative prosopagnosia is distinct from the more familiar apperceptive prosopagnosia which is defined by difficulties identifying individuals by visual facial features, and is typically caused by damage to posterior portions of the ventral visual stream in and around the fusiform gyrus. Instead, individuals with associative prosopagnosia have intact face perception so that they typically perform well on difficult fact perception tasks such as the Benton Face Inventory. Their problems arise in associating and retrieving information about the face: who that person is, their name, their relationship to the person and any other biographical details. The particular constellation of deficits can often be predicted by the lateralization of the lesion. Left-lateralized ATL damage commonly impairs the ability to retrieve proper names of familiar (e.g. one’s sister) and famous faces (Semenza, 2006) and can impair the ability to learn new face-name associations. Right-lateralized damage usually impairs recognition of famous or personally familiar faces, loss of feelings of familiarity or a failure to recollect person-specific semantic information (Gainotti, 2007a,b). A similar left-right dissociation is observed in FTD (Snowden et al., 2004). Thus, the left ATL appears to be more engaged when people’s names must be retrieved, whereas the right ATL mediates between faces and biographical knowledge, which may be a precursor to name retrieval.

There is evidence that the recognition deficits following ATL lesions are not based solely on loss of access to verbal information, such as names. Barton and Cherkasova (2003) required a group of focal lesion patients with various face processing deficits to mentally imagine and answer questions about various famous faces. Mentally imaging Marilyn Monroe’s face, for instance, requires the linkage of a name to a face stored in visual long-term memory. The authors found that patients with damage to the fusiform gyrus had poor face recognition abilities but relatively normal facial imagery skills. However, a patient with bilateral ATL lesions had normal face perception but was not able to evoke mental images of famous faces such as Hitler. This finding is from a case study so it must be interpreted cautiously, however, it hints that the ATLs role in person memory extends beyond semantic memory to include the retrieval of visual long-term memories.

The associative prosopagnosia caused by ATL lesions typically gives rise to a multi-modal person identification deficit, probably because such lesions tend to destroy both dorsal and ventral aspects of the ATL. Several lines of evidence indicate that there is sensory specificity in the ATL (reviewed by Skipper et al., 2011). Auditory and visual processing streams remain distinct in the ATL except for in the polar cap and along the STS (Ding et al., 2009) thus it seems likely that vocal identity and facial identity are processed in distinct portions of the ATL. Indeed, vocal identity discrimination is associated with sensitivity in the superior ATL (in the upper bank of the anterior STS) in both humans and monkeys (Belin and Zatorre, 2003; Belin et al., 2004; Andics et al., 2010), while facial identity discrimination is associated with sensitivity in the ventral ATL.

Specific function of the ventral ATL in person memory

There is a remarkable convergence of findings from PET, fMRI, single-unit recordings in monkeys and humans, and lesion studies all linking the ventral ATL to visual-person-memory functions. Its specific function is in coding facial identity by linking specific faces to semantic, episodic and emotional knowledge. Cells in this region have association formation capabilities. For instance, single neurons in the ventral ATL of monkeys that initially responded to only one abstract pattern would later respond to a second abstract pattern that had been associated via training with the first (Sakai and Miyashita 1991). In addition, cells in the ventral ATL in monkeys can represent an associative pairing acquired via training between faces and abstract patterns (Eifuku et al., 2010).

In humans it has been reported that patients with left ATL lesions are unable to form new associations between names and pictures of objects (Sharon et al., 2011). Tsukiura et al. (2009) showed that successful encoding of person-related semantics with their names was associated with left ATL activity. The association formation capabilities of the ATL may rely on the hippocampus during the initial encoding phase. Recently, it was shown that 25 h after encoding face-location associations, there was an increase in the functional coupling between the fusiform face area (FFA), spatial representational areas in the posterior parietal cortex and the left ventral ATL (Nieuwenhuis et al., 2011).

It should be mentioned that ATL face patches in monkeys are sensitive to novel and familiar faces (Tsao et al., 2008). This has also been reported in fMRI studies by our laboratory and others (Kriegeskorte et al., 2007; Nestor et al., 2011; R.J. Von der Heide, L.M. Skipper and I.R. Olson, submitted for publication). Based on this, it has also been argued that the ATL is part of a network for perceptual face discrimination (Nestor et al., 2011). The evidence for this idea is complicated. On the one hand, perceptual deficits in face recognition are almost never reported after unilateral ATL resection (reviewed in Olson et al., 2007). On the other hand, cells in monkey ventral ATL respond to some perceptual changes—changes in facial identity—but not to other perceptual changes—such as changes in facial orientation or rotation (Eifuku et al., 2010, 2011). There is also a growing literature suggesting that a small, medial portion of the ATL, called perirhinal cortex, is involved in differentiating complex visual stimuli with many overlapping features, such as faces, but not simpler visual stimuli (Graham et al., 2010).

In sum, the evidence is strong evidence that the ventral ATL has a mnemonic role in person identification and that this process relies on the association formation capacities of cells in this region. Interestingly, portions of the ventral ATL are sensitive to the exact perceptual attributes required for person identification.

The ATLs and social networks

Since the ventral ATL has an important role in person memory, it would make sense if it also had a role in the maintenance of social networks, a behavior that partly depends on person memory capacity. Kanai et al. (2012) reported that the amount of gray matter in the left entorhinal cortex/inferior-medial ATL correlated with the number of Facebook friends. Another group found that gray matter in a more superior portion of the ATL, the left anterior superior ATL, correlated with social network size and theory of mind abilities (Lewis et al., 2011). Intriguingly, an fMRI study of monkeys found that the amount of gray matter volume in the anterior superior ATL, running into the pole, correlated with the size of the social group in which the monkeys were housed (Sallet et al., 2012). In fact, the only brain imaging study of the whole brain that did not report a correlation between social network size and ATL gray matter volume was by Bickart et al. (2010).

Neuroimaging studies of high-level social cognition

The last piece of evidence linking the ATL to social processing is from neuroimaging. A large number of studies investigating various aspects of social cognition, such as theory of mind, have reported activations in the dorsal and polar ATL (reviewed by Olson et al., 2007; Simmons and Martin, 2009; Wong and Gallate, 2012). For instance, we showed study participants Heider and Simmel social attribution stimuli (Heider and Simmel, 1944) or similar non-social moving stimuli and asked them to make judgments about friendliness or weight. Social activations were found along both banks of the STS extending into the pole (Ross and Olson, 2010). Other laboratories have reported nearly identical findings (Castelli et al., 2000; Saffran et al., 2003; Schultz et al., 2003; Ohnishi et al., 2004). A range of theory of mind manipulations, such as presenting brief vignettes or cartoons about theory of mind, also activate the ATL; see Figure 3.

Fig. 3.

Task and fMRI activations from (Ross and Olson, 2010). From top to bottom: Heider and Simmel (1944) social attribution task (baseline task had same low-level visual characteristics but lacked the impression of animacy and biological motion); a verbal theory of mind task (baseline task had the same task demands, contained the same number of words, and contained proper nouns but lacked a theory of mind element); a semantic comparison task (baseline task required the comparison of two non-social words that were matched to the social words on many psycholinguistic variables).

Other high-level social tasks evoke activations in the ATL as well: moral judgments (Moll et al., 2005), social vs non-social gestures (Straube et al., 2010), socio-emotional stories (Ferstl and von Cramon, 2002), music believed to be composed by a human vs a computer (Steinbeis and Koelsch, 2009) and sounds evoking a social scene such as footsteps (Saarela and Hari, 2008).

What is striking about these findings is the large variation in stimuli and tasks that all evoke activations in the ATL. The consistency of ATL activations to high-level social tasks has led to its inclusion in the ‘social brain network’ outlined by Frith and Frith (2003, 2010); however, its precise role in this network has not been clearly elucidated. This is because the common underlying cognitive component that relates these tasks to one another remains obscure. In the next section, we discuss why we think the ATL’s particular role in social cognition is encoding and retrieving social concepts.

Semantic memory and the ATL: the social knowledge hypothesis

Semantic memory, also referred to as conceptual knowledge, refers to knowledge of objects, word meaning, facts and people without reference to a particular time or place (Tulving, 1972). For instance, knowing that the Eiffel Tower is located in Paris is an instance of semantic memory. Remembering your trip to see it is an instance of episodic memory. One influential view of ATL function proposes that this region serves as an amodal ‘hub’ for semantic memory (Patterson et al., 2007). As such, this region serves to link together sensory specific and semantic associations located throughout the brain (McClelland and Rogers, 2003; Patterson et al., 2007). Evidence for this view is drawn primary from studies of patients with semantic dementia, mentioned earlier. Patients with this disorder have semantic deficits that are characterized by amodal receptive and expressive semantic deficits in response to pictures, words, sounds and even olfactory information (Bozeat et al., 2000; Coccia et al. 2004; Luzzi et al., 2007); reviewed by Patterson et al. (2007).

A central contention of the Hub Account is that the ATL is more sensitive to specific, rather than general semantic information retrieval. For instance, some studies have shown that semantic dementia patients’ success or failure on any given semantic test is in part determined by the specificity of information required by the task (Rogers and Patterson, 2007). Hodges et al. tested a semantic dementia patient on a simple picture naming task four times over the course of 2 years. Over time the patient’s name for the same picture became less specific and crossed larger category boundaries. For instance, when shown a rooster at the first testing session, the patient named it as a ‘chicken.’ In subsequent sessions, he could only name it as a ‘bird’ and finally labeled the picture as a ‘dog’ (Hodges et al., 1995). A socially important type of specific information—the names of friends and family—is one of the earliest types of knowledge lost in semantic dementia (Hodges et al., 1995) and is frequently described as a loss of specific-level processing (Patterson et al., 2007).

The problem with this example is that socialness is confounded with semantic specificity. We contrasted the effects of socialness and specificity in two fMRI experiments (L.M. Skipper and I.R. Olson, submitted for publication) using different types of stimuli (pictoral and verbal), and different tasks. We found a strong effect of social content in the ATL. This region was highly active in response to pictures of people, words that name people and words that describe personality traits. However there was no region within the ATL that was sensitive to specific level information, even at extremely liberal thresholds. Thus, our findings failed to replicate the ATL findings of an earlier PET study (Rogers et al., 2006). Instead, specificity effects were observed in the posterior inferior temporal lobes and inferior occipital lobes, a finding that partially replicates a prior neuroimaging study of semantic specificity (Tyler et al., 2004).

There are other findings that are also problematic for the specificity contention of the Hub Account. First, five of the six neuroimaging studies that are typically cited (e.g. (Patterson et al., 2007) in support of the ATL-semantic specificity link used social stimuli: familiar faces, proper names or familiar voices (Gorno-Tempini et al., 1998; Gorno-Tempini and Price, 2001; Nakamura et al., 2001; Tsukiura et al., 2006) so again, specificity and social factors are confounded. Second, it is possible that difficulties retrieving highly specific information is actually due to a different, correlated factor: atypicality/uncommonness. In one study, patients with semantic dementia were required to name and classify exemplars of different categories of knowledge. The patients had greatest difficulty naming atypical members of categories such as hairless cats (Mayberry et al., 2011).

Aside from the specificity problem, there are other challenges to general semantic accounts of the ATL, like the Hub Account. First, it is not clear that a semantic hub is necessary for semantic processing (Simmons and Martin, 2009; Simmons et al., 2010). Second, evidence for general semantic functions of the ATL is overwhelmingly drawn from studies of patients with semantic dementia, who have cell loss in regions outside of the ATL. Finally, ATL activations are not evident across the majority of fMRI studies on semantic memory (Visser et al., 2009). A comprehensive meta-analysis of the neuroimaging literature found that the most commonly activated region in studies of general semantic memory was the left angular gyrus (Binder et al., 2009).

Social concepts and the ATL

The information presented in the last section argues for a revision to the dominant view—the Hub Account—of the functionality of the ATL. Here, we propose two revisions: (i) that the ATL is not a general-purpose semantic processor. Instead, portions of the ATL are particularly involved in processing concepts with social-emotional content; and (ii) that the ATL is not an amodal semantic processor. Instead, it contains distinct sensory subdivisions. We note that this later point is not controversial among anatomists, having been reported in several studies (Kondo et al., 2003; Ding et al., 2009; Blaizot et al., 2010).

A wealth of findings reviewed earlier (see also Moll et al., 2005; Zahn et al., 2007; Zahn et al., 2009; Ross and Olson, 2010, 2011; L.M. Skipper and I.R. Olson, submitted for publication) support the view that the ATL is involved in encoding and retrieving social concepts. Canonical concepts in this category would be concepts about people—personality traits, proper names, biographical information, etc. However, other seemingly non-social concepts may be processed by this region because they have personal significance and emotional tone. For instance, both the physical appearance and the particular ring tone of my iPhone can evoke the concept of ‘my iPhone’. However, that concept is also attached to several emotions—feelings of familiarity, possessiveness and affection. Other objects, such as a particular vase, may not have any personal significance so the social knowledge hypothesis would predict that the ATL is not required to retrieve the conceptual information linked to that particular vase.

The social knowledge hypothesis makes several predictions. First, social concepts should preferentially activate portions of the ATL. Zahn et al. (2007) tested this by comparing brain activations while subjects made semantic similarity judgments of two separate classes of lexical stimuli. The first class of lexical stimuli was words describing positive (honor—brave) or negative (tactless—impolite) human social concepts. The second class of word stimuli described behaviors related to animal use and biological function (nutritious—useful) that can, in principle, apply to humans as well. Words of each class were presented in pairs and the task was to judge whether the words were semantically related or not. The results showed that a small region of the superior ATL was sensitive to social concepts (Zahn et al., 2007). Our laboratory replicated these findings (Ross and Olson, 2010). Similarly, patients with FTD who have cell loss in the right superior ATL exhibit disproportionate impairments in understanding social concepts compared to other abstract, but non-social concepts (Zahn et al., 2009). We recently found that verbal information about personality traits activates the superior-polar ATL (L.M. Skipper and I.R. Olson, submitted for publication).

A second prediction is that tasks requiring the processing and retrieval of social concepts should show overlapping activations in the ATL, even when the tasks are substantially different in design, stimulus quality and task demands. We recently tested this by comparing the functional overlap between perceptually and cognitively distinct tasks whose only commonality was a social component (Ross and Olson, 2010). One task was verbal and highly semantic, using social word stimuli such as ‘friendly’. Another task was visual, using film clips of geometric objects interacting in either a social or nonsocial manner (see Figure 3). The results showed that these tasks elicited overlapping activations in the left superior ATL.

A third prediction is that non-social objects should activate the ATL when they are imbued with social-emotional meaning. We tested this by training subjects to associate social or non-social adjectives with novel objects. In the scanner, the task was to look at the object and recall the associated adjective. Associated with social traits were found to preferentially activate the ATL (Skipper et al., 2011).

Fourth, it is known that cells in monkey ATL, and thus presumably human ATL, can quickly form associations between faces and other stimuli (Eifuku et al., 2010, 2011). This predicts that the ATL should be sensitive to the association between people and traits. One example of this is a stereotype—a culturally held belief about specific groups of people. At the heart of most stereotypes is a strong association between certain types of people and specific personality or behavioral traits.

A small number of studies have linked the ATL to stereotyping. In two separate studies, Gallate and colleagues found that inhibiting the left or right ATL by applying repetitive transcranial magnetic stimulation (TMS) decreases implicit stereotyping as measured by the implicit association task (IAT). This task measures the associative strength between two conceptual categories such as ‘women’ and ‘math’. The TMS effects were specific to the social version of the task suggesting that the ATLs are more sensitive to associations about people than to associations about things (Gallate et al., 2011; Wong et al., 2012). It has also been reported that IAT performance is perturbed after ATL damage and that affective trait associations between verbal information and faces fails to occur when the temporal pole and amygdala are damaged (Gozzi et al., 2009; Toderov and Olson, 2008). One fMRI study reported that the retrieval of stereotypic knowledge, as compared to other types of conceptual knowledge, activated the ATL along with other regions commonly recruited in social cognition tasks such as medial prefrontal cortex (Contreras et al., 2012). Even more tellingly, David Amodio’s laboratory asked Caucasion study participants to classify pairs of Black or White faces according to a friendship judgment (e.g. Which of these two people would you more likely befriend?) or a trait judgment (e.g. Which of these two people is likely to be more athletic?) while being scanned. Activations in the left ATL correlated with a behavioral measure of bias during the friendship judgments and also a behavioral measure of stereotyping during the trait judgments (Gilbert et al., 2012).

Selectivity of the superior ATL for high-level social information

The ATL is discussed in three distinct literatures: sentence processing, semantic memory and social cognition. This raised the question of whether the superior ATL subregion commonly reported in theory of mind, morality, and social word processing is selective for social information? The evidence does not permit us to state that this region is modular and only sensitive to social concepts, especially since the ATL is most likely comprised of several distinct substructures. However, the reviewed evidence makes it quite clear that portions of the ATL show greater sensitivity to social concepts than similar non-social concepts. Indeed, it is rare to observe ATL activations in the absence of social cues. For instance, although ATL activations are frequently reported in studies of sentence processing (Hickok and Poeppel, 2007), most of the sentence stimuli used in these studies are about people, while the comparison condition is usually word strings. Even the processing of stimuli that is only implicitly social, such as the sound of human footsteps, activates the ATL (Saarela and Hari, 2008). In sum, portions of the ATL are more sensitive to social stimuli than other types of stimuli that have been tested. However, it is premature to state that it is completely selective for social concepts.

CONCLUSIONS AND OPEN QUESTIONS

It has been argued that primates’ social groups are so distinct from those of other species because of their ability to use complex forms of social knowledge to predict the behavior of others and to manipulate others (Cheney and Seyfarth, 2007). The neural basis of such knowledge has received little attention. In this article, we reviewed evidence showing that portions of the ATL are involved in encoding and representing social knowledge. In an earlier review of this literature, we proposed that the ATL’s associate highly processed perceptual information with emotional information to form a personal semantic store (Olson et al., 2007). In this review, we elaborated on this idea and discussed additional findings supporting this contention.

To summarize the data supporting the ATL’s involvement in social memory:

1. The ATL appears to have a role in affiliative behaviors that underlie group formation. Bilateral ATL damage in humans and monkeys leads to changes in these behaviors.

2. The ATL has a critical role in person memory as supported by findings from neuropsychology, electrophysiology and neuroimaging. There are two important axes of functionality: left-right lateralization and dorsal-ventral sensory processing. The left ATL is associated with proper name retrieval, the right with feelings of familiarity and retrieval of biographical information. The dorsal ATL is associated with identification of particular individual’s voices, the ventral ATL with the identification of particular individual’s faces.

3. The ventral ATL has a role in person memory at both encoding and retrieval. The encoding function is based on the ability of cells in this region to rapidly associate faces with other pieces of information including affective tone.

4. The superior-polar ATL is involved in more abstract forms of social processing, such as theory of mind. Its functional role may be to encode and retrieve social concepts including trait information that are required to understand social behavior, derive social meaning and maintain social bonds in an ever-changing social landscape. Polar aspects of the ATL are highly interconnected with two neuromodulatory regions, the amygdala and the hypothalamus (Kondo et al., 2003, 2005) which may serve to tag salient or relevant information, making it easier to retrieve later on (Adolphs, 2010).

The social knowledge hypothesis of ATL function reconciles the extensive literatures on general semantic processing (Patterson et al., 2007) and imaging studies in social cognition that have implicated the ATLs as part of the ‘social brain’ (Moll et al., 2005; Olson et al., 2007; Frith and Frith, 2010). Previously, it was proposed that the ATL’s role in social cognition is to process social scripts or schemas (Frith and Frith, 2003); however, we believe the data better support the social knowledge hypothesis. Several questions remain unanswered however.

First, is the ATL inherently sensitive to social knowledge or, instead, does this type of information receive prioritized processing due to its inherent salience (Wong and Gallate, 2012)? Social stimuli may be a canonical class of salient concepts to social animals. We favor this view as it is very parsimonious, it affords the ATL a greater role in conceptual processing, and it accounts for a wider range of findings. It is plausible that the ATL would be sensitive to seemingly non-social concepts that have personal significance and emotional tone. By this token, naming of famous landmarks may be impaired following ATL damage (reviewed by (Tranel, 2009)) and famous landmarks may activate the ATL (e.g. Ross and Olson, 2011; R.J. Von der Heide, L.M. Skipper and I.R. Olson, submitted for publication), because these seemingly neutral items have become social and emotional by shared cultural consensus. The Eiffel Tower is not just a lattice tower made of iron. It symbolizes a country (France), a key destination for world travelers and the idea of French romance (Ross and Olson, 2011). Many of us have emotional memories of visiting the Eiffel Tower Some everyday objects such as laptop computers could also be imbued with emotion (I adore my Mackintosh laptop) to the degree that the emotional value is part-and-parcel of the conceptual meaning of that object.

Second, what is the relationship between language and social processing? As noted by Syal and Finley (2011), social motivation is a necessary requirement for normal language acquisition . In addition, the content of human communication is primarily social information such as gossip (Dunbar, 1996; Mesoudi et al., 2006), which raises the question of whether language evolved to support social processing. Interestingly, the ATLs have been implicated not only in social processing but also in higher-order aspects of language, such as sentence processing and semantic memory.

Third, many of the findings relating high-level social processing, such as theory of mind, to the ATL implicate superior-polar regions. In non-human primates, the superior ATL, but not inferior ATL, contains cells that are responsive to social-emotional stimuli (Kondo et al., 2003). The polar tip contains bidirectional connections with medial and orbital regions of the frontal lobe, implicating this region in high-level social processing (reviewed by (Moran et al., 1987). Whether the face sensitivity on the inferior surface is more closely aligned with high-level visual processing than with more general social processing per se is not known.

Finally, we would like to offer a speculation as to the relationship between the ventral ATL, the orbitofrontal cortex and the amygdala. We begin with the argument that was first stated by others—that memories are useful only to the degree that they help make advantageous decisions (Murray and Izquierdo, 2007). Take the example of person memories. As we discussed earlier, the ventral ATL contains neurons that can make rapid associations between faces and other stimuli (Eifuku et al., 2010), potentially forming our storehouse of biographic information. The amygdala and hypothalamus may interact with the ventral ATL to give these person-based memories emotional tone. However at this stage, the memories are static and useless because there is no way for them to guide decision making. This step occurs through an interaction of the ventral ATL with lateral orbitofrontal cortex via the uncinate fasciculus. The orbitofrontal cortex is thought to represent expected outcomes such as the size and probability of reward combined with the effort that must be expended to achieve it (Padoa-Schioppa and Assad, 2006; Murray and Izquierdo, 2007). The orbitofrontal cortex uses a particular type of affect (that associated with reward and punishment) to guide decision making between multiple competing stimuli (Murray and Izquierdo, 2007). For instance, if you need to decide who to trust with a secret—Marian or Rebecca—the action of the orbitofrontal cortex would allow you to make an optimal choice based on past experiences with Marian and Rebecca stored in the ventral ATL. The reciprocal relationship between the ATL–orbitofrontal cortex–amygdala complex allows the positive or negative outcome of this decision to be stored in the ATL as part of the biographic memories of Marian and Rebecca.

This view predicts that bilateral ATL damage should cause a type of ‘associative social agnosia’ where inappropriate behavior and perhaps a reduction in interest arises because social entities, social behaviors and social terms, are stripped of rich, personal meaning. In contrast, focal orbitofrontal lesions should lead to a pattern of behavior characterized by intact social knowledge but impaired utilization of such knowledge.

There is some evidence for this prediction. ATL damage or atrophy can cause loss of person knowledge (reviewed in Gainotti, 2007a; Olson et al., 2007), hypoemotionality to visual stimuli (as in Kluver-Bucy Syndrome) and deficient understanding of social words compared with non-social words (Zahn et al., 2009). It has also been reported that adult patients with acquired sociopathy due to orbitofrontal damage have intact knowledge of social rules and norms, though they are unable (or unwilling) to use this knowledge to guide decision making (Saver and Damasio, 1991). Interestingly, three studies have reported that the structure that links the ATL to orbitofrontal cortex, the uncinate fasiculus, is less organized in psychopaths or people with antisocial personality disorder compared with matched controls (Craig et al., 2009; Motzkin et al., 2011; Sundram et al., 2012). These findings suggest that the ATL and its connectivity to other limbic regions may have an important role in modulating higher-level social behaviors.