Bilateral amygdala damage impairs the acquisition and use of common ground in social interaction

Abstract

Objective

The development of “common ground,” or mutual knowledge of shared information, is believed to require the ability to update a mental representation of another person’s thoughts and knowledge based on verbal information and nonverbal social and emotional signals, in order to facilitate economical communication. As in other forms of everyday social communication, the development of common ground likely requires the orchestration of multiple cognitive processes supported by various neural systems. Here, we investigate the contribution of the amygdala to these processes.

Methods

SM, a patient with complete, focal, bilateral amygdala damage and deficits in social and emotional processing, and five healthy comparison participants, each interacted with a familiar partner. We investigated the participants’ ability to develop and use referential labels across twenty-four dynamic, collaborative interactions. Participants verbally directed their partner how to arrange a set of 12 abstract tangrams while separated by a low barrier, allowing them to see each other but hiding their tangrams.

Results

In contrast to comparison participants, SM exhibited an impaired rate of learning across trials and did not show the typical simplification in the labels generated during the interactions. Detailed analyses of SM’s interactional discourse and social behavior suggested that she has impaired perspective-taking or what can be interpreted as deficient “theory of mind,” manifested in abnormal “language-in-use.”

Conclusions

These results support the conclusion that the amygdala, a structure critical for social and emotional processing, plays an important role in the acquisition and use of common ground and in social communication more broadly.

Introduction

Referential communication plays a critical role in the routine use and understanding of language. One way to understand how interlocutors construct and resolve references in conversation is through the notion of common ground, i.e., the beliefs, knowledge, and assumptions about current and previous communicative interactions that are mutually shared between speakers and listeners, and that facilitates rapid and economical communication (Clark, 1992). For example, comprehension of the reference “nine-eleven” relies on our shared knowledge that the reference is to the date 9/11/01 and the attacks against the U.S. that took place on that day. Common ground is developed through a rich, dynamic, interactive process that requires the orchestration of numerous cognitive systems. Among these the role of memory has been of considerable interest (e.g., Clark, 1992; Horton, 2007; Horton & Gerrig, 2005) as the traditional assumption has been that conversational partners routinely search an explicit record of previous communicative exchanges and continuously update this record with new events (Clark, 1992; Clark & Marshall, 1981). Similarly, the traditional literature on common ground has also implicated social and emotional processes because both interlocutors must be able to create and update a “model” of what is known in the other person’s mind based on verbal and nonverbal social and emotional cues necessary for understanding the thoughts and feelings of others (Krauss & Fussell, 1996). This ability to understand and infer the thoughts of others requires perspective-taking, or the ability to “know what another knows,” perhaps akin to “theory of mind,” is believed to be necessary for constructing appropriate and effective references utilizing common ground between interlocutors (Krauss & Fussell, 1996; Clark, 1996; but see Horton & Keysar, 1996; Keysar, 2007).

Earlier work from our laboratory focused on one of the basic cognitive systems implicated in common ground, namely determining the memory requisites of common ground. This work suggested that an explicit record of shared knowledge is not necessary for the development of common ground. Rather, some forms of common ground can apparently be mediated by more implicit memory systems. Using a collaborative referencing paradigm, we had patients with hippocampal amnesia verbally direct their familiar partners (e.g., spouse, friend) how to arrange a set of 12 abstract tangrams (Figure 1) while separated by a low barrier, which allowed them to see each other, but hid their workspaces and tangrams. Despite severe declarative memory impairments, these patients displayed robust collaborative learning for referential labels, at a rate equal to that of healthy comparison participants, resulting in increasingly rapid and economical communication (or common ground) with their partner (Duff, Hengst, Tranel, & Cohen, 2006). Across the twenty-four trials, and consistent with (Clark’s (1992)) collaborative referencing model, common ground was displayed as verbal labels for the tangrams became increasingly concise and simplified and the amount of overt collaboration (e.g., number of words, turns) declined across interactions. For example, the first time an amnesic participant attempted to describe Card 4 in Fig. 1 he stated looks almost … the opposite of somebody slumped down, on the ground with the same type of head. By the end of the task, after 24 trials, the reference was simply siesta man. We speculated that the collaborative learning demonstrated by the amnesic patients was mediated not only by components of their preserved non-declarative memory, but may also be facilitated by the collaboration and social interaction with their partner. Indeed, previous research has suggested that the development of common ground may also involve the ability to update one’s representation of another’s mind based on social and emotional signals (Clark, 1996; Krauss & Fussell, 1996). Therefore, if intact social and emotional processing is a critical ingredient for learning referential labels and for the development of common ground in this task, it can be hypothesized that damage to neural structures important for such processing will impair the development and use of common ground in social interaction.

Figure 1.

The set of 12 tangrams used by the participants during the collaborative referencing task. Note: the numbers below each image were not on the participants’ cards.

One candidate region that may be involved in the ability to perceive and utilize social and emotional cues during social interaction is the amygdala. The amygdala is important for a variety of basic social and emotional processes, such as the recognition of emotional facial expressions (Adolphs, Tranel, & Damasio, 1998; Adolphs, Tranel, Damasio, & Damasio, 1994; Morris et al., 1996), processing emotional and social stimuli such as scenes and pictures (Hariri, Tessitore, Mattay, Fera, & Weinberger, 2002; Norris, Chen, Zhu, Small, & Cacioppo, 2004), the ability to attribute social and emotion characteristics to non-biological objects (i.e., to anthropomorphize; Heberlein & Adolphs, 2004), the integration of emotional cues for advantageous complex decision-making (Bechara, Damasio, Damasio, & Lee, 1999), and the enhancement of declarative memories with emotional content (Adolphs, Cahill, Schul, & Babinsky, 1997; Cahill, Babinsky, Markowitsch, & McGaugh, 1995).

Previous research with the patient SM, who has complete, focal, bilateral amygdala damage, has also revealed additional abnormalities in her ability to process social and emotional information which may have direct impacts on social interaction and communication. These include abnormal eye contact during social exchanges (Spezio, Huang, Castelli, & Adolphs, 2007), an abnormal sense of “personal space” (Kennedy, Glascher, Tyszka, & Adolphs, 2009; Tranel & Hyman, 1990), impairments utilizing social and emotional cues to recognize complex social emotions (e.g., embarrassment; Adolphs, Baron-Cohen, & Tranel, 2002), and the ability to form complex social judgments (e.g., trustworthiness; Adolphs et al., 1998). Moreover, recent evidence suggests that, despite preserved declarative memory, SM displays impairments in the ability to appropriately use new social and emotional information to update her moral judgments of others (Croft et al., 2009). Since the development of common ground also requires the ability to use social and emotional verbal and nonverbal information (e.g., tone of voice, facial expressions) to flexibly update one’s knowledge of another person’s perspective and knowledge (Krauss & Fussell, 1996), amygdala damage may also impair the development and use of common ground in social interaction.

Further evidence for the importance of the amygdala for social interaction comes from studies that have examined perspective-taking using laboratory-based theory of mind tasks (e.g., understanding vignettes, faux pas). This research suggests that the amygdala may be critically important for the ability to take another’s perspective, or theory of mind (Shaw et al., 2004; Stone, Baron-Cohen, Calder, Keane, & Young, 2003). Since effective communication requires interlocutors to take each other’s perspective in order to understand and place each utterance in the context of their common ground, the amygdala may be important for the development and use of common ground. In the collaborative referencing paradigm, speakers must use perspective-taking to create effective descriptions, and must monitor the listener’s level of comprehension to revise or “refashion” ineffective descriptions in order to converge on a mutual perspective (Clark, 1992; Krauss & Fussell, 1996). Across subsequent interactions, speakers take the other’s perspective to “know what they know” in order to assess what information is mutually shared in order to create more efficient utterances (Krauss & Fussell, 1996; Krauss, Fussell, & Chen, 1995).

Using the collaborative referencing paradigm from our previous work (Duff et al., 2006), the current study tests the hypothesis that the amygdala plays a critical role in the acquisition and use of common ground. We investigated the ability of SM and five healthy comparison participants, each interacting with a familiar partner, to develop and use referential labels across a series of highly dynamic and social interactions. This study, which provides a detailed analysis of a patient with bilateral amygdala damage interacting socially in a protocol closer to real-world communication than previous laboratory tasks, helps to elucidate the role of the amygdala in complex communication and common ground, and helps to characterize further the amygdala’s contribution to social interaction more broadly.

Methods

Participants

SM is a right-handed woman with complete, focal, bilateral amygdala damage with minimal damage elsewhere, who was forty-two years old at the time of the study. She has 12 years of education. SM’s neuroanatomical and neuropsychological profiles have been published in extensive detail elsewhere (see Adolphs & Tranel, 2000; Tranel & Hyman, 1990). She has a mostly normal neuropsychological profile, including normal performance on standardized tests of visuospatial and visuoperceptive abilities, speech, language, and memory, including working memory. SM’s intellectual functioning is in the upper end of the low average range (Wechsler Adult Intelligence Scale-R Full-Scale IQ (WAIS FSIQ)=88). SM does not have difficulty in her general use of language, and this has been documented extensively since our earliest publication of her case more than two decades ago (Tranel & Hyman, 1990). In casual conversation, in fact, there are no abnormalities in her verbal interchanges, other than her hoarse voice (see Tranel et al., 2006, for further information relevant to this point). However, as described earlier, SM exhibits deficits in numerous aspects of social and emotional processing (see Adolphs & Tranel, 2000 for review).

Five healthy, normal women, free of neurological or psychiatric conditions and matched to SM on age (mean=43.8±8.9), education (mean=13.6±2.2), and handedness, served as comparison participants. The estimated intellectual functioning of these 5 comparison participants was in the middle of the average range (FSIQ ~100), based on the WAIS-III Vocabulary and Matrix Reasoning subtests. (As a follow up analysis, we collected additional data from two brain-damaged comparison participants who were IQ-matched to SM, see Results for further details.)

Each participant selected a familiar communication partner (e.g., spouse, friend, with at least 5 years of communication history), free of neurological and psychiatric conditions, with whom they completed the collaborative referencing task. The familiar partner for SM was a 33 year-old male who has known SM and her family for more than seven years. The familiar communication partners for the comparison participants included 2 friends and 3 spouses with a mean time known of 14.8 years. It should be noted that in our previous work (Duff et al., 2006; Gallegos et al., 2007), we have found no difference in performance based on how long the pairs knew each other or based on the nature of the relationship (e.g., spouse vs. friend) on the dependent variables reported here. In fact, previous collaborative referencing studies (e.g., Clark & Wilkes-Gibbs, 1986) using pairs who were complete strangers, have documented successful task performance on the dependent variables of interest. All participants gave informed written consent approved by the Institutional Review Board of the University of Iowa.

Collaborative referencing task procedure

Procedures for the collaborative referencing task followed Duff et al. (2006, 2008). All participant pairs (SM or matched comparisons and their familiar communication partners) performed the collaborative referencing task across 24 trials, with six trials conducted in each of four sessions, two sessions per day with at least 30 minutes between sessions.

During the collaborative referencing task, participants sat facing each other, with identical boards numbered 1–12 and 12 identical cards with abstract Chinese tangrams, shown in Figure 1. A low barrier was placed between the pair, which allowed them to see each other’s faces, but hid their workspaces and tangrams. For all trials, SM or matched comparison participant was assigned the role of “director” and their familiar partner was assigned the role of “matcher.” The cards were placed on the director’s (SM or matched comparison) board in a predetermined, unique order for each trial. The director verbally communicated to the matcher (familiar partner) how to fill the numbered spaces with their cards, so that at the end of the trial, both boards would look identical. However, both participants were allowed to communicate freely, including the use of any facial expressions or gestures, and no restrictions were placed on the matcher’s communication. Each trial was terminated when the participants believed they had placed all of the cards correctly. The time to complete each trial was recorded in seconds, however participants were told that the time was not as important as accuracy of the card placements.

Data analysis

Transcribing the interactions

Sessions were videotaped and transcribed in their entirety including task instructions, each of the 24 trials, and conversations between trials using a three-stage consensus procedure (see Duff et al., 2008). Briefly, in the first stage, the original transcriber transcribed all utterances, audible sounds, and pause times from the audio portion of the taped interactions. In the second stage, the original transcriber watched the videotaped interactions and added card placements and made corrections to the audio content of the transcript. In the third stage, a consensus transcriber and the original transcriber viewed the video together and generated the final version of the transcript, or the consensus transcription. Corrections and additions were made through discussion and consensus.

Coding communicative resources

As a preliminary means of characterizing the data set, we coded communicative resources employed across each of the 24 trials and by both participants in a pair (the director and the matcher). This analysis was conducted only on data from the 24 individual trials and not from the task instructions or between trial talk and for only two types of resources: interactional turns and words. Interactional turns were defined as utterances produced by one individual and could include both verbal and nonverbal resources. Some turns consisted of a gesture or nonverbal back-channel response (e.g., head nod) alone. Turn boundaries were denoted by a change in speaker. When two individuals spoke simultaneously, each speaker’s utterance was counted as a turn. Across the entire data set (for SM and her partner and the 5 healthy comparison pairs) a total of 5,960 interactional turns were coded (SM and her partner = 1,270 interaction turns; comparison pairs = 4,690, M = 938.0, SD = 260.8).

Words were broadly defined with little emphasis placed on morphological or syntactic form. Consistent with our previous work (see Duff et al., 2008), in order to capture all aspects of the discourse, including verbal effort, each word in a false start was counted (e.g., the tail tha-, the tail is pointing up = 8 words), fillers (i.e., uh, um) were counted as words (e.g., um they both = 3 words), contractions were counted as one word (e.g., can’t = 1 word), and verbal back-channel or continuer responses (i.e., uh huh, yeah, mhm) were each counted as one word (e.g., uh huh = 1 word). Across the entire data set (for SM and her partner and the 5 healthy comparison pairs) a total of 31,226 words were coded (SM and her partner = 8,178 words; comparison pairs = 23,048, M = 4,609.6, SD = 1,779.6).

Initial description word counts

One of the primary dependent variables in the current study is the initial description word count. We have used this measure in our previous work (Duff et al., 2006) and have found it to be valuable in characterizing the discourse, perspective, and learning of the individual target participants (SM and the matched healthy comparison participants) prior to any input from the their familiar partner (the matcher). The initial description is defined as a director’s first attempt at describing each of the 12 cards, and includes their entire turn, before the matcher provides any input (verbal or nonverbal). These initial descriptions were marked in the transcripts and were then edited to remove any words that did not directly relate to the referencing of the individuals cards such as task management (e.g., the next one is; wait, what number are we on), mazing (e.g., exact repetition, abandoned phrases, fillers), and discourse makers (e.g., okay, alright) (see Duff et al., 2006 Supplementary Methods for a complete list). The remaining words in the initial description were then tallied.

Reliability of coding communicative resources

All coding for this data set was completed by the first author, and by research assistants who were blind to the goals and hypotheses of the study. Reliability ratings were obtained for approximately 12% of the trials (three trials randomly selected per pair). For the total word count, inter- and intra-rater reliabilities were 99% and 98%, respectively. For interactional turns, inter- and intra-rater reliabilities were 99% and 99%, respectively. Finally, for the initial description word count, inter- and intra-rater reliabilities were 97% and 98%, respectively.

Results

Learning and the development of common ground

Accuracy

SM and her partner performed similarly to comparison pairs in terms of card placement accuracy. On the first trial, SM and her partner placed 9 of the 12 cards accurately (comparisons’ average=8±2.7). By Trial 2 or 3, SM and the comparison group had essentially reached ceiling on accuracy, which is a common finding in this task (e.g., Duff et al., 2006; Hupet, Chantraine, & Neff, 1993).

Time to completion

Across trials, both SM and her partner and comparison pairs showed decreases in the amount of time required to complete each trial (Figure 2a). Strikingly, however, SM and her partner took more than three times as long to complete Trial 1 (34:56 (min:sec)) as comparison pairs (9:45±4:18 (min:sec)). After transforming the slope of SM’s regression line to z-scores, we found a significant difference between the slope of SM’s line and the mean slope of the comparison participants in the first block of the task, between Trials 1 and 6 (z=6.55; p<.0001).

Figure 2.

SM shows impairments in the ability to acquire and use common ground. (a) Time required to complete each trial. (b) Total number of words used in the initial descriptions per trial (both shown with session-by-session linear trends). (Error bars represent the standard error of the mean.)

Initial description word count

The initial description word count variable also revealed striking differences between SM and comparison participants. First, SM did not show the typical decrease in the total number of words used in the initial description of the cards across trials (SM Trial 1 = 100 words, Trial 24 = 82 words; comparisons’ Trial 1=177.6±57.1 words, on average, Trial 24 = 36.2±11.7 words, on average; Figure 2b). SM uses fewer words in her initial description of the cards on the first trial, and does not show the same rate of reduction in words, resulting in a significantly different slope than comparison directors between Trials 1 and 6 (z=2.92; p<.001). An example of this phenomenon is shown in Table 1. SM shows little reduction in the number of words used in the description across trials (Trial 1=10 words, Trial 24=7 words), while the comparison participant shows a much greater reduction in words (Trial 1=12 words, Trial 24=2 words). The example also illustrates how, during the final trials, labels produced by comparison participants become extremely concise and use only the words that are critical for uniquely referencing the tangram card.

Table 1.

Example of edited initial descriptions for Card 5 in Fig. 1 by SM and a comparison director across trials.

Trial	SM	Comparison
1	a man or a person- a shape of a body	box head like reading a book almost like he’s kinda kneeling
2	a guy, the other praying guy	it’s another guy praying but he is upright more and no triangle coming off the box
3	the other praying guy	the man kneeling with the disconnected box head
4	that other praying guy	man praying head attached
5	the other praying guy	man kneeling facing the right head attached
6	the praying guy with a- not with the heel, but not with the head attached	man kneeling detached box head
7	a guy prayin’ with the head not attached	man kneeling to detached block head
8	the praying guy with the head	man kneeling box head disconnected
9	the praying guy with the head unattached	man kneeling disconnected head
10	the praying guy with the head unattached	kneeling man head attached
11	the praying guy with the head unattached	man kneeling box head detached
12	the praying guy with the head unattached	detached head
13	the praying guy with the head unattached	the other head disconnected
14	the praying guy with the head unattached	kneeler disconnected box head
15	the praying guy with the head unattached	kneeling disconnected head
16	the praying guy with the head un attached	kneeling disconnected head
17	the praying guy with the head unattached	kneeling man disconnected head
18	the praying guy with the head un attached	detached kneeler
19	the praying guy with the head un attached	disconnected kneeler
20	a praying guy with the head un attached	disconnected kneeler
21	the praying guy with the head un attached	disconnected kneeler
22	the praying guy with the head un attached	disconnected kneeler
23	the praying guy with the head un attached	disconnected kneeler
24	the praying guy with the head un attached	disconnected kneeler

Follow up Analyses

In order to better understand the differences between the pairs in their pattern of performance (e.g., the large discrepancies in Trial 1, and the smaller but striking differences across trials) detailed follow-up analyses of the participants’ performances were conducted. Given the literature on the role of the amygdala in perspective-taking and the importance of perspective-taking in successfully performing the collaborative referencing task, the follow-up analyses place special emphasis on this ability.

Content of the descriptions

When a director makes the first attempt at referencing a card, the director creates a description that he/she (presumably) believes will be understood by the matcher. We analyzed the semantic content of the initial descriptions of Trial 1, which correspond to the first time the directors see the cards. Due to the nature of the tangrams (see Fig. 1), we and others (Clark & Wilkes-Gibbs, 1986) have noted that participants tend to describe the cards using biological characteristics (e.g., words such as head, body, man). (Exceptions are cards 9 and 10 in Figure 1, and these were excluded from the current analyses as they were consistently described with non-biological references (e.g., house, barn) by all pairs.) During Trial 1, we found that while comparison directors used biological references for the majority of their descriptions (62.0 ± 8.3%), SM did not. In fact, she described the majority of the cards in strikingly non-biological terms, using more geometric referents (e.g., A boat, it’s got a like a triangle at the end and it’s got a square.), and only described 40% with biological referents (z=2.62; p<.004). SM’s use of biological referents is more than one standard deviation below that of the comparisons’ mean. Also, certain descriptions provided by SM were highly unusual in their visuospatial perspective. For example, SM referred to Card 1 in Fig. 1 as a car hood throughout the task, while the majority of comparison subjects referred to it as a man kicking. As the task continued, on Trial 24, comparison directors continue to use more references with biological characteristics (76.0±5.4%) than SM, who only describes 60% of the cards with biological characteristics (z=2.92; p<.002). Overall, the lack of biological characteristics in her descriptions is consistent with previous research reporting impairments in SM’s ability to spontaneously anthropomorphize, especially in regard to geometrically shaped objects that, although they can be accurately described in non-biologically oriented language, tend to be assigned “human” characteristics under conditions of goal-directed movement (Heberlein & Adolphs, 2004).

Also, across trials SM’s descriptions were inflexible in their content with little variability in the words used to describe the cards. In contrast, while comparison participant’s labels retained the essence of the semantic content across trials, their references were quite variable in terms of the exact words used. For example, as illustrated in Table 1, SM describes Card 5 in Fig. 1 using the words the praying guy with the head unattached from Trials 9–24, while a comparison participant’s descriptions varied across trials as the pair settled on the most compact reference that included the most salient features: man kneeling box head detached; kneeling man disconnected head; detached kneeler; disconnected kneeler. Comparison participants presumably utilize the mutual understanding of the shared perspective, and recognize that the literal words are not as critical as the stability of the perspective across trials.

Refashioning

In the collaborative referencing task there is much conversational back-and-forth, as participants work to converge on a shared perspective for each of the stimuli. Clark & Wilkes-Gibbs (1986) have outlined this process. First, the director provides an initial description that he/she assumes will be effective. The listener then accepts or rejects this initial description; if rejected the speaker must “refashion” their description by either expanding the description or abandoning it and replacing the description with a new perspective. This “acceptance cycle” then continues until there is mutual acceptance of a description.

Attempts at refashioning were recorded by analyzing the interaction for expansions and repairs of the initial descriptions and each attempt by the director at a new description was tallied. Importantly, attempts at refashioning were analyzed only for directors. Following Clark (1992), offers of a new description by the matcher are classified as “replacements” (e.g., Matcher: Does it look like a guy with a triangle on his head?).

Overall, SM’s initial descriptions on Trial 1 tend to be too short and lacking sufficient information and detail for acceptance by her partner. As a consequence, SM had to make twice as many refashioning attempts (46 attempts) as comparison directors (22.6±12.3 attempts, on average) for the set of 12 cards (z=2.05; p<.02). In fact, for SM, five out of twelve cards required six or more attempts at refashioning, while across the entire data set for all five comparison participants (a total of 60 card descriptions) there were only three instances where comparison participants used six or more attempts at refashioning. Moreover, SM’s attempts at refashioning were highly repetitive and did not add new information, suggesting that she had difficultly spontaneously adopting a new perspective for the card. However, if her partner offered a suggestion, or a replacement, she could typically adopt this visuospatial perspective. For example, when her partner suggests the replacement, Does it look like rabbit ears? for Card 3 in Fig. 1, SM is able to use this perspective in later trials (e.g., the guy with the rabbit ears).

Partner monitoring through eye gaze

The development and use of common ground requires perceiving and understanding the thoughts, feelings, and knowledge of others. During collaboration it is critical for speakers to perceive and recognize the listeners’ level of understanding, and adjust further output based on this, either by being more descriptive in cases of confusion (e.g., refashioning), or by taking advantage of mutual understanding to be more economical in communication. Previous research has shown that one way speakers monitor their partners for understanding is by looking at their faces (as well as monitoring vocal tone and gestures; Clark & Krych, 2004). One modification to Clark’s original collaborative referencing task in our set-up is the use of a partial barrier, which allows partners to see each other and utilize gaze as a form of partner monitoring and management. Since it has been shown that forms of partner monitoring can improve the accuracy and efficiency of collaborative task performance (Clark & Krych, 2004), we analyzed SM’s ability to monitor her partner though gaze at several time points across the task (Trials 1, 12 and 24) to better understand the factors contributing to her impaired performance.

Based on analysis of the videotaped interaction, the timing and duration of gaze directed at the face of the partner were noted on the transcript in relationship to the conversation and turn boundaries, or instances when a change in speaker occurred. Following procedures outlined by Turkstra, Brehm, and Montgomery (2006) eye gaze was determined for directors by viewing the video and recording the number of frames for which the director was looking at the conversational partner. Each frame corresponds to approximately 33 msec. Gaze analysis was performed for the first, twelfth, and twenty-fourth trials for SM and four comparison participants. Gaze analysis could not be performed reliably on one comparison participant due to interference from eyeglasses, thus this participant’s data were excluded from these analyses. For all other participants, reliability ratings for the duration of gaze were obtained for approximately 12% of the data. Inter- and intra-rater reliabilities were 98% and 97%, respectively.

Consistent with previous research showing that SM has normal overall gaze to faces during conversations (Spezio et al., 2007), we found that SM was well within the normal variation of percentage of time spent looking at her partner (13%, comparisons’=16.7±11.4%; z=0.23; p=.41) on Trial 1. However, we found that healthy directors tended to look at their partner at specific times during the task, especially during or after the directors’ initiating description. Comparison directors monitor their partner’s level of comprehension by looking at their partner during or after 58.1±24.1% of the initiating descriptions during Trial 1. However, SM only looks at her partner during or after 16% of the initiating descriptions (more than 1 SD below the comparisons’ mean) in Trial 1 (z=1.71; p<.05). Due to the nature of the task, where the director must be looking at the cards as well, it seems as though when they look at their partner is more critical than how much they look at their partner.

Later in the task, SM differs from comparison participants in both aspects of gaze: duration and timing. To illustrate this, take the contrast between Trial 1 and Trial 24. As detailed in Table 2, comparison participants do not decrease the total amount of time gazing (Trial 1=16.7±11.4%; Trial 24=22.3±11.3%) or the percentage of gazes during initial descriptions (Trial 1=58.1±24.1%; Trial 24=51.5±27.7%). However, SM shows striking decreases both for the amount of time (Trial 1=13.5%; Trial 24=3.0%) and for the percentage of gazes during initial descriptions (Trial 1=16.7%; Trial 24=8.3%).

Table 2.

SM displays impaired timing of gaze during social interaction, but is within normal limits in terms of overall time spent looking at her partner. While comparison participants direct their gaze towards their partner during or after the majority of the initial descriptions, SM is impaired at monitoring her partner during these epochs.

		Trial 1			Trial 24
Percentage of total time spent looking at face of partner during the trial	SM Normal comparison (n=5)	13.5% 16.7±11.4%		z=0.23; p=.41	3.0% 22.3±11.3%		z=1.69; p<.05
Percentage of initial descriptions where participant looks at partner during or after intial description	SM Normal comparison (n=5)	16.7% 58.1 ±24.1%		z=1.71; p<.05	8.3% 51.5±27.7%		z=1.55; p=.059

IQ-matched brain-damaged comparison participants

It is possible that IQ can contribute to performance on the collaborative referencing task, and SM and the five healthy comparison participants were not perfectly matched on IQ. To address this issue further, we collected additional data, using an identical protocol, from two brain-damaged comparison participants (BDC participants) and their familiar communication partners. BDC-1 was a 55-year-old female, with 12 years of education and a WAIS-III FSIQ of 86 with damage to the lateral aspects of the left posterior middle frontal gyrus, ventral postcentral gyrus, and lateral occipital polar region, based on MRI data; she did not have any amygdala or hippocampal damage. BDC-2 was a 57-year-old female, with 12 years of education and a WAIS-III FSIQ of 85 with damage to the dorsal aspects of the right central and postcentral gyri, based on MRI data; she did not have any amygdala or hippocampal damage. Thus, both of these participants are perfectly matched to SM on IQ (recall that SM has a FSIQ of 88), and also matched to SM on education and sex (as well as being in the same general age bracket). Both of these participants had no impairments on standardized tests of memory, including working memory, speech, language, and visuospatial processing. During the first trial, unlike SM and her partner, the BDC pairs performed within the normal range for both of the main dependent variables—the amount of time to complete the trial and number of words in the initial descriptions. Across trials, the BDC pairs displayed significant reductions in both time (on average Trial 1: 9:25 (min:sec); Trial 24: 40.5 sec) and, in striking contrast to SM, initial description word count (on average Trial 1: 255.5 words; Trial 24: 56 words). Moreover, unlike SM and her partner, the BDC pairs have a similar rate of reduction in both measures as healthy comparison participants in Session 1 (BDC pairs’ slope compared to the healthy comparisons; time: t=1.29; p=.25; words: t=1.93; p=0.11). For example, in the first trial, the BDC-1’s description for card 2 in Fig. 1 was “a big round barrel, but it’s got a slit in the one side and it prongs out on the top and then it’s got a diamond right in the center of the top,” which was reduced to simply “the barrel” in the final trial. Additionally, and also in contrast to SM and her partner, our follow up analyses revealed no significant differences between the BDC pairs and the healthy comparison participants. The BDC participants were within the same range as healthy comparison participants in the use of biological references both on the first trial (on average 65%; t=0.44; p=.68;) and final trial (on average 70%; t=1.4; p=.20), and on the number of refashioning attempts (on average 6.5 total attempts across the entire first trial; t=1.7; p=.14). Gaze analysis was performed on BDC-1 (gaze analysis could not be performed on BDC-2 due to technical difficulties related to camera angle). BDC-1 performed normally on both aspects of the gaze analysis. Specifically, the BDC participant looked at her partner during or after 75% of the initial descriptions on the first trial (z=0.69; p=.24), and 41.6% of the initial descriptions on the final trial (z=.35; p=.36). The percentage of time spent looking towards her partner on the first trial was 26.5% (z=0.91; p=.17), while on the final trial it was 10.5% (z=1.03; p=.15). In sum, these IQ-matched BDC participants performed within normal limits on all of our major dependent variables, in sharp contrast to SM’s defective performances on these same measures. Along with the previously reported data from the healthy comparison participants, this contrast—using two perfectly IQ–matched brain-damaged patients—helps to eliminate concerns that the findings in SM could be explained by IQ factors.

Discussion

The goal of the current study was to understand the role of the amygdala in supporting social communication and the development of common ground in a dynamic collaborative setting. While SM and her partner were able to arrive at labels for each card that were used consistently throughout the task and there was a reduction in the amount of time to complete each trial) and their card placement accuracy was indistinguishable from healthy comparisons, SM displayed a range of deficits that significantly disrupted the development and use of common ground in her interactions with her partner compared to the healthy pairs. While healthy comparison pairs, as well as an IQ-matched brain damaged comparison participant pair, showed robust decreases in the amount of time and number of initial description words needed to describe the cards across trials, SM and her partner exhibited impairments on both measures. These results are in stark contrast to our previous work with patients with bilateral hippocampal damage, who, despite severe declarative memory impairments, showed rates of learning, as measured by both time and the number of words in the initial description, that did not differ from comparison pairs (Duff et al., 2006). Despite SM’s normal declarative memory, the nature and severity of her social and emotional deficits appear to impair her ability to demonstrate normal learning in this collaborative and socially interactive task.

Perhaps the most fundamental measure of learning and acquisition of common ground is the reduction of communicative resources (i.e., words in the initial description) used to reference the cards across trials. On this measure, and in contrast to healthy comparison participants, the IQ-matched brain-damaged comparison participant, and even patients with hippocampal amnesia, who all demonstrated striking reductions in the number of words used across trials to reference the 12 cards, SM showed almost no reduction (see Fig 1). In fact, her references are noteworthy for their rigidity across trials. One possible explanation for this rigidity is that SM is over-relying on her intact declarative memory for which reference was previously successful and does not draw on the socially or emotionally salient aspects of the interaction to further shape the reference. While speculative, we wonder if this is in some way related to known deficits in the modulation of emotional memory following amygdala damage (cf. Adolphs et al., 1997). That is, across each trial, comparison pairs settle on simplified and concise references that reflect the most salient features of the stimuli but also the salient features of their interactions around the cards. Many of the shared perspectives and labels generated in comparison pair sessions were the result of highly social and emotional communication associated with playful teasing, mock anger, and eureka moments when they both “saw” the same figure in the cards (e.g., Oh! That’s your bunny) (see Duff et al., 2008). Not only was this type of communication almost entirely absent from SM’s interactions with her partner, but even when it was present, SM did not draw on and use these interactions to shape her references like comparison pairs did. While the majority of work on the amygdala’s role in emotional memory modulation has focused on enhancements in fact-based and episodic memory paradigms, its role in shaping everyday interactions and social and emotional communication also warrants investigation.

The ability to build common ground requires that speakers update their mental representation of another’s mind over time in order to build on shared knowledge to develop richer common ground for more rapid and economical communication (e.g., shorter labels across time) (Krauss & Fussell, 1996). While it has been shown that the development of an initial utterance may be egocentric, and may not fully include the listener’s perspective (Horton & Keysar, 1996; Keysar, Barr, & Horton, 1998), research has suggested that in a collaborative setting (i.e., where there are no experimental restrictions on the form and frequency of the partner’s communication), speakers adjust and fine-tune their utterances and mental representations to incorporate the listener’s perspective through the feedback they receive about the listener’s state of comprehension (Krauss & Fussell, 1991; 1996). In fact, in the collaborative referencing task, when directors do not receive feedback from the matcher about their state of comprehension, either when they cannot hear the matcher, or when the matcher is not co-present, directors do not show significant decreases in the number of words used to describe the stimuli across trials (Hupet & Chantraine, 1992; Krauss & Weinheimer, 1966). In other words, the robust decreases in the number of words used to describe the stimuli displayed by healthy participants is not simply an effect of explicit memory (Duff et al., 2006) or repetition (Hupet & Chantraine, 1992), but only occurs during social interaction when speakers are able to utilize listener feedback to update their common ground (Hupet & Chantraine, 1992; Krauss & Weinheimer, 1966). In accord with this, SM shows little to no reduction in the number of words used in the initial description across trials (see Fig. 2b), and in this sense her performance is not unlike healthy participants who perform this task without a partner or social interaction. SM receives feedback from her partner, but does not seem to update her representation of her partner’s level of understanding. Recent evidence from our laboratory suggests that SM is impaired at assigning affective value to information to properly update one’s judgment of another person (Croft et al., 2009). Taken together, these findings raise the possibility that SM is unable to place sufficient weight upon the feedback that she is receiving from her partner to affect her future output and shorten the labels.

Previous research has shown that the amygdala is critical for recognizing and understanding certain social signals (e.g., facial expressions) (Adolphs et al., 2002; Adolphs et al., 1998). During social interaction, participants utilize certain subtle communicative acts to represent their state of mind. For example, feedback about one’s level of understanding is not only conveyed through verbal responses (e.g., okay, got it), rather, feedback, especially about lack of understanding, can be conveyed through silence (reviewed in Krauss, Fussell & Chen, 1995) and nonverbal signals such as facial expressions (Bavelas & Chovil, 2006; Brunner, 1979; Chovil, 1991). SM is impaired at recognizing facial expressions of complex mental states and “social” emotions (e.g., boredom, embarrassment) from static photographs (Adolphs et al., 2002). The necessity of the amygdala for recognizing basic prosodic cues (e.g., basic emotions) has provided mixed results (e.g., Adolphs & Tranel, 1999; Scott et al., 1997), it remains to be seen if SM is able to recognize and use more subtle prosodic indicators which would be important for this task (e.g., frustration, confusion) for updating her knowledge of another’s state of mind and using this information to make interactional and communicative decisions. Further research is required to understand if impaired recognition of some of the more subtle communicative nonverbal aspects of naturalistic social interaction is contributing to the impairment in developing common ground seen here.

Efficient communication during social interaction requires verbal and nonverbal actions of interlocutors to be coordinated (for review see Bernieri & Rosenthal, 1991). This allows for accurate perception and understanding of the other person’s acts, as well as enhances rapport and feelings of cohesion during the interaction (Chartrand & Bargh, 1999; Hatfield, Cacioppo, & Rapson, 1994). For example, shifts in gaze are coordinated with the timing of speech (Kendon, 1967). In the collaborative referencing task, this can be seen as comparison participants direct their gaze towards their partner during or after the majority of the initial descriptions, which serves as an important mechanism for monitoring the actions of their partner. SM does not monitor her partner for understanding through gaze at critical moments as comparison participants do, thus, potentially missing critical nonverbal feedback (e.g., raised eyebrows or other looks of confusion) information to update her mental representation and guide future output. Interestingly, even on the last trial, comparison directors continue to coordinate and time their gaze towards their partner. While most all of the collaborative work to develop the shared perspective and reference is complete by this stage of the task, comparison pairs continue to appropriately time their eye gaze as a form of task management (e.g., looking at partner for confirmation that the card has been placed before they begin describing the next card) and to support the ongoing social interaction, as when pairs continue to tease each other or play games to make the task more fun or go faster (see Duff, Hengst, Tranel, & Cohen, 2009). In contrast, SM only looks at her partner once during the entire last trial, at the end while she is waiting for her partner to place the final card. This suggests that SM may have impairments in coordinating gaze to speech at salient times during the interaction.

Previous work has documented SM’s deficit in directing gaze towards the eyes of her conversational partner (Spezio et al., 2007). Is it possible that the impairments observed in our collaborative referencing task are due to deficits in direct eye contact? We do not believe that the impairments displayed by SM can be explained by this factor, for one simple reason: direct eye contact is not necessary to perform normally on this task. Previous research with this task (e.g., Clark & Wilkes-Gibbs, 1986) has used a complete barrier that does not allow participants to see one another. Despite having no direct eye contact throughout the task, these participants still show normal rates of reduction in words and time similar to those of the healthy and BDC pairs reported here. However, our set up involves a partial barrier allowing participants to see one another and permitting a wider variety of non-verbal communication. The use and timing of these nonverbal methods of communication (e.g., looks of confusion, nods) become important for coordinating communicative activity. That is, it is important for participants to look towards their partner at the correct time to utilize these cues. On such measures, SM is impaired, and this potentially contributes to her overall impaired performance on this task. But impaired eye contact, per se, would not explain her impaired performance.

Common ground is not only important for efficient communication, but is also critical for strengthening social bonds and developing interpersonal relationships (Enfield & Levinson, 2006). These findings have important implications for the social abilities of patients with amygdala damage. While SM has been previously described has having impairments in certain facets of social interaction (e.g., recognition of emotional expressions), she has always been described has having somewhat normal, albeit disinhibited, real-world social interaction abilities (Adolphs & Tranel, 2000). However, this study suggests that perhaps some her deficits in facets of social interaction may have effects on her overall ability to communicate socially and develop common ground. Anecdotally, for example, while having many acquaintances, SM lacks close relationships. While there may be other factors involved in her inability to gain close companionship, perhaps deficits in the ability to build and use common ground for creating and maintaining interpersonal bonds play a role.

The amygdala has been implicated in complex mentalizing tasks requiring perspective-taking and theory of mind (Heberlein & Adolphs, 2004; Shaw et al., 2004; Stone et al., 2003). However, this is the first study to look at the necessity of the amygdala for processes such as perspective-taking during social interaction in a naturalistic context, with the demands of real-time online processing. These results suggest that the amygdala is important for taking another’s perspective to understand and appreciate the thoughts and knowledge of others. Importantly, everyday social interaction and communication requires the integration and orchestration of multiple neural systems. The work presented here is a rich demonstration of the contribution of the amygdala to this network and of how perspective-taking is deployed in everyday social interaction to facilitate communication.

Limitations and Future Directions

One obvious limitation of this study is the fact that it is based on a single case of focal, bilateral amygdala damage. However, case studies, and especially studies of patient SM, have provided many important advances in the field of cognitive neuroscience by allowing us to specifically study the necessity of the amygdala for numerous cognitive processes. Previous research has suggested that developmental lesions to the amygdala may impair perspective-taking and theory of mind more so than adult-onset lesions (Shaw et al., 2004), and SM’s lesion is presumed to have occurred during adolescence or possibly even childhood (Adolphs & Tranel, 2000), which raises interesting questions regarding the necessity of developmental amygdala lesions to produce the kind of perspective-taking deficits observed in the current study. Finally, because social interaction is so complex, it is difficult to parse out the underlying factors for many of the deficits observed here. Therefore, it would be beneficial for future studies to examine specific components of social interaction suggested to be impaired here, such as recognition of subtle nonverbal social signals (e.g., expressions or vocal tones conveying frustration), and coordination of social interactions (e.g., gaze timing). Future studies could also aim to understand the interactions of multiple neural structures for everyday social interaction and communication, for example by examining patients with brain damage to multiple cognitive systems, or disorders characterized by social impairments, such as autism or traumatic brain injury.