Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 May 10.
Published in final edited form as: Neuropsychologia. 2018 Jun 19;117:332–338. doi: 10.1016/j.neuropsychologia.2018.06.012

Patients with hippocampal amnesia successfully integrate gesture and speech

Caitlin Hilverman 1, Sharice Clough 2, Melissa C Duff 1, Susan Wagner Cook 3,4
PMCID: PMC6510398  NIHMSID: NIHMS978766  PMID: 29932960

Abstract

During conversation, people integrate information from co-speech hand gestures with information in spoken language. For example, after hearing the sentence, “A piece of the log flew up and hit Carl in the face” while viewing a gesture directed at the nose, people tend to later report that the log hit Carl in the nose (information only in gesture) rather than in the face (information in speech). The cognitive and neural mechanisms that support the integration of gesture with speech are unclear. One possibility is that the hippocampus – known for its role in relational memory and information integration – is necessary for integrating gesture and speech. To test this possibility, we examined how patients with hippocampal amnesia and healthy and brain-damaged comparison participants integrate information from gesture with speech in a narrative retelling task. Participants watched videos of an experimenter telling narratives that included hand gestures that contained supplementary information. Participants were asked to retell the narratives and their spoken retellings were assessed for the presence of information from gesture. Although patients with amnesia reported fewer narrative features overall, their retellings included information that was present only in gesture, revealing that they had integrated information from gesture with speech. Interestingly, patients with amnesia were significantly more likely to express content from gesture in their spoken retellings than comparison groups. Thus, a functioning hippocampus is not necessary for gesture-speech integration. Furthermore, providing unique information in gesture may enhance communication for individuals with declarative memory impairment, possibly via non-declarative memory mechanisms.

Introduction

Speakers’ gestures can communicate information to their listeners. Listeners extract and integrate information from the gestures that they view, even when the information in gesture is distinct from information in spoken language. For example, a speaker might say “A piece of the log flew up and hit Carl in the face” while gesturing to the nose. In this example, the speaker has provided unique or supplemental information in gesture, about the specific location affected. Listeners are sensitive to this sort of supplemental information in gesture. Participants who view a video of an experimenter telling a narrative that contains gestures providing such supplemental information subsequently incorporate the supplementary information from gesture into their speech (e.g., reporting, “A log hit him in the nose”) (Cassell, McNeill, & McCullough, 1998). These findings reveal that speakers’ gestures are not simply maintained in a motoric or gestural form. Instead, listeners integrate information from gesture into their semantic representations of the narrative, and this information influences the words that listeners subsequently use to retell the narrative. Here we ask: what memory systems do listeners use to create representations integrating information from gesture with speech? We investigate this question in three groups of participants – patients with hippocampal amnesia and healthy and brain-damaged comparison groups – to test whether and how the hippocampus contributes to speech and gesture integration.

The hippocampus may have a role in gesture integration due to its role in relational (or associative) memory binding (Davachi, 2006; Eichenbaum & Cohen, 2001; Ryan, Althoff, Whitlow, & Cohen, 2000). The hippocampus supports relational flexibility that allows for the encoding of co-occurrences of people, places, and things and the spatial, temporal, and interactional relations among them (see Konkel & Cohen, 2009). Furthermore, the hippocampus supports the reconstruction and recombination of information, allowing information to be used in novel contexts and situations (Eichenbaum & Cohen, 2001). Representations created by the hippocampus appear akin to the multimodal representations created by co-occurring speech and gesture. Indeed, patients with hippocampal amnesia – and declarative memory impairment – are impaired in their ability to encode, retrieve, and imagine complex, multimodal representations: they provide fewer episodic details than healthy comparison participants in their narratives about past and future events (Kurczek et al., 2015; Race, Keane, & Verfaellie, 2011) and they gesture less than healthy comparison participants when describing experiences from their remote past (Hilverman, Cook, & Duff, 2016). Thus, it is possible that the hippocampus is responsible for binding information in gesture together with information in speech. If so, hippocampal damage would be expected to disrupt this binding.

The hippocampus also has a clearly established role in memory integration. Memory integration is the process by which new memories are integrated with existing memories in the brain, and is thought to occur via recruitment of overlapping neural areas (Schlichting & Preston, 2015a). For example, if you encounter a dog being walked by a woman in the park and then later in the week encounter that same dog being walked by a man on your street, the presence of the dog would trigger the reactivation of the previous memory, and your prior memory would be integrated with the new experience (i.e., the man and women co-owners of the dog). The ability to integrate memories into interconnected representations has a multitude of behavioral ramifications; interconnected representations are used to infer relationships, navigate through space, make decisions, and create and imagine events and experiences (Schlichting & Preston, 2015). The hippocampal-medial prefrontal circuit has been implicated in integration tasks in which participants learned new associations related to previously learned stimuli during fMRI imaging (Schlichting, Mumford, & Preston, 2015). Given this role of the hippocampus in the integration of memories, the hippocampus might also be involved in integrating information in gesture with information in spoken language.

Despite the contribution of the hippocampus to both memory integration and relational memory, previous work suggests that gesture may be processed outside of the hippocampus and medial temporal lobe. Producing gesture while learning new words enhances word learning in people with hippocampal amnesia, who are severely impaired at this ability (Hilverman, Cook, & Duff, in revision). Similarly, hand gestures produced by patients with amnesia reflect prior experiences (Hilverman, Duff, & Cook, in prep). Conversely, patients with Parkinson’s Disease – which affects non-declarative or procedural memory – do not produce gestures that reflect their prior experiences (Klooster, Cook, Uc, & Duff, 2015). The fact that gesture can support memory and learning even in the absence of a functioning hippocampus leaves open the possibility that memory mechanisms beyond the hippocampal declarative memory system may support speech and gesture processing and integration.

To test these alternatives, we had patients with amnesia and comparison groups complete a narrative retell task. Participants watched short videos of an experimenter telling a narrative. In the video, the experimenter produced gestures, including gestures that were supplementary to the information in speech (e.g., gesturing a punching motion with the word “hit”). Immediately afterwards, participants retold the narrative. If the hippocampus supports the integration of speech and gesture, the information from supplemental gesture should be absent from the immediate spoken retellings of patients with hippocampal amnesia. Alternatively, if areas outside of the hippocampus and medial temporal lobe support integration of speech and gesture, the information from gesture should be present in the speech retelling of all participants, including patients with hippocampal amnesia.

Methods

Participants

Participants included 4 (one female) patients with bilateral hippocampal damage and severe declarative memory impairment (HC), 4 (three female), brain-damaged comparison (BDC) patients with damage outside of the medial temporal lobe (MTL) (bilateral ventromedial prefrontal cortex [vmPFC] and no declarative memory impairment), and 19 (8 female) healthy comparison (NC) participants that were matched to both patient groups on age, handedness, sex, and years of education. All patients in the HC and BDC groups have non-progressive lesions. Matching of patients and comparison participants increases statistical power to detect differences across groups, as our sample size and number of trials was necessarily small due to the rare nature of the population of people with amnesia.

For the HC group, three patients experienced anoxic/hypoxic episodes (1846, 2363, 2563) resulting in bilateral hippocampal damage and the fourth contracted herpes simplex encephalitis (1951) leading to more extensive bilateral MTL damage affecting the hippocampus, amygdala, and surrounding cortices (Figure 1). Structural MRI examinations completed on 3 of the 4 patients confirmed bilateral hippocampal damage and volumetric analyses revealed significantly reduced hippocampal volumes. Participant 2563 wears a pacemaker and is unable to undergo MRI examination; damage to hippocampus was confirmed by computerized tomography. For the three anoxic patients, there is no damage to the lateral temporal lobes or anterior temporal lobes.

Figure 1.

Figure 1

Open in a new tab

Magnetic resonance scans of hippocampal patients. Images are coronal slices through four points along the hippocampus from T1-weighed scans. Volume changes can be noted in the hippocampal region for patients 1846 and 2363 and significant bilateral MTL damage including the hippocampus can be noted in patient 1951. R = right, L = left, A = anterior, P = posterior, NC = healthy comparison brain.

Tests of neuropsychological functioning revealed a severe and selective impairment in declarative memory (M =57.9; Wechsler Memory Scale-III General Memory Index) while measures of verbal IQ, vocabulary, and semantic knowledge were within the normal range as measured by standardized tests (Appendix A). Patients also perform normally on experimental measures of non-declarative or procedural memory (Cavaco et al., 2011).

The BDC group provides evidence as to whether any observed deficits in the performance of patients with amnesia are due specifically to hippocampal damage or arise in association with brain damage more generally. BDC participants all had bilateral damage to the ventromedial prefrontal cortex. Like the participants with hippocampal amnesia, the BDC group performed in the normal range on neuropsychological tests of intelligence and language, were free of aphasia, and had no motor impairments that prevented them from gesturing. In critical contrast to the participants with hippocampal amnesia, the BDC group had no lesions in the medial temporal lobe and performed within normal limits on standardized tests of declarative memory (Appendix A).

Non-brain damaged healthy comparison participants (NC) included 19 individuals without any neurological or psychiatric disease that were individually matched to each of the HC and BDC participants on sex, age, handedness, and education.

Stimuli

An adult native English speaker was videotaped narrating four stories about a cartoon man named Carl who experienced a variety of unfortunate events (Appendix B). Each narration was about 30 seconds long, consisted of six sentences, and contained two gestures containing supplementary information about the features being told in speech. Each narrative also had two additional gestures present at other points of the narrative that did not contain supplemental information (e.g., gesturing a big eye with the words “big googly eye”), in order to make the presence of gesture more natural. Two videos were created for each narrative with the supplemental information varying across the videos; this design controls for the possibility that participants might spontaneously generate the information from the supplemental gesture from the spoken narrative. For example, in the Frankenstein narrative, when hearing the sentence “He got a flower to give to the girl,” one video showed the speaker making a flower-picking gesture with the word got, while the other showed a cutting gesture (Figure 2). The two non- supplementary gestures were identical in both video recordings of each narrative.

Figure 2.

Figure 2

Open in a new tab

Experimenter in the video producing a supplementary gesture pair. Participants saw either the picking gesture (left) or the snipping gesture (right).

Procedure

Participants were part of a larger study involving three visits to the laboratory, each four weeks apart. Data presented here were collected on the first visit. Each participant viewed one version of each of the four stories on a laptop screen. While the video was played, a picture corresponding to that video’s narrative was also present on the screen. The picture depicted a scene of the narrative being told. Immediately after each video ended, the video disappeared leaving only the picture on the screen (see Appendix B). Participants were then prompted to retell what happened in that particular narrative with the picture on the screen remaining as a cue.

Coding

To examine whether supplementary information in gesture was integrated into the participants’ retelling of the narrative, we focused on the spoken words that participants produced when retelling the specific portions of the stories that had been described with accompanying supplemental gesture. An independent coder who was blind to which supplemental gesture each participant had viewed assigned the spoken explanations of these elements to categories. The retellings were coded as (1) identical to speech – an exact replication of the word spoken in the video (e.g., saying “He searched for a recipe,” after having heard the word searched, (2) identical to gesture – a replication of what had been expressed in one of the gestures (e.g., saying “He looked on the computer for a recipe,” after having seen a typing gesture along with the spoken word searched), (3) related to gesture – a word semantically related to one of the gestures (e.g., saying “He looked up a recipe,” when they saw a typing gesture), and (4) other – the feature was mentioned but lacked the specificity to match either speech or one of the supplemental gestures (e.g., saying “He found a recipe,” when they saw a typing gesture and heard searched). Features were coded as identical to gesture or related to gesture when the coder could infer from the retellings that a supplemental gesture was observed. Features coded as related to gesture were relatively rare (see Table 1). After coding, we confirmed that all responses coded as identical to gesture or related to gesture indeed reflected the supplemental gesture that had been observed. See Appendix C for a compilation of the specific responses given for the features accompanied by supplementary gesture and how they were categorized.

Table 1.

Proportions of gestures produced by each group by category.

Group Identical to Gesture Related to Gesture Identical to Speech Other
Amnesic 0.53 (9/17) 0.12 (2/17) 0.24 (4/17) 0.12 (2/17)
Healthy 0.32 (47/149) 0.07 (11/149) 0.48 (72/149) 0.13 (19/149)
BDC 0.38 (12/31) 0.03 (1/31) 0.45 (14/31) 0.13 (4/31)
Open in a new tab

Results

We first analyzed the total number of features accompanied by supplementary gesture that participants retold in speech by group. With two supplementary gestures per narrative (4 narratives), there was a total of 8 possible features that could be expressed in speech in their retellings. The comparison groups performed well, with healthy comparison participants retelling 7.45 (SD = 0.69) and brain-damaged comparison participants retelling 7.75 (SD = 0.50) features, on average (Figure 3). Patients with amnesia retold only 4.25 (SD = 1.50) features, on average. The HSE patient retold 5 features, while the three anoxic patients retold 6, 3, and 3, respectively

Figure 3.

Figure 3

Open in a new tab

Mean number of gestured-with features retold by group, with each point representing a participant. The maximum number was 8. The HC group was significantly less likely to retell a narrative feature than the NC group.

We assessed the difference in likelihood of reporting a feature in speech with a logistic mixed effect regression model. We included a fixed effect for group with random by-subject and by-narrative intercepts. The healthy comparison group served as the reference group. As can be seen in the overall height of the bars in Figure 3, patients with amnesia were significantly less likely to retell features in speech than the healthy comparison group (B = −2.71, z = −5.23, p < .001). Brain-damaged comparison participants were not significantly different in their likelihood to retell features relative to the healthy comparison group (B = 0.85, z = 0.80, p = 0.42).

We next examined the categories assessed by our coding system to determine whether information from the gesture was present in the spoken retellings. Healthy and brain-damaged comparison groups produced features containing words that were identical or related to the information present in gesture 39% and 41% of the time, respectively, while patients with amnesia produced features containing words that were identical or related to the gesture 65% of the time that they recalled the feature (Table 1; this information is also visible in Figure 3).

To analyze this, we used a logistic mixed effect regression model that predicted the likelihood of a retold feature containing information that was present only in gesture (1) over information present in speech (0). There was a fixed effect for group (with healthy comparisons serving as the reference group) and random by-participant, by-feature, and by demographically-matched participant pair intercepts. Patients with amnesia were significantly more likely to include supplementary information from gesture in their retellings than healthy comparison participants (B = 1.29, z = 1.968, p = .05). This same effect was not significant for the brain-damaged vs. healthy comparison groups (B = 0.17, z = 0.35, p = .72).

Discussion

Patients with amnesia successfully integrated supplemental information from gesture into their representation of the narrative that they viewed. Despite retelling significantly fewer features than comparison groups overall, their speech was more likely to contain information that was present uniquely in gesture than healthy comparison participants. This suggests that, despite its role in both encoding and retrieving multimodal memory representations, the hippocampus is not required for the integration of information in gesture with information in spoken language over short timescales.

Given the role of the hippocampus in relational memory and integration, this finding is surprising. However, there are differences between integrating memories of events and integrating gesture with speech. Unlike memory event integration, speech and gesture overlap in content and in time; the onset of co-speech gesture tends to slightly precede the onset of the word or concept that it accompanies (McNeill, 1995; Schegloff, 1987), and the stroke of the gesture is concurrent with related speech (McNeill, 1992). Indeed, when gesture and spoken language are uncoupled temporally, listeners are less likely to construct a meaningful representation from them (Pruner & Cook, under review; Habets, Kita, Shao, Ozyurek, & Hagoort, 2011); presenting an informative gesture just 1 second later than would naturally be produced leads to less learning via gesture. The timing of the presentation of materials to be integrated also affects memory formation; events that occur closer in time (e.g., on same day) are better integrated than events that occur further apart in time (e.g., across days; Zeithamova & Preston, 2017). Here, as in naturally occurring language, speech and gesture were tightly coupled temporally. Our findings suggest that when gesture and speech co-occur in time, the hippocampus may not be necessary for the integration of this information in memory. Although patients with hippocampal amnesia integrated information present in gesture at higher rates than comparison participants, it remains an open question whether and how gesture would be integrated in amnesia if the information was temporally decoupled.

Future work should also explore if co-speech integration carries over to long-term memory. For example, patients with amnesia can successfully recall word pairs immediately after exposure but cannot maintain this pairing after a delay (Squire, 2017). Additionally, prior work on gesture-speech integration has found hippocampal activation in gesture-speech integration when the task involves a memory delay. In one such study, healthy participants viewed videos containing meaningful gestures, unrelated gestures, or no movements while they underwent neuroimaging (Straube, Green, Weis, Chatterjee, & Kircher, 2008). They later performed a recognition task. Hippocampal activation during encoding was correlated with subsequent performance on the recognition task in the condition that involved integration of meaningful gestures and speech. This suggests that the hippocampus may be involved in the encoding and retrieval of multimodal representations into long-term memory, while our data suggest that the hippocampus is not necessary for successful integration over short timescales.

In this study, the information in gesture was temporally and semantically related to the information in spoken language. Embedding spoken language in a semantically rich context may help support processing. For example, healthy people show benefits in maintaining information over short timescales – such as short-term serial recall of words – when presented in meaningful and familiar verbal contexts (full sentences) rather than in unfamiliar verbal contexts (lists) (Baddeley, Hitch, & Allen, 2009; Bor, Cumming, Scott, & Owen, 2004; Race, Palombo, Cadden, Burke, & Verfaellie, 2015). Similarly, patients with amnesia perform significantly better, and as well as comparison participants, on verbal paired associate learning when the pairings are based on semantic or phonological similarities between the two words but are unable to learn unrelated word pairs (Winocur & Weiskrantz, 1976). Patients with amnesia also demonstrate better memory for word lists when embedded in a meaningful narrative than when presented with just the word list alone (Kovner, Mattis, & Goldmeier, 1983). Gesture is a naturally-occurring coupling of visual and verbal information that is related in both time and meaning. Patients with amnesia appear to exploit this coupling of semantically related information to benefit their behavioral performance.

This finding is also perhaps less surprising when considering other work examining gesture in patients with amnesia. Both producing and perceiving hand gesture affects behavior and enhances behavioral performance in amnesia. When talking about how to complete a spatial-motor task called the Tower of Hanoi, patients with amnesia produce gestures that reflect their previous experiences with the task (Hilverman, Duff, & Cook, in prep). Furthermore, producing observed gestures at encoding enhances recognition performance for new word-object pairings after a delay (Hilverman, Cook, & Duff, in revision). The current work extends this finding to integration of information in gesture with speech. Future work should examine the extent to which gesture can be exploited as a possible compensatory strategy for people with memory impairment.

These data suggest that gesture can be useful in a language processing context. There are well-documented deficits in language use and processing in patients with amnesia (Duff & Brown-Schmidt, 2012; 2017). For example, patients with amnesia are impaired in their ability to interpret pronouns; relative to healthy comparison participants, they are less likely to identify the intended referent from a pronoun when listening to a sentence (Kurczek, Brown-Schmidt, & Duff, 2013). This deficit stems from their inability to maintain and integrate information over even very short timescales. The presence and integration of supplementary information in gesture appears to enhance the ability to maintain this information. It is therefore possible that the introduction of information in gesture could enhance other domains of language processing in this population.

The work reported here is a first step at identifying the mechanisms in memory that support the integration of gesture with spoken language. The hippocampus and surrounding medial temporal lobe do not appear to be required for the integration and creation of multimodal representations of information in speech and gesture over very short timescales.

Acknowledgments

Supported by CH (NIDCD F32-DC016580-01), SWC (NSF IIS-1217137, IES R305A130016), and MCD (NIDCD R01-DC011755).

Appendix A. Neuropsychological test results for HC and BDC groups

Subject Sex Age HD Ed Etiol Damage HC Volume WAIS III FSIQ WMS III GMI AVLT Delay CFT Delay WAIS -III Vocab WAIS -III Info BNT TT
1846 F 58 R 14 Anoxia Bilateral HC −4.23 84 57 3 6 8 8 43 41
2363 M 58 R 18 Anoxia Bilateral HC −2.64 98 73 0 0 10 13 58 44
2563 M 59 L 16 Anoxia Bilateral HC N/A 102 75 1 7 12 12 52 44
1951 M 62 R 16 HSE Bilateral HC + MTL −8.10 121 57 2 4 9 11 49 44
HC Mean 59.3 15.5 −5.0 101.2 65.5 1.5 4.3 9.8 11 50.5 43.3
BDCMean 69.5 13 N/A 115.5 116.3 N/A N/A N/A N/A 57 43.8
Open in a new tab

Demographic, neuroanatomical, and neuropsychological characteristics of participants with hippocampal amnesia and brain-damaged comparisons; Note: Hd.=Handedness. Ed.=years of completed education. HSE=Herpes Simplex Encephalitis. HC=hippocampus. +MTL=damage extending into the greater medial temporal lobes. N/A=no available data. Volumetric data are z-scores as measured through high resolution volumetric MRI and compared to a matched healthy comparison group (Allen, Tranel, Bruss, & Damasio, 2006; Buchanan et al., 2005). WMS-III GMI=Wechsler Memory Scale–III General Memory Index. WAIS-III FSIQ=Wechsler Adult Intelligence Scale–III Full Scale Intelligence Quotient. AVLT = Auditory Verbal Learning Test, 15 min delay score; CFT = Complex Figure Test, recall score; BNT=Boston Naming Test. TT=Token Test. Bolded scores are impaired as defined as 2 or more standard deviations below normative data.

Appendix B. Carl stories

graphic file with name nihms978766u1.jpg

  1. One day Carl decided he wanted to try his luck on the flying trapeze.

  2. He went to the store and bought a new outfit covered in stars (STARS) that he thought would make him look like a professional.

  3. Then he caught a ride (HITCHHIKE/TAXI) down to the nearby circus.to talk to the Ringmaster.

  4. The Ringmaster was desperate for a trapeze artist and asked Carl to do his first show that very same night (TONIGHT).

  5. But Carl didn’t mention that he had never actually been on a trapeze before.

  6. So as soon as Carl got up on the bar, he got scared and let go and flew off into the crowd (FLIP, SOAR).

graphic file with name nihms978766u2.jpg

  1. Carl wanted to start a fire in his backyard so he got an ax to split wood.

  2. All of his friends told him to get face protection (GOGGLES, MASK) but he didn’t think he needed it.

  3. He took the ax outside and wildly chopped at the wood (AX SWING).

  4. His neighbor was watching and came over and asked if he’d chop some logs for her too.

  5. So Carl got excited and chopped faster and faster (AX SWING).

  6. And of course, when he least expected it, half of a log flew up and hit him in the face (NOSE, FOREHEAD).

graphic file with name nihms978766u3.jpg

  1. Carl decided to try a new recipe for his friends when he had them over for dinner.

  2. He searched and searched (BOOK/COMPUTER) for a new recipe to try and finally found one for meatballs.

  3. He ground up meat himself and then formed the meat into balls (BALL).

  4. When his friends come over, he starting cooking the meatballs (OVEN/STOVE).

  5. Then he went in the other room and talked (TALK) to his friends.

  6. But he forgot about the meatballs and when he went back into the kitchen they were burnt to a crisp.

graphic file with name nihms978766u4.jpg

  1. For Halloween, Carl decided he wanted to be Frankenstein (BOLTS).

  2. He was going to a Halloween party and he knew that the girl he liked would be there and he wanted to impress her.

  3. So he went to the costume store and got bolts for his neck and one big googly eye (EYE).

  4. Then on his way to the party, he stopped and got a flower (PICKED/CUT) to give to the girl.

  5. Before he even got to the party, he saw her outside and got excited and ran toward her.

  6. But she didn’t recognize him and got scared and she hit him (PUNCH/SLAP) in the head.

Appendix C. Participant responses by coding category

Features Speech identical Gesture identical Gesture related Other
Speech: Got a flower.
Gesture: picked/cut 		got picked/plucked snipped/cut picked up (picked) purchased/bought (cut) gave brought took presented found
Speech: She hit him.
Gesture: punched/slap 		hit punched/bopped/
socked slapped/whacked/watted		 hauled off (punched)
Speech: Searched for a recipe.
Gesture: computer/cookbo ok 		searched cookbook/books computer went through recipes (cookbook)
looked up (computer)		 looked found
Speech: Cooked the meatballs.
Gesture: stovetop/oven 		cooked oven/baked/in stovetop/on/pan grilled
Speech: Got a ride.
Gesture: hitchhike/taxi 		got went walked happened by ran trotted
Speech: Flew off into the crowd.
Gesture: tumbled/shot 		flew tumbled/spun flew away fell/dropped (shot) crashed wound up in jumped off ended up landed
Speech: Face protection.
Gesture: mask/goggles 		face protection goggles/eye protection mask/head gear protectors
Speech: Hit him in the face.
Gesture: nose/forehead 		face forehead/head nose eye (nose) got hurt hit
Open in a new tab

References

  1. Baddeley AD, Hitch GJ, Allen RJ. Working memory and binding in sentence recall. Journal of Memory and Language. 2009;61(3):438–456. doi: 10.1016/j.jml.2009.05.004. [DOI] [Google Scholar]
  2. Bor D, Cumming N, Scott CEL, Owen AM. Prefrontal cortical involvement in verbal encoding strategies. European Journal of Neuroscience. 2004;19(12):3365–3370. doi: 10.1111/j.1460-9568.2004.03438.x. [DOI] [PubMed] [Google Scholar]
  3. Cassell J, McNeill D, McCullough K-E. Speech-gesture mismatches: evidence for one underlying representation of linguistic and nonlinguistic information. Pragmatics & Cognition. 1998;6(2) [Google Scholar]
  4. Cavaco S, Anderson SW, Correia M, Magalhaes M, Pereira C, Tuna A, … Damasio H. Task-specific contribution of the human striatum to perceptual-motor skill learning. Journal of Clinical and Experimental Neuropsychology. 2011;33(1):51–62. doi: 10.1080/13803395.2010.493144. [DOI] [PubMed] [Google Scholar]
  5. Davachi L. Item, context and relational episodic encoding in humans. Current Opinion in Neurobiology. 2006;16(6):693–700. doi: 10.1016/j.conb.2006.10.012. [DOI] [PubMed] [Google Scholar]
  6. Duff MC, Brown-Schmidt S. The hippocampus and the flexible use and processing of language. Frontiers in Human Neuroscience. 2012 Apr;6:69. doi: 10.3389/fnhum.2012.00069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Duff MC, Brown-Schmidt S. Hippocampal contributions to language use and processing. In: Hannula D, Duff MC, editors. The hippocampus from cells to systems: Structure, connectivity, and functional contributions to memory and flexible cognition. 2017. pp. 503–536. [Google Scholar]
  8. Eichenbaum H, Cohen NJ. From Conditioning to Conscious Recollection: Memory Systems of the Brain. Oxford: Oxford University Press; 2001. [Google Scholar]
  9. Habets B, Kita S, Shao Z, Ozyurek A, Hagoort P. The role of synchrony and ambiguity in speech-gesture integration during comprehension. Journal of Cognitive Neuroscience. 2011;23(8):1845–54. doi: 10.1162/jocn.2010.21462. [DOI] [PubMed] [Google Scholar]
  10. Hilverman C, Cook SW, Duff MC. Hippocampal declarative memory supports gesture production: Evidence from amnesia. Cortex. 2016;85:25–36. doi: 10.1016/j.cortex.2016.09.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Hilverman Cook, Duff Hand gestures support word learning in patients with hippocampal amnesia. doi: 10.1002/hipo.22840. (in revision) [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Hilverman Duff, Cook Hand gestures reflect motor experiences supported by procedural memory (in prep) [Google Scholar]
  13. Kelly SD, Ozyürek A, Maris E. Two sides of the same coin: speech and gesture mutually interact to enhance comprehension. Psychological Science. 2010;21(2):260–7. doi: 10.1177/0956797609357327. [DOI] [PubMed] [Google Scholar]
  14. Klooster NB, Cook SW, Uc EY, Duff MC. Gestures make memories, but what kind? Patients with impaired procedural memory display disruptions in gesture production and comprehension. Frontiers in Human Neuroscience. 2015 Jan;8:1–13. doi: 10.3389/fnhum.2014.01054. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Konkel A, Cohen NJ. Relational memory and the hippocampus: Representations and methods. Frontiers in Neuroscience. 2009 Sep;3:166–174. doi: 10.3389/neuro.01.023.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Kovner R, Mattis S, Goldmeier E. A technique for promoting robust free recall in chronic organic amnesia. Journal of Clinical Neuropsychology. 1983;5(1):65–71. doi: 10.1080/01688638308401151. [DOI] [PubMed] [Google Scholar]
  17. Kurczek J, Brown-Schmidt S, Duff M. Hippocampal contributions to language: Evidence of referential processing deficits in amnesia. Journal of Experimental Psychology: General. 2013;142(4):1346–1354. doi: 10.1037/a0034026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Kurczek J, Wechsler E, Ahuja S, Jensen U, Cohen NJ, Tranel D, Duff M. Differential contributions of hippocampus and medial prefrontal cortex to self-projection and self-referential processing. Neuropsychologia. 2015;73:116–126. doi: 10.1016/j.neuropsychologia.2015.05.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. McNeill D. Speech and Gesture Integration. 1995;(7) [Google Scholar]
  20. McNeill D. Hand and Mind: What Gestures Reveal About Thought. University of Chicago Press; 1992. [Google Scholar]
  21. Ozyürek A, Willems RM, Kita S, Hagoort P. On-line integration of semantic information from speech and gesture: insights from event-related brain potentials. Journal of Cognitive Neuroscience. 2007;19(4):605–16. doi: 10.1162/jocn.2007.19.4.605. [DOI] [PubMed] [Google Scholar]
  22. Pruner, Cook Improved Learning when Hand Gestures Anticipate Speech under revew. [Google Scholar]
  23. Race E, Keane MM, Verfaellie M. Medial temporal lobe damage causes deficits in episodic memory and episodic future thinking not attributable to deficits in narrative construction. The Journal of Neuroscience : The Official Journal of the Society for Neuroscience. 2011;31(28):10262–9. doi: 10.1523/JNEUROSCI.1145-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Ryan JD, Althoff RR, Whitlow S, Cohen NJ. Amnesia is a deficit in relational memory. Psychological Science : A Journal of the American Psychological Society /APS. 2000;11(6):454–461. doi: 10.1111/1467-9280.00288. [DOI] [PubMed] [Google Scholar]
  25. Schegloff EA. Analyzing Single Episodes of Interaction: An Exercise in Conversation Analysis. Social Psychology Quarterly. 1987;50(2):101–114. [Google Scholar]
  26. Schlichting ML, Mumford JA, Preston AR. Learning-related representational changes reveal dissociable integration and separation signatures in the hippocampus and prefrontal cortex. Nature Communications. 2015;6:1–10. doi: 10.1038/ncomms9151. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Schlichting ML, Preston A. Memory integration: neural mechanisms and implications for behavior. Current Opinion in Behavioral Sciences. 2015 Feb;(1):1–8. doi: 10.1038/nmeth.2839.A. [DOI] [PMC free article] [PubMed]
  28. Squire LR. Memory for relations in the short term and the long term after medial temporal lobe damage. Hippocampus. 2017;27(5):608–612. doi: 10.1002/hipo.22716. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Straube B, Green A, Weis S, Chatterjee A, Kircher T. Memory Effects of Speech and Gesture Bindin : Cortical and Hippocampal Activation in Relation to Subsequent Memory Performance. Journal of Cognitive Neuroscience. 2008;21(4):821–836. doi: 10.1162/jocn.2009.21053. [DOI] [PubMed] [Google Scholar]
  30. Winocur G, Weiskrantz L. An investigation of paired-associate learning in amnesic patients. Neuropsychologia. 1976;14(1):97–110. doi: 10.1016/0028-3932(76)90011-7. [DOI] [PubMed] [Google Scholar]
  31. Zeithamova D, Preston AR. Temporal Proximity Promotes Integration of Overlapping Events. Journal of Cognitive Neursocience. 2017;29(8):1311–1323. doi: 10.1162/jocn. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES