The hippocampus and the flexible use and processing of language

Abstract

Fundamental to all human languages is an unlimited expressive capacity and creative flexibility that allow speakers to rapidly generate novel and complex utterances. In turn, listeners interpret language “on-line,” incrementally integrating multiple sources of information as words unfold over time. A challenge for theories of language processing has been to understand how speakers and listeners generate, gather, integrate, and maintain representations in service of language processing. We propose that many of the processes by which we use language place high demands on and receive contributions from the hippocampal declarative memory system. The hippocampal declarative memory system is long known to support relational binding and representational flexibility. Recent findings demonstrate that these same functions are engaged during the real-time processes that support behavior in-the-moment. Such findings point to the hippocampus as a potentially key contributor to cognitive functions that require on-line integration of multiple sources of information, such as on-line language processing. Evidence supporting this view comes from findings that individuals with hippocampal amnesia show deficits in the use of language flexibly and on-line. We conclude that the relational binding and representational flexibility afforded by the hippocampal declarative memory system positions the hippocampus as a key contributor to language use and processing.

The role of hippocampus (and related MTL structures) in the formation and subsequent retrieval of new enduring (long-term) memory is well-established (Cohen and Squire, 1980; Squire, 1992; Cohen and Eichenbaum, 1993; Gabrieli, 1998). The hippocampus also plays a central role in support of relational memory binding; the encoding of the co-occurrences of people, places, and things along with their spatial, temporal, and interactional relations into long-term memory representations (Cohen and Eichenbaum, 1993; Ryan et al., 2000; Eichenbaum and Cohen, 2001; Davachi, 2006). Recent findings, however, challenge the traditional view that the hippocampus contributes exclusively to long-term memory by revealing declarative memory deficits in patients with hippocampal amnesia following minimal delays and when all necessary information is immediately available (e.g., Hannula et al., 2006; Barense et al., 2007; Warren et al., 2010). These results converge with fMRI findings of hippocampal activation for relational learning over short delays (e.g., Ranganath and D'Esposito, 2001; Hannula and Ranganath, 2008). These findings suggest that new hippocampus-dependent declarative memory representations are available early enough in processing of a stimulus to influence behavioral performance “on-line,” that is, rapidly enough to change the initial interpretation of a stimulus event. The strong implication is that hippocampus-dependent representations will be deployed and rapidly available when any materials are processed in an ongoing fashion, and that the performance of patients with hippocampal lesions and declarative memory deficits will consequentially suffer. Indeed, on-line maintenance and processing of simple (Warren et al., 2010) and complex (Olson et al., 2006; Barense et al., 2007) objects, scenes (Ryan et al., 2000; Hannula et al., 2006) and, as we shall see, the on-line interpretation of a sentence, are impaired in patients with hippocampal amnesia.

Language requires the rapid and incremental processing of flexible and contextually defined form-meaning mappings. These linguistic forms are formulated in rich, multi-modal contexts and are interpreted with respect to multiple sources of information. How this is accomplished in the brain, however, is the source of unresolved theoretical debates and open questions surrounding the underpinnings of language use and processing.

Attempts to link aspects of memory to particular properties of language are longstanding. Much of the work linking the demands of language processing—and the theories that guide it—has focused on “working memory” and/or “executive control processes,” functions putatively associated with prefrontal cortex mechanisms. Certainly, the semantic memory system provides critical support for language processing, in terms of maintaining enduring representations of word meanings and world knowledge. Additionally, a number of prominent theories specify a critical role in language use for episodic memory (e.g., Clark and Marshall, 1978; Goldinger, 1998). However, the hippocampus-dependent declarative memory system as a whole has received considerably less attention as a potential candidate memory mechanism supporting on-line language processing, that is, the incremental processing of language as it unfolds in real-time. This is in part due to the long-held assumption that the hippocampal declarative memory system contributes only to long-term memory representations (not those that are available quickly enough to guide on-line information processing). Further, semantic knowledge (supporting lexical access) becomes independent of the hippocampus over time via neocortical consolidation processes (Alvarez and Squire, 1994; McClelland et al., 1995). As a result, the hippocampus-dependent declarative memory system has not received serious consideration as a neural/cognitive system involved in language use and processing in the moment.

Here we propose the hippocampus as a key contributor to language use and processing. Our proposal draws on the relational binding and representational flexibility that are hallmark characteristics of the hippocampal declarative memory system, along with recent findings stretching the scope of hippocampus-dependent processes to functions that operate in-the-moment and on the narrowest of time-scales. In this review we delineate an emerging view of the role of the functions of the hippocampus and its implications for theories of language processing and use. Specifically, we propose that many of the processes by which we produce and understand language also place high demands on and receive contributions from the hippocampal declarative memory system. We will review the fundamental properties of language with special attention to those aspects that would seem to place particularly high demands on the hippocampal declarative memory system. We present evidence from patients with hippocampal amnesia who show deficits in the flexible and on-line use of language. We will also point to aspects of language use that appear independent of the hippocampal memory system. Finally, we raise some open questions that follow from our proposal.

Fundamental properties of language

The fundamental properties of language include its arbitrariness, its incrementality, its flexible and creative expression, and the fact that it is multimodal. These properties place critical demands on our ability to learn, process, and use language across the lifespan. In this section, we review each of these properties.

Language is a system of arbitrary relations

Language conveys meaning using a set of largely arbitrary symbols, arranged temporally (and spatially, in the case of signed languages) following the patterns of the language. The relationship between the phonological form of a word and its meaning is largely arbitrary. That is, with some limited exceptions (e.g., onomatopoeia and other limited word form and meaning associations, Maurer et al., 2006), conceptual information cannot be derived from the acoustic signal itself. The lack of a direct correspondence of meaning and form is perhaps most clearly exemplified by the cross-linguistic variation in the phonetic realization of symbols used to denote a given meaning. For example the meaning associated with BIRD is denoted in various languages as “oiseau” (French), “Vogel” (German), “zintkala” (Lakota), and “burung” (Malay). Even within a language, language understanding is complicated by what is known as the many-to-many mapping problem: Across talkers and phonetic contexts, a given acoustic signal can map onto different meanings [e.g., in some American English dialects, the phoneme sequence in/pIn/ maps onto both the meanings PIN and PEN (e.g., a vowel merger)] (Peterson and Barney, 1952; Liberman et al., 1967). Conversely, a given meaning can be expressed by different acoustic signals. The way in which symbols are arranged in a language is also largely arbitrary. In English, adjectives generally precede nouns (e.g., the small butterfly), whereas in Spanish they typically follow the noun (e.g., la mariposa pequeña). In English and Spanish, the default word order is subject-verb-object (e.g., The girl caught the butterfly), whereas in Korean and Japanese it is subject-object-verb. Using language critically involves learning and using these language- (or dialect-) specific relational mappings.

Language processing is incremental

Spoken language unfolds over time, at a rate of about 150–200 words per minute (Levelt, 1989; Tauroza and Allison, 1990). As a result, many words and phrases are ambiguous, at least temporarily, causing a proliferation of possible syntactic and semantic interpretations as they accrue. Because the meaning of many words is unclear until later in the sentence, language is processed incrementally and multiple sources of information must be generated, integrated, and maintained in real-time to create meaning. Evidence from studies using on-line methods reveal that: (a) words are integrated immediately into the on-going interpretation of an utterance, with listeners making provisional commitments to interpretations of lexical (Allopenna et al., 1998), referential (Chambers et al., 2002; Hanna et al., 2003), and syntactic ambiguities (Tanenhaus et al., 1995; Wilson and Garnsey, 2009) as the utterance unfolds; and (b) listeners make sophisticated linguistic predictions about upcoming material (Federmeier and Kutas, 1999; DeLong et al., 2005; Federmeier, 2007), and modulate those predictions based on integration of language with concurrent scenes (Altmann and Kamide, 1999, 2007), past and inferred scenes (Altmann and Kamide, 2009), and characteristics of the sentential subject (Kamide et al., 2003) or talker (Van Berkum et al., 2008, in press; Tesink et al., 2009). This work points to rapid, bi-directional communication between multiple levels of representation. In healthy adults, this would include well-established (or enduring long-term) representations of syntax, semantics, pragmatics and phonology, along with social and contextual knowledge.

Language use is flexible and creative

Fundamental to all human languages are a virtually unlimited expressive capacity and a creative flexibility that allow speakers to rapidly generate novel and complex utterances. All language use involves, to a greater or lesser degree, some level of creativity and flexibility, as individuals rhetorically or poetically select particular sounds and meanings in crafting their utterances and selecting which details to represent for a specific listener on a particular occasion (e.g., Tannen, 1989; Norrick, 1998). Indeed, in using language speakers do not simply reproduce: they recreate, repurpose, and recontextualize (Voloshinov, 1973; Bakhtin, 1986; Prior, 2001; Maybin and Swann, 2007). Sociolinguistic research points to the diverse functions that language flexibility and creativity serve across everyday settings as we reconstruct, reenact, and retell the events and narratives of our lives (Tannen, 1989; Ochs and Capps, 2001), and engage in playful verbal banter (Crystal, 1998; Sherzer, 2002).

Particularly in dialog settings, language use involves the flexible tailoring of language to the situation and dialog partner. In a classic technique for studying dialog termed the Referential Communication Task (Krauss and Weinheimer, 1964, 1966; Fussell and Krauss, 1992), pairs of naive participants, separated by a barrier, work together to complete a joint task. Participants are each given a set of ∼12 cards with distinct but potentially confusable images (e.g., Figure 1), and one participant, the director, instructs another participant, the matcher, to re-arrange her cards into a different order over multiple rounds of play. Studies using the Referential Communication Task have demonstrated that dialog partners jointly create shared names for the images, speeding the accuracy and efficiency of game play across rounds (Clark and Wilkes-Gibbs, 1986). These names are not freely extended to new partners or non-participatory partners (e.g., overhearers, Schober and Clark, 1989; Wilkes-Gibbs and Clark, 1992; Brown-Schmidt, 2009a), suggesting that when the labels are interactively established, dialog partners also encode information about who that knowledge is shared with, i.e., information about their common ground (Clark, 1992, 1996).

Figure 1.

Set up and tangram figures for Referential Communication Task.

In addition to tailoring descriptions to individual partners, dialog partners also show convergence in other aspects of language use; these convergence effects are argued to be key evidence for alignment theories of conversation (Pickering and Garrod, 2004), in which the adaptation of language to partner and context is seen as key to its success. For example, listeners show perceptual learning for vocal qualities of individual talkers that subsequently affect both explicit judgments (Goldinger, 1998; Nygaard and Pisoni, 1998; Evans and Iverson, 2003; Kraljic and Samuel, 2006, 2007; Maye et al., 2008), and implicit measures of on-line language processing (Dahan et al., 2008; Trude and Brown-Schmidt, 2012). In conversation, dialog partners show a wide range of imitative effects, including phonetic convergence (Pardo, 2006; Bailly and Lelong, 2010), as well as convergence of gaze, facial expression, posture, and foot-tapping (Kendon, 1970; Chartrand and Bargh, 1999; Richardson and Dale, 2005; Richardson et al., 2007). Social effects on phonetic convergence, including perceived attractiveness and other-race biases (Babel, 2009), along with evidence that mimicry increases rapport and facilitates communication (LaFrance and Broadbent, 1976; LaFrance, 1979; Chartrand and Bargh, 1999; Richardson and Dale, 2005) suggest these processes are non-obligatory and that they may serve critical social functions.

Convergence extends to higher-level language structures including task-relevant descriptions and mental models of game play (Garrod and Anderson, 1987; Schober, 1993; Brennan and Clark, 1996), and syntactic form (Bock, 1986, 1989; also Levelt and Kelter, 1982; Branigan et al., 2000; Gries, 2005; Haywood et al., 2005; Reitter et al., 2006; Reitter and Moore, 2007). Such convergence effects are modulated by the status of the conversational participants (stronger effects with addressees than overhearers; Branigan et al., 2007), and are argued to be a critical component of successful communication (Clark and Marshall, 1981; Garrod and Doherty, 1994).

Language use is multi-modal

Typical language use in face-to-face conversation has the special property that the communication sources and receivers are bodies situated in the physical world. The result of this primary observation is that the domain of what is considered “language” for the purposes of the conversation encompasses far more than the stream of spoken words. Instead, the way the speaker's body moves as she speaks (e.g., gaze and gesture), and the way her language and her body interface with the physical world (e.g., pointing at objects) become integral to the conversation—the language—itself. Thus, understanding how language is used in conversation requires appreciating how the spoken (or signed) component of language is integrated with the world and with the bodies that produce and comprehend that language.

The relevant context for language use, or referential domain (Chambers et al., 2002; Landragin, 2006), is multiply determined. For talk about the co-present visual world, the relevant referential domain is based on an integration of information from the physical world, discourse history, and task goals. For example, consider a scenario in which Annie and James are at a museum, and Annie wishes to comment on one of Klein's Blue Monochromes. Displayed alone, she could refer to the painting as “this piece” or even “it,” relying on the physical context to highlight the intended referent over other salient but non-present referents (e.g., the Mona Lisa). However, if multiple pieces were in view, to uniquely identify the referent she would have to modify her expression with respect to the context, saying “the blue painting” or “the monochrome one,” depending on the properties of other objects in the context (Olson, 1970; Osgood, 1971). The physical context not only constrains how we can refer to entities; it also changes the interpretation process, allowing listeners to combine scene information with the ongoing sentence to make predictions about upcoming material (Altmann and Kamide, 1999) and to rule out alternative interpretations of syntactic (Tanenhaus et al., 1995; Spivey et al., 2002), and referential (Eberhard et al., 1995; Sedivy et al., 1999) ambiguities.

Speaker gaze is another source of information that constrains the referential domain. During conversation, speakers typically gaze at the objects in a scene in the order in which they will name them, at a latency of approximately −800 to −1000 ms (Meyer et al., 1998; Griffin and Bock, 2000; Meyer, 2004). As a result, the addressee in face-to-face conversation can take advantage of the gaze-speech link to facilitate comprehension processes. Speakers also make facial and manual gestures, as well as other body movements. These gestures can carry independent meaning as in the case of iconic gestures (Holle and Gunter, 2007; also see Wagner Cook and Tanenhaus, 2009), and can identify a referent or referential domain (see Landragin, 2006), as in the case of deixis or pointing gestures (Clark and Krych, 2004). These findings suggest that using language requires not only integrating across the words in a sentence, but also integrating across multiple, rich sources of information from different modalities, including information about the visual world, gaze and gesture.

The relationship between language and memory

Although these fundamental properties of language are considered universal and are accomplished rapidly and with seemingly little effort, how they are accomplished in the brain is the source of unresolved theoretical debates and open questions surrounding the underpinnings of language use and processing. Attempts to link aspects of memory to particular properties of language are longstanding. In the language literature, a dominant theme is the relationship between language processing and working memory, specifically the hypothesis that central functions including working memory and cognitive control are required to execute language processing (Daneman and Carpenter, 1980, 1983; Just and Carpenter, 1992; Just et al., 1996; Caplan and Waters, 1999; Walker, 1996; Novick et al., 2005; although a prominent alternative view emphasizes the role of experience, rather than capacity limits in capturing individual differences, MacDonald and Christiansen, 2002). Working memory is an attractive candidate mechanism for meeting the demands of incremental language processing due to its purported on-line maintenance capacity and the relative ease of correlating performance on working memory and other executive function tasks with language processes in healthy subjects (e.g., Nilsen and Graham, 2009; Brown-Schmidt, 2009b; Lin et al., 2010; cf. Otten and Van Berkum, 2009). The attention to working memory is also likely related to the traditional view of the hippocampal declarative memory system as making contributions only to long-term memory representations, and not to those available rapidly enough to guide information processing in the moment.

Another approach to the memory-language interface has been to examine the relationship between types of language knowledge and the types of memory that support them. Ullman (2004) proposed the declarative/procedural model. On this model, the mental lexicon of memorized arbitrary word-specific knowledge depends on the medial temporal lobe substrates of declarative memory. In contrast, the mental grammar depends on the network of frontal, basal-ganglia, parietal and cerebellar structures of procedural memory. The model has been used to account for (among other things) dissociations in amnesia between impaired lexical and spared grammatical learning. Indeed, an extensive body of work in the memory literature demonstrates deficits in vocabulary acquisition in patients with hippocampal damage and declarative memory deficits (e.g., Gabrieli et al., 1988; Bayley and Squire, 2002, 2005; Schmolck et al., 2002; Manns et al., 2003; O'Kane et al., 2004; cf. Vargha-Khadem et al., 1997; Sharon et al., 2011). Consistent with the declarative/procedural model, these deficits are observed in the context of intact learning of the patterns and statistical regularities of the grammar as measured by artificial grammar learning tasks (Knowlton et al., 1992; Knowlton and Squire, 1994, 1996).

In the memory literature, the relationship between language and declarative memory has largely been confined to word learning and interpreted in the context of semantic memory. This likely has historical roots. The nature of observed impairment in patients with hippocampal amnesia, such as H.M., has been described as a deficit that is specific to the domain of memory (leaving intellect and other cognitive abilities such as language and attention intact) and within the domain of memory (affecting only certain forms of memory). Following bilateral resection of the medial temporal lobes, H.M. was considered to have normal language ability as he was not aphasic and performed within normal limits on neuropsychological tests of language (Scoville and Milner, 1957; Milner et al., 1968). Furthermore, H.M.'s premorbid vocabulary and remote episodic memory were judged to be largely intact suggesting that the hippocampus is not the storage site for declarative memory. Rather, declarative memory can become independent of the hippocampus over time via neocortical consolidation processes (Alvarez and Squire, 1994; McClelland et al., 1995). These findings have encouraged a view of memory as a cognitive capacity distinct from the various cognitive domains (language, spatial processing, etc.) that it serves. As a result, the impairment in amnesia is seen as exclusive to a specific aspect of memory functioning. While subsequent reports later raised questions about H.M.'s language abilities, citing lexical, phrasal, and sentence-level language disruptions (e.g., Lackner, 1974; Corkin, 1984; MacKay et al., 1998a,b), conclusively linking these deficits to the functions of the hippocampus have been difficult given HM's extensive medial temporal lobe damage extending beyond the hippocampus, his decades long history of treatment for epilepsy, and his advancing age (see Schmolck et al., 2000, 2002).

Acknowledging, of course, that not all aspects of language are hippocampus-dependent, these conceptual and historical factors have kept the hippocampal declarative memory system from serious consideration as a neural/cognitive system involved in language use and processing (for exceptions see Clark and Marshall, 1978; Goldinger, 1998). Our proposal stems from recent discoveries regarding the surprising contribution of the hippocampus to a variety of disparate functions and tasks, and over a time scale typically considered within the domain of working memory. These findings challenge and expand the traditional view of hippocampal function. We argue that these new findings have significant implications for theories of language use and processing, and should encourage increased interest in the relationship between language and declarative memory.

Hippocampal declarative memory: new discoveries and their implications for language use

The hippocampus (and related MTL structures) plays a critical role in the formation and retrieval of new enduring (long-term) declarative memories and in the support of declarative memory use (Cohen and Squire, 1980; Cohen, 1984; Squire, 1987, 1992; Gabrieli, 1998; Eichenbaum and Cohen, 2001). Two hallmark features of the hippocampal declarative memory system include its role in the creation and integration of relational representations and the flexible expression of those representations. Relational representations are created and supported through the binding of the arbitrary co-occurrences of people, places, and things of a scene or event. These binding operations link the spatial, temporal and interactional relations the components of an event, thus establishing the larger record of one's experience over time (e.g., a word and its meaning; an object and its location; Eichenbaum and Cohen, 2001; Davachi, 2006; Konkel et al., 2008; Ranganath, 2010).

These relational (declarative) representations, which are supported by the hippocampus, are uniquely flexible, permitting integration with other types of representations. Through the interaction of the hippocampal system with various neocortical storage sites that are also involved in the initial encoding of stimuli, these representations are accessible to other processing systems (as when a rich autobiographical memory is evoked by the sound of a familiar song), and are readily extended to use in novel contexts (O'Keefe and Nadel, 1978; Cohen, 1984; Squire, 1992; Dusek and Eichenbaum, 1997; Eichenbaum and Cohen, 2001).

Recent evidence challenging the traditional view of the hippocampal declarative memory system as contributing exclusively to long-term memory has shown that patients with hippocampal amnesia exhibit deficits in declarative memory even when there are very short delays and no interposed delays at all, and even when all the necessary information is in view (Hannula et al., 2006; Olson et al., 2006; Barense et al., 2007; Warren et al., 2010). These results converge with fMRI findings of hippocampal activation for relational learning over similarly short delays (e.g., Ranganath and D'Esposito, 2001; Hannula and Ranganath, 2008). These findings suggest that new hippocampus-dependent representations are available rapidly enough to influence ongoing processing when: new information is perceived; old information is retrieved; and representations are held on-line to be evaluated, manipulated, integrated, and used in service of behavioral performance. Although the cited studies used visual or visuospatial stimuli, the strong implication is that the hallmark flexibility and integration of hippocampus-dependent representations will be deployed and rapidly available when any materials are processed in an ongoing fashion, and that the performance of patients with hippocampal lesions and declarative memory deficits will consequentially suffer.

These provocative findings regarding the time course of hippocampal contributions to on-line processing have profound implications for theories of language processing and use. We have been conducting a line of work examining the contribution of declarative memory to communication and language use in patients with hippocampal damage and severe declarative memory deficits (Duff et al., 2006, 2007, 2008a,b, 2009, 2011; Kurczek and Duff, 2011). This work has revealed deficits across various aspects of linguistic and discourse functions suggesting that patients with hippocampal amnesia have difficulty establishing, recovering, maintaining and using declarative memory representations throughout a conversation particularly when the demands on flexible and creative uses of language are high. We recently extended this line of work to examine on-line language processing (Rubin et al., 2011) and have initial, tantalizing evidence for hippocampal mediation in real-time language processing.

Here we propose that many of the processes by which we produce and understand language place high demands on and receive contributions from the hippocampal declarative memory system. Established functions of hippocampal declarative memory system include its relational binding and representational flexibility, allowing the creation and flexible integration of different types of representations. These functions support the formation and maintenance of new memories, and new research from the visual domain demonstrates the hippocampus is critically involved in the on-line maintenance and processing of relational representations as well. At the heart of our proposal is that these same features—on-line processing of rich representations from multiple domains—are key demands of the flexible use and on-line processing of language.

The hippocampus and the flexible use and processing of language

Hippocampal amnesia disrupts the flexible and creative use of language

Language use—especially in the dialog contexts typical of everyday use—is highly flexible. This points to the significance of underlying cognitive mechanisms and neural substrates that are adept at handling flexible relations. Consider the mappings of words to meanings: While a given word may have a typical meaning, words and phrases often have consistently different meanings in different contexts, and for different people. Form-meaning mappings change depending on variables including dialect (e.g., pop vs. soda), whether the addressee is a friend vs. a stranger (e.g., my house vs. the green house, third from the left; Fussell and Krauss, 1989), and the addressee's knowledge and expertise (e.g., Rockefeller center vs. the square with the flags; Issacs and Clark, 1987; Horton and Keysar, 1996). These phenomena suggest that language processing relies on flexible mappings between meanings and words. In some cases word-meaning mappings reflect enduring, long-term associations; in other cases these mappings are generated during the course of a conversation. A number of findings suggest that the flexibility important for conversational language use is provided by the hippocampal declarative memory system.

In our first study in this line of work, we investigated the ability of individuals with hippocampal amnesia to acquire and use referential labels for novel picture cards across repeated collaborative interactions with a familiar partner using a modified version of the Referential Communication Task described above (Duff et al., 2006). In the study, amnesic patients sat across from a familiar partner (friend, spouse) and they each had a board with 12 numbered spaces and a set of 12 cards displaying Chinese tangrams (e.g., Figure 1). A low barrier was placed on the table between them preventing a view of each others' cards but allowing them to see each other's facial expressions and gestures. The amnesics were always the director and communicated to their familiar partners (always the matcher) how to complete their board with the cards so that at the end of the trial the two boards looked alike. The task was presented as a game and participant pairs were instructed to communicate freely and to have fun. The pairs played the game 24 times over 2 days (six trials per session, two sessions per day). We found that despite severe declarative memory impairments, amnesic participants developed and used unique labels for the cards. Across trials these labels became increasingly concise and simplified. In fact, the rate of learning exhibited by amnesic participants, measured by the reduction in time and words necessary to complete each trial, did not differ from that of healthy participants.

These findings were important in a number of regards. First, in contrast to other studies of word learning or semantic memory (e.g., Gabrieli et al., 1988), the learning exhibited by the amnesic participants in our collaborative referencing task involved a normal rate of acquisition. (It is important to note that the amnesic patients were not required to learn arbitrarily related labels, but rather drew on preexisting semantic representations, such as “siesta man” for a figure that could be seen as a man resting or reclining.) Second, these findings suggested that amnesic patients could acquire and use common ground in their communicative interactions. This challenged views of the memory determinants of common ground and suggested that common ground may have multiple forms and determinants, dependent upon the contributions of different memory systems in the brain (Duff et al., 2006). Third, these findings suggest that these interactive sessions offer a potent means of observing the complex interaction of memory and language and of testing the unique contribution of distinct memory systems to meeting the demands of language use. Our modified Referential Communication Task was demanding, particularly of the amnesic patients, and the interactive sessions provided an exceptionally rich corpus of data on language use across extended and repeated interactions and on the particular communication practices and resources of conversational language in amnesia (see Duff et al., 2008b).

While these findings demonstrated surprising sparing of new learning in amnesia, subsequent analyses showed that despite preserved learning, language use was not entirely normal: In a follow-up analysis of the referential forms used to describe the picture cards, we examined the use of definite reference (e.g., the windmill vs. a windmill) (Duff et al., 2011). Definite reference signals to the listener that the speaker believes the referent is uniquely identifiable in the joint representation of the local context, or common ground (Clark and Marshall, 1978). Comparison participants marked the referential forms with a definite reference (e.g., the windmill) 90% of the time whereas amnesic patients were at near chance levels (56%) using indefinite articles (e.g., a windmill) nearly as often (Duff et al., 2011). That is, even though the amnesic patients had described the tangrams multiple times and were using concise labels, even after 24 trials the patients were still using an indefinite reference, as if they were encountering the tangrams and generating the labels for the very first time. These findings highlight the role of declarative memory in flexibly tailoring utterances for specific communication partners to reflect joint knowledge. These findings also point to the role of declarative memory in language beyond semantic memory or word learning by linking a deficit in declarative memory to disruptions in referential processes (e.g., marking noun phrases as definite or indefinite; also see Kurczek and Duff, 2011 for further evidence of deficits in referential processing).

In subsequent analyses aimed at characterizing discourse processes in amnesia, we found that, while the semantic content of the references were similar across amnesia and comparison participants (siesta man vs. lazy man) the patients exhibited a general lack of flexibility in their referential expressions (Duff et al., 2008b). As the task progressed, comparison participants rapidly omitted all non-essential words for identifying target cards, including verbs (e.g., leaning against a tree is one → leaning against a tree eleven), articles (e.g., the bird → bird), and place numbers (e.g., four is the kicker → the kicker). Some comparisons even began to drop morphological endings to further shorten one word labels (e.g., kicking → kick). Amnesic patients, in contrast, did not shorten their utterances in these ways, making the average length of their referential expressions twice as long as comparison participants. The social discourse of the patients also differed from comparison participants in the lack of personal and communal knowledge (e.g., comparisons referenced Elaine dancing, Kramer, and Jerry's cereal from the show Seinfeld) and in acknowledging and using multiple perspectives (e.g., comparisons distinguished perspectives, as in: Number two is to me an Indian, to you the stock market). Yet, amnesic patients still showed intact learning in the task. This suggests that some aspects of common ground are hippocampus-independent, and thus preserved in amnesia, whereas other aspects of common ground are hippocampus-dependent. The patients' lack of discursive flexibility in language production and the striking absence of high-level discourse features such as acknowledging and using the perspectives of others, and explicitly drawing upon personal and communal knowledge suggest that hippocampus-dependent common ground functions include the flexible use of enduring representations in utterance formulation (Duff et al., 2008b, 2011).

Studies of creative language use also point to critical contributions from the hippocampus-dependent declarative memory. Creativity requires the rapid combination and recombination of existing mental representations to create novel ideas and ways of thinking (Damasio, 2001; Bristol and Viskontas, 2006). Examples of creative language use include everyday discourse practices such as reported speech, in which speakers represent or reenact words or thoughts from other times and/or places (e.g., If I ever have kids I'm going to tell them, please don't say mean things to me; Tannen, 1989; McCarthy, 1998). Similarly, verbal play involves playing with the sounds and meanings of words through the use of puns, voices and sound effects, teasing, and telling funny stories (Crystal, 1998; Sherzer, 2002). According to our proposal, the hippocampal declarative memory system supports these processes through the creation, updating, and juxtaposition of mental representations and for their flexible and novel use in dialog contexts. We have found deficits in the use of both of these discourse practices in patients with hippocampal amnesia.

In our study on reported speech, we found that although there were no group differences in the amount of talk, in the conversational sessions of patients with hippocampal amnesia interacting with a clinician there were only half as many reported speech episodes (RSEs) (M = 30.3; SD = 16.9) as there were in sessions with healthy comparisons (M = 61.5; SD = 30.1), a difference that was statistically significant (Duff et al., 2007). As might be expected from patients with anterograde amnesia, the RSEs that referred to recent events (i.e., post-amnesia-onset events) were more schematic and less detailed than those produced by healthy participants. However, this restricted use of reported speech was not limited to reports about the recent time period. Rather, even when producing vivid remote memories, individuals with amnesia are less likely to use reported speech when representing these memories in communicative interactions. We speculated that this was because, although the static display of remote memories becomes independent of the hippocampus over time, the hippocampus plays a critical role in the flexible expression of declarative memory representations in novel situations (Cohen and Eichenbaum, 1993; Eichenbaum and Cohen, 2001; see Duff et al., 2007). Reported speech requires flexible access to our record of events as well as the ability to flexibly and creatively generate unique combinations of the reconstructed elements (what details to represent, what details to omit, to meet the specific interactional goals of this telling, on this occasion, with this communication partner). Reported speech also requires maintaining, relating, and flexibly moving back and forth (mentally) between different time frames. As a result, it places significant demands on the flexible and creative processing capabilities of the hippocampal declarative memory system and thus we observe striking deficits in patients with hippocampal amnesia. In terms of cognitive and neural specificity, it is interesting to note that damage to the ventromedial prefrontal cortex (vmPFC), shown repeatedly to be involved in social and emotional processing and future thinking, does not impair these aspects of reported speech use (Duff et al., 2009).

Similarly, in a study on verbal play in amnesia, we found that amnesia patients produced significantly fewer verbal play episodes (N = 187; M = 46.7) than comparison participants (N = 395; M = 98.7). The quality of the amnesics' verbal play episodes also differed. In contrast to comparison participants, verbal play episodes produced by the patients were more rotely and repetitively produced (as when an amnesia patient reproduced the same joke, nearly verbatim, multiple times). For amnesic participants, verbal play was also less skillfully deployed (i.e., few productions combining verbal, prosodic, and gestural resources), and playful themes were not sustained across stretches of interaction or returned to in subsequent interactions. These findings suggest that hippocampal amnesia impairs the ability to creatively and flexibility deploy the communicative and cognitive resources necessary to meet the moment-to-moment demands of interactional discourse. This is consistent with deficits in creative thinking more generally in amnesia (Duff et al., 2009). Again, in terms of specificity, damage to the vmPFC does not impair these aspects of verbal play (Gupta et al., 2012).

Hippocampal amnesia impairs the incremental processing of language

Separate from the question of the accuracy of language use and understanding is the question of the time-course of language understanding. As stated above, we propose that the same processes by which the hippocampus creates and integrates representations in the formation of new memories, and maintains representations on-line to be evaluated and used in service of behavioral performance, are the same processes necessary for the on-line processing of language. The deficits in language production described above point to difficulty in establishing, recovering, maintaining and using representations across the course of a conversation. An open question is whether language comprehension might also be compromised by hippocampal damage, even in cases where memory demands are minimal. In on-going research, we are finding striking new evidence that hippocampal damage does, in fact, confer deficits in on-line language processing.

As we have discussed, a critical component to language use is circumscribing the relevant domain of interpretation. One key way in which dialog partners constrain the relevant domain is through attention to what information is mutually known between conversational partners, i.e., common ground (Clark, 1992). In healthy individuals, common ground is used rapidly to rule out potential linguistic competitors, allowing the listener to identify what the speaker is referring to (Hanna et al., 2003; Heller et al., 2008; Brown-Schmidt et al., 2008). Interestingly, despite being central to language, little is known about how this information is represented in memory. In a recent study (Rubin et al., 2011), we assessed the role of declarative memory in the use of common ground during on-line referential ambiguity resolution. Rubin and colleagues created situations in which a linguistic ambiguity was eliminated if common ground information was taken into account: participants saw a computer display with two different ducks, whereas the experimenter saw a display with only one duck. Replicating previous findings (Hanna et al., 2003), we found that both amnesics and healthy comparisons reliably looked to the jointly visible duck when the experimenter said Look at the duck. However, even when they successfully fixated the target, amnesia patients were more likely to look at the competitor object than comparison participants. This result suggests that while the ultimate interpretation of language was comparable between amnesics and healthy comparisons, the on-line process was not. A condition in which common ground was established linguistically (see Hanna et al., 2003; Brown-Schmidt, 2012) replicated these findings; however, amnesic patients' performance dropped to chance when a brief minimal, filled delay followed establishment of linguistic common ground.

These findings reveal novel deficits in hippocampal amnesia in using recent experience to guide on-line language processing. Minimal delays were sufficient to eliminate the influence of linguistic history, leaving the patients unable to resolve a simple linguistic ambiguity. Moreover, competitor objects had a more powerful pull on the fixations of amnesia patients than comparison participants, even in the absence of explicit memory demands, raising interesting questions about the distinctiveness of the representations formed by the patients and their ability to use immediately available information to resolve linguistic competition. These results provide key initial evidence for the significance of the hippocampal declarative memory system in on-line language processing. An open question is whether the observed deficits in competition resolution are present at all levels of the language processing system.

Language and the multiple memory systems of the brain

We have argued that many of the processes by which we produce and understand language place demands on and receive contributions from the hippocampal declarative memory system. Indeed, aspects of language use are impaired in patients with hippocampal amnesia when the task demands acquisition of arbitrary relations (e.g., word-meaning mappings), incremental processing, or the flexible and creative use of language. Other aspects of language use appear independent of hippocampal declarative memory. For example, learning of patterns and statistical regularities of the grammar (e.g., Knowlton and Squire, 1996) and syntactic priming (Ferreira et al., 2008) are intact in patients with hippocampal amnesia. Furthermore, much of the work on convergence, mimicry, and adaptation in communication and social interaction, thought to be critical for conversational success, suggest non-declarative mechanisms (e.g., Chartrand and Bargh, 1999; Pickering and Garrod, 2004). A rich and dynamic view of the language processing system that includes all its properties (i.e., its arbitrariness, its multimodality, its incrementality, and its flexible and creative expression) opens up a plethora of questions about which memory systems are called upon in service of meeting the demands of language use.

A compelling approach to addressing these questions is to examine language at the intersection of declarative and non-declarative memory systems and to view the activities of language (e.g., word learning; developing and using common ground) as necessitating a division of labor between the memory systems (see Davis and Gaskell, 2009; Gupta and Tisdale, 2009; Duff et al., 2011). For example, integrated within a Complementary Learning Systems framework (e.g., McClelland et al., 1995), Gupta and colleagues argue that word learning requires contributions from both memory systems: Whereas learning the phonological form of a word depends on non-declarative memory mechanisms, learning the semantic representation of a word depends on declarative memory (Gupta, 2011). This view is in contrast to Ullman's (2004) proposal, which posits that declarative and procedural memory underlie in the lexicon and the grammar, respectively. The approach taken by Gupta and colleagues is also consistent with our work on the memory determinants of common ground. We have suggested that common ground has multiple forms and determinants dependent on the contribution of different memory systems in the brain with procedural memory supporting the gradual learning seen when amnesic patients learn concise referential labels, and declarative memory supporting the ability to linguistically mark those labels as part of shared knowledge (impaired in amnesic patients; Duff et al., 2006, 2008a,b, 2011).

Conclusion

The language processing system is a rich and dynamic set of processes that reach far beyond the arrangement of abstract linguistic symbols according to grammatical rules. Instead, language processing involves the rapid and incremental processing of flexible, creative, and contextually defined form-meaning mappings. These linguistic forms are produced and interpreted with respect to rich and jointly defined multi-modal contexts and are integrated with information from multiple domains including gesture, entities in the physical world, and partner-specific representations of voice quality and jointly established meaning.

Here we challenge a longstanding view of the hippocampus as narrowly serving memory functions, and as such, making limited contributions to the use and processing of language. In doing so, we reach beyond findings of some language deficits in patient HM (e.g., MacKay et al., 1998a,b) in arguing that understanding the role of the hippocampus in language processing will require examining language use in real-time and in rich, multi-modal contexts. Recent findings pointing to deficits in real-time processing and in the absence of delays stretch the scope of hippocampus-dependent processes to functions that operate in-the-moment and on the narrowest of time-scales. Given these findings, along with other evidence that the hippocampus is critically involved in the binding of arbitrary relations across modalities and domains, the hippocampus emerges as a potentially key contributor to the real-time use and processing of language.

Against this backdrop of theoretical proposals and findings, we propose a new view of the role of the hippocampus in language processing in which the hippocampus is integrally involved in the retrieval and binding of information across domains that is critical for on-line language processing. New evidence supporting this view comes from findings that individuals with hippocampal amnesia show deficits in language use (Duff et al., 2007, 2008a,b, 2009, 2011; Kurczek and Duff, 2011) and on-line processing (Rubin et al., 2011). Open questions include whether the flexible and novel use of remote episodic memory, the processing of established semantic memory (e.g., arbitrary word-meaning relations), and the integration of multi-modal information during on-line language processing, such as gesture and object representations, are impaired in amnesia as well.

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.