Memory: Organization and Control

Abstract

A major goal of memory research is to understand how cognitive processes in memory are supported at the level of brain systems and network representations. Especially promising in this direction are new findings in humans and animals that converge in indicating a key role for the hippocampus in the systematic organization of memories. New findings also indicate that the prefrontal cortex may play an equally important role in the active control of memory organization during both encoding and retrieval. Observations about the dialog between the hippocampus and prefrontal cortex provide new insights into the operation of the larger brain system that serves memory.

INTRODUCTION

There is a long history of tension between the view that memories are stored independently as individual associations and the idea that new information is integrated within systematic organizational structures. In the first half of the 20th century, the association and organization views of memory came into conflict in battles between the characterizations of stimulus-response learning (c.f. Spence 1950) and cognitive maps (Tolman 1948), and the distinction was highlighted in Bartlett’s (1932) and Piaget’s (1928) ideas on memory organization in schemas. Critics described the organizational views proposed at that time as vague, but the pioneers of modern cognitive science proposed specific forms of systematic organization in which memories are embedded (e.g., Bower 1970, Collins & Quillian 1969, Mandler 1972; see also Holland 2008). Of particular relevance to this review, Mandler (1972, 2011) proposed three types of memory organization (Figure 1): an associative structure in which multiple events are linked by direct and indirect associations within a network, a sequential structure involving a temporal organization of serial events, and a schematic structure involving a hierarchical or similarly complex organization of items in memory (Mandler used different names for these organizations). Mandler did not attempt to explain the brain mechanisms that underlie these structures and had no expectation that these organizations could be directly observed. Instead, he based his theory of their existence on results from cleverly designed studies that identified types of memory organization by their consequences in memory judgments. In this review, I argue that new approaches in neuroscience are revealing these organizational structures within neural networks and identifying distinct brain mechanisms that guide encoding and retrieval of information within these organizations. A full understanding of how the brain organizes and controls memory requires a synthesis of findings in humans, in which we can best characterize these organizations and identify the key brain areas involved, with findings in animal models, in which we can examine how networks of neurons—the elements of information processing—support the organization and control processes.

Figure 1.

Three forms of memory organization and the role of the hippocampus in humans and animals. (Left column) Elements A, B, and C are related in ways specific to each type of organization. (Middle column) Hippocampal activation in (a) associative (Zeithamova et al. 2012), (b) sequential (Ezzyat & Davachi 2014), and (c) schematic (Zalesak & Heckers 2009) memory organizations. (Right column) Graphs depicting the results in memory performance of rats with hippocampal lesions compared to a control group without lesions. (a) Rats with hippocampal lesions succeed in learning individual elements and associations (AB and BC) but fail in linking indirectly related elements in an associative organization (reflected in the low preference for the indirectly related element association AC) (Bunsey & Eichenbaum 1996). (b) Rats with hippocampal lesions succeed in remembering items in a list but fail in remembering the order of the items in the sequential organization (Fortin et al. 2002). (c) Rats with hippocampal lesions succeed in learning trained choices of all pairings in a five-item hierarchy (A–E; B over C and C over D are shown) but fail in inferring relations between indirectly related elements (B and D) in a hierarchical schematic organization (Dusek & Eichenbaum 1997).

Beginning in the latter half of the twentieth century, neuroscientific research revealed that the hippocampus is the hub of a brain system that supports memory organization. This article begins with an overview of the type of memory organization that is dependent on the hippocampus, then focuses on recent analyses of hippocampal neuronal activity patterns that provide insights about the nature of memory organizations supported by the hippocampus. I then consider additional evidence that memory organization is actively controlled by the prefrontal cortex via its interactions with the hippocampus. Parallels between the findings of behavioral and physiological studies in humans and animals and the resulting conceptual advances about the organization and control of memory by these brain areas are highlighted.

THE HIPPOCAMPUS AND MEMORY ORGANIZATION

In describing the type of memory that is supported by the hippocampus, researchers have emphasized important features of memory impairment in humans with amnesia consequent to hippocampal region damage. Memory dependent on the hippocampal region has been characterized as “declarative” (Cohen & Squire 1980, p. 209) and “explicit” (Graf & Schacter 1985, p. 501), terms that highlight our capacity to remember specific events and facts through direct efforts to access memories via conscious recollection. Characterization of the cognitive processes involved in memory dependent on the hippocampus has distinguished the ability to recognize a previously experienced stimulus via recollection of the stimulus in the context of other information associated with the experience from a sense of familiarity with the stimulus independent of the context in which it was experienced (reviewed in Eichenbaum et al. 2007, Yonelinas & Parks 2007). Furthermore, Tulving (1972) distinguished episodic memory, the ability to recall specific personal experiences that occur in a unique spatial and temporal context, from semantic memory, the accumulated knowledge about the world abstracted from many experiences and not dependent on any specific event during which the information was obtained. Episodic memory is severely impaired following hippocampal damage, even under conditions in which semantic memory is relatively intact (Vargha-Khadem et al. 1997), although the acquisition of new semantic memories is also impaired following hippocampal damage (Bayley et al. 2008, Gabrieli et al. 1988, O’Kane et al. 2004).

Research has also demonstrated properties of recollection that are dependent on the hippocampus in animals. Conscious recollection, typically observed through subjective report in humans, is beyond direct access in animals. However, there are objective measures of memory performance supported by recollection in humans that have been applied to validate animal models of recollection-based memory. One approach examines recognition memory through an analysis of receiver operating characteristics (ROCs) in which subjects study a list of items and are then tested on a larger list. On the larger list, subjects identify old items that were on the original list and new items that were not. The proportion of correct identifications of old items (hits) is compared to the proportion of incorrect identifications of new items as old (false alarms) across a wide range of response biases (Macmillan & Creelman 2005). The ROC function from these data is typically characterized by two prominent dimensions that distinguish recollection and familiarity (for a detailed description, see Yonelinas 2001). Applying the same basic experimental design in an animal model, research has demonstrated that the ROC function for recognition memory in rats is similar to that observed in humans (Figure 2) (Fortin et al. 2004; for a review, see Eichenbaum et al. 2010). Furthermore, the ROCs favor recollection in rats under the same conditions that favor recollection in humans (Sauvage et al. 2008); the same is true for conditions favoring familiarity (Sauvage et al. 2010), thus validating the animal model.

Figure 2.

Receiver operating characteristic (ROC) analysis of performance on variants of recognition memory in rats. (a) Item recognition. The ROC function is characterized by both an offset in the y intercept and bowing of the ROC curve, which is strikingly similar to the ROC function for item recognition in humans (Fortin et al. 2004). (b) Associative recognition. The ROCs for item pairs are characterized by loss of the bowing of the ROC function while the offset of the y intercept is maintained, as is the case when humans are tested in recognition of word pairs (Sauvage et al. 2008). (c) Response deadline. When subjects are required to respond rapidly, the offset in the y intercept of the ROC is lost and the curvilinear shape is maintained, as is also the case in humans (Sauvage et al. 2010).

Importantly, considerable evidence indicates that the recollection component of the ROCs is differentially impaired by hippocampal damage in humans (reviewed in Eichenbaum et al. 2007, Yonelinas & Parks 2007; for an alternative perspective that focuses on the different contents of memories in recollection and familiarity, see Wixted & Squire 2011). This deficit is also observed in rats such that damage to the hippocampus in rats selectively impairs recollection-based performance, whereas lesions to another part of the medial temporal lobe (the amygdala) selectively impairs familiarity-based performance, confirming in animals the importance of the medial temporal lobe in these features of memory and providing an anatomical double-dissociation of recollection and familiarity processes (Sauvage et al. 2008, 2010; also see Bowles et al. 2007). These observations support the view that the fundamental cognitive processes that underlie recollection and its dependence on the hippocampus are conserved across species.

Notably, Mandler (1972) was among the first to distinguish the two processes in recognition memory that we now call recollection and familiarity. He argued that the most important distinction between these processes is that familiarity for an individual item occurs via the integration of featural elements that compose a single percept, whereas recollection of an item occurs via elaboration of its associates within their organizational structure. In this review, I argue that Mandler’s three organizational structures (Figure 1) provide a good characterization of the nature of hippocampus-dependent recollective memory.

Associative Organization

In ROC analyses, a demand for memory of specific associations is imposed by using a study list composed of word pairs (e.g., army–table, baseball–saddle) and then testing the ability of the subject to distinguish old pairings (army–table) from new rearranged pairings of the same words (army–saddle). This manipulation strengthens the reliance on recollection of the specific associations for the word pairs because all of the individual words are used in the study phase and are thus equally familiar in the test phase. As a consequence, the ROC function becomes exclusively recollection based in both humans (using word pairs) and animals (using odor pairs), and performance is dramatically impaired following hippocampal damage (Sauvage et al. 2008, Yonelinas & Parks 2007).

In addition to associations between specific items, Mandler (1972) also recognized associations between each item and the larger context of associations into which it fits. Thus, for example, studies on recognition typically employ highly familiar words such that the test does not ask whether one recognizes each word per se (typically all the words are highly familiar) but rather whether one recognizes each word within the context of the studied list. This feature of associative organization in recognition is particularly relevant in a naturalistic test of recognition of items in context that measures the preferential exploration of novel over familiar objects in context in humans and monkeys (Pascalis et al. 2004). In this task, the subject first briefly studies a novel object within a visual context and then, after a delay, is presented with the same object and a new object to view. Humans typically spend more time looking at a novel object than a familiar one, and this simple form of recognition depends on the hippocampus (Pascalis et al. 2004). Importantly, the same test can be applied in monkeys; these studies have shown that the novelty preference depends on the objects being presented in the same background visual context, showing that the object memory is context dependent (Bachevalier et al. 2015). Hippocampal damage also severely impairs this preferential viewing effect in monkeys (Nemanic et al. 2004, Zola et al. 2000), but this deficit occurs only in context-dependent recognition (Bachevalier et al. 2015). In a version of the test developed for rodents, subjects initially explore duplicates of a novel object in a familiar environment and then, following a delay, are presented with one of those objects and a new object replacing the duplicate. Most studies have reported no effect from hippocampal damage when the object is presented in the same context as the original experience. However, hippocampal lesions do impair the ability to identify an object taken out of its context, as is reflected by preferential exploration of a familiar object in a novel spatial context or even in a novel place in the familiar environment, or the ability to identify an object presented out of the initially experienced order of multiple objects, consistent with memory for temporal context being a defining feature of hippocampus-dependent memory (Eacott & Norman 2004, Langston & Wood 2010; see also Cohen et al. 2013).

The associative transitivity test assesses Mandler’s (1972) characterization of associative organization structure as a set of items that are directly and indirectly linked such that cuing by a subset of items supports the ability to recall the entire set, an ability known as associative inference. In this test, subjects are trained on associations between pairs of objects that share a common element (AB and BC) and then tested for the existence of the associative network (ABC) via assessment of knowledge about the indirectly related elements (AC). In both humans and animals, the hippocampus is not essential to training on individual associations (AB and BC) but plays a critical role in probe tests in which subjects must infer relations between indirectly related elements (AC) (Figure 1a; Bunsey & Eichenbaum 1996, Preston et al. 2004).

Sequential Organization

As introduced by Tulving (1972), episodic memories are defined by the temporal organization of the events that compose personal experiences. There is substantial evidence that the hippocampus is activated in association with memory for temporal order in humans (Howard et al. 2014; reviewed in Eichenbaum 2014). Numerous studies have reported hippocampal activation associated with successful memory for sequences of faces or objects, reconstruction of the order of scenes in a movie clip, identification of items out of order in a familiar sequence, and bridging of a temporal gap between ordered stimuli (Figure 1b; reviewed in Eichenbaum 2014). Correspondingly, selective hippocampal damage in humans results in deficits in remembering the order of words in a list (Mayes et al. 2001) and the order of objects visited in a virtual environment (Spiers et al. 2001) even when recognition memory for individual words and objects was intact.

As in humans, selective hippocampal damage in animals results in impairments in memory for the order of studied object stimuli even when the same animals could recognize the individual stimuli (Fortin et al. 2002, Kesner et al. 2002). In these experiments, animals are presented in each trial with a unique series of odor stimuli, then, following a delay, they are required to judge which of a pair of stimuli arbitrarily selected from the list occurred earlier. Normal rats perform well at the task, but rats with hippocampal damage fail (Figure 1b). Control tests showed that animals with hippocampal damage could distinguish and identify the individual odors on the list even when they could not remember the order in which they had appeared. This contrast strikingly reveals that memory for order is a defining feature of hippocampal memory function in animals, as it is in humans (see also Ergorul & Eichenbaum 2004).

Schematic Organization

Although many tests of semantic memory involve remembering individual facts, it is well known that factual knowledge is embedded within schematic organizations (e.g., Collins & Quillian 1969, Piaget 1928). Studies on human hippocampal function in semantic organization have focused on tasks that require the subject to learn multiple associations or choices between objects when different associations or choice pairings share common elements. The studies then test for a representation that integrates learning about all of the objects by probing for knowledge about indirect relations among elements never experienced together. In some studies, subjects learn a set of overlapping choice problems (choose A over B, B over C, C over D, and D over E) and show acquisition of a hierarchical schematic organization (A over B over C over D over E) by appropriate choices on a probe test of transitive inference between newer experience pairs (e.g., B over D). In both humans and animals, the hippocampus is not essential to learning the individual problems but plays a critical role in probe tests that reveal the establishment of a schematic organization (Figure 1c; Dusek & Eichenbaum 1997, Zalesak & Heckers 2009). In another test, called transverse patterning, subjects are tested for the ability to learn a set of overlapping pairwise choices (choose A over B, B over C, and Cover A). Humans and animals with hippocampal damage can learn two unambiguous pairs (e.g., A over B, B over C), but not the full set, which requires a circular schematic organization (Dusek & Eichenbaum 1998, Rickard et al. 2006).

HOW ARE MEMORIES ORGANIZED BY NEURAL NETWORKS WITHIN THE HIPPOCAMPUS?

To begin thinking about how neural networks might support memory organization, we can consider Hebb’s (1949) proposal about cell assemblies and phase sequences. Hebb theorized that the unit of perceptual and memory processing was the cell assembly, a set of locally interconnected neurons in which the efficacy of connections was increased when they were activated during a specific event and whose coordinated activity thus represented the concept of that event. Hebb went on to propose that associative learning is based on a linking of cell assemblies via shared neuronal elements and that such a set of overlapping cell assemblies formed a phase sequence. Furthermore, Hebb proposed that networks of concept representations can be organized through shared elements of a larger set of cell assemblies. In his generic example, Hebb described three cell assemblies that were pairwise associated by overlapping elements such that the phase sequences could support an inference between concepts in two cell assemblies that were only indirectly linked by overlapping elements of separate associates. These ideas are being realized in findings, described below, that show how neural networks in the hippocampus support memory for associated items within a context, memory for the order of serially presented items, and memory for a hierarchical structure.

How can we employ Hebb’s model to explore memory organization by cell assemblies in real neural networks? Techniques have been developed that examine the activity of populations of neurons, as measured by functional magnetic resonance imaging (fMRI) and by multielectrode single-neuron recording in an approach called representational similarity analysis (RSA) (Kriegeskorte et al. 2008). One popular version of this approach begins by constructing population vectors from the blood-oxygen-level-dependent (BOLD) signal of a large array of voxels [often called multivoxel pattern analysis (MVPA)] or from the firing rates of a set of single neurons within a brain area (Haxby et al. 2014). Population vectors taken under different experimental conditions are then compared by simple correlations (e.g., Pearson’s r) that measure the degree of similarity, which is interpreted as reflecting the extent of overlap between cell assemblies that represent specific events. RSA is now widely used in brain imaging studies to test whether activity patterns evoked during the encoding of memories are reinstated during a subsequent delay or retrieval test and whether activity patterns reflect specific memories as associated by similarity in activation patterns or as serially organized by temporally correlated patterns.

In addition, RSA has been further employed in some studies to reveal hierarchical organizations of neural network representation. To accomplish this, the measures of similarity of population vectors for different events are compared and organized to reveal the overall structure of linkages between events. When two events evoke similar population patterns, the functional networks are viewed as close in representational space, forming a very tight phase sequence. When the events evoke less-correlated population patterns, the networks are farther apart in representational space, reflecting indirectly linked cell assemblies; when the events evoke uncorrelated population patterns, the networks are independent. These measures of distances in representational space can then be compared to identify hierarchical structures of organization. This approach has been very successful in revealing the organization of perceptual categories in cortical areas using RSA on fMRI or single-neuron recording data. For example, RSA of multivoxel fMRIs of ventral temporal cortex responses has been employed to identify the hierarchical structure of a representation of the phylogenetic scale (Connolly et al. 2012), and RSA of many single-neuron responses in the temporal cortices of monkeys has been employed to identify a hierarchical organization of representations of species and body parts (Kiani et al. 2007). This approach has also been applied to the organization of memories by the hippocampus, as described in the following sections.

Neural Network Representation of Associative Organizations

Studies on human memory using fMRI have shown that activation of the hippocampus predicts successful memory encoding of face–name (Sperling et al. 2003, Zeineh et al. 2003), face–house (Henke et al. 1997), and word–word associations (Henke et al. 1999) and that even single neurons in the human hippocampus encode specific item–item associations (Ison et al. 2015). Notably, the hippocampus is activated in association with successful memory for both item–item and item–context associations, and the magnitude of hippocampal activation is correlated with the number of associations bound in the memory (Staresina & Davachi 2008).

In addition, considerable evidence shows that specific stimuli are encoded with the spatial context of their experience within the human hippocampus. For example, when human subjects recalled imagined scenes that applied to specific verbal items, the hippocampal activation reflected recall of the item and scene rather than the item alone (Davachi et al. 2003). In a more recent study using RSA, Libby et al. (2014) scanned subjects as they performed a working-memory task that demanded memory for locations of objects presented on a screen. Successful memory was associated with maintenance during the delay of the same hippocampal activation pattern observed during encoding. Supporting theories about the importance of context coding, RSA studies have shown that successful memory is associated with reinstatement of the representation of the relevant spatial or temporal context occupied by the item during initial study (Flegal et al. 2014, Kyle et al. 2015; for reviews, see Davachi 2006, Eichenbaum et al. 2007) and with increased dissimilarity of spatial and temporal context representations between memories (Copara et al. 2014).

Zeithamova et al. (2012) built on Mandler’s (1972) assertion that associative organization supports memory for indirect as well as direct associations between elements using Preston and colleagues’ (2004) paradigm, in which subjects learn overlapping pairwise associations between objects (e.g., AB and BC) from which they can make inferences between indirectly related elements (AC) (see also Bunsey & Eichenbaum 1996). Zeithamova et al. (2012) used RSA to show that learning the second, overlapping pair (BC) reinstates the specific hippocampal representation of the earlier learned pair (AB) and that this content-specific hippocampal activation signals subsequent success in judgments about the indirect association. These findings indicate that the development of associational networks depends on reinstatement of preexisting associations into which the new information is assimilated and show that the subsequent interleaved network supports novel inferences from memory. A study in which subjects learned partially overlapping associations produced a similar result; in this study, hippocampal activation predicted subsequent integration of these pairings, as revealed by generalization across never-paired but indirectly associated items (Shohamy &Wagner 2008).

Numerous studies on animals report that hippocampal neurons activate during the exploration of specific objects at particular places, suggesting that representations of events are embedded within the spatial firing patterns (place fields) of those neurons. For example, following tone-cued fear conditioning, hippocampal neurons come to be driven by the conditioned tone stimulus when the animal is within the place field of that neuron (Moita et al. 2003). In addition, in rats performing a variant of the novel object exploration task, hippocampal neurons fire in association with specific objects and their familiarity, which are embedded within the spatial firing patterns of these neurons (Manns & Eichenbaum 2009). In rats performing a context-guided object–reward association task, hippocampal neurons fire when animals sample specific objects within particular locations and spatial contexts (Komorowski et al. 2009). These and other studies (e.g., Tse et al. 2007) suggest that the hippocampus associates events within a spatial contextual framework. At the same time, when multiple events share features other than neighborliness within space, a more complex organizational structure emerges (see the section Neural Network Representation of Schematic Organizations; McKenzie et al. 2014, 2016).

Neural Network Representation of Sequential Organizations

RSA of hippocampal activation during sequence memory has revealed that pattern similarity across items in a list is associated with subsequent successful order memory (DuBrow & Davachi 2014). Ezzyat & Davachi (2014) tested whether hippocampal networks could also bridge associations between distinct events that occurred in the same or different contexts (scenes). In this task, subjects report that objects appearing across a contextual boundary are more separated in time than objects appearing within the same context, even when the actual temporal separation is the same. Pattern similarity in hippocampal activation for object cues presented across contexts was correlated with subjects reporting closer temporal proximity, suggesting that the hippocampus binds events across time independent of visual context boundaries. Another study has shown a broad gradient of similarity in hippocampal representations across long periods of time (Nielson et al. 2015), although different mechanisms may support within-experience and between-experience representations (Davachi & DuBrow 2015). In addition, Hsieh et al. (2014) reported that pattern similarity in hippocampal activation signaled the combination of object and temporal position information in sequence learning, indicating that hippocampal activation patterns encode specific items and the order in which they occur (Figure 3a). Furthermore, hippocampal activation predicts an accurate estimate of the chronological order of stimuli in a list (Jenkins & Ranganath, 2010). In studies of hippocampal neuronal activity in humans, neurons fire in sequence in association with learning (Paz et al. 2010) and memory (Gelbard-Sagiv et al. 2008) of the flow of events experienced in movie clips.

Figure 3.

Representational similarity analysis (RSA) of sequential organization in the hippocampus of humans and rats. (a) In a human fMRI (left), the correlation matrix shows pattern similarity across repetitions of combinations of object and temporal position sequences. The warmest colors, representing the highest similarity for repetition of each object/position element, gradually become cooler with the decreased similarity between elements that are successively more separated. Similarity scores (right) are also shown gradually decreasing with increasing temporal distance. Panel adapted with permission from Hsieh et al. (2014). (b) In rat single-neuron recordings, animals were required to remember an odor (A or B) that began each trial and match it to an odor presented after a delay. (Left) Idealized sequences of binned firing rates of three idealized hippocampal neurons; red indicates a high firing rate, yellow a lower firing rate, and blue no activity. To measure representational similarity for trials beginning with identical odors (e.g., odor A trials), average population vectors for odd- and even-numbered trials are cross-correlated. To measure representational similarity for trials beginning with different odors (A versus B), average population vectors for A trials and B trials are cross-correlated. These correlations were compared to correlations for random ordering of the neurons’ activity patterns. (Right) The greatest similarity occurred in trials beginning with the same odor. Less, but still above random, similarity occurred for different odors, indicating both coding of temporal organization of each trial type and coding of the general temporal structure common to all trials. Temporal coding was observed in neurons whose activity was temporally modulated but not in cells whose activity was not temporally modulated (nontime cells). Panel adapted with permission from MacDonald et al. (2013).

There is also growing evidence that memory for the flow of events in experiences is mediated directly by representations of time and order in hippocampal neurons in animals (reviewed in Eichenbaum 2014). One study showed that neural ensemble activity patterns in the hippocampus gradually change as rats sample sequences of odors and that this signal of a continuously evolving temporal context predicted success in remembering the odor sequence (Manns et al. 2007). Furthermore, several studies have identified hippocampal principal neurons that fire at a particular moment in time during a temporally structured event (Kraus et al. 2013, Naya & Suzuki 2011, Pastalkova et al. 2008). These time cells compose temporal maps of specific experiences that represent the flow of temporal context and the memories contained within, parallel to the way place cells organize events within a spatial context. In these studies, the location of the animal is held constant or firing patterns associated with elapsed time are distinguished from those associated with spatial and behavioral variables, and the firing patterns of these cells are dependent on the duration of the critical events. Time cells have been observed in a variety of behavioral paradigms that involve bridging a temporal gap, including delay periods in maze tasks, bridging of temporal gaps between associated nonspatial cues, and trace eyelid conditioning (reviewed in Eichenbaum 2014). Furthermore, some of these studies have demonstrated close links between the emergence of time cell sequences and the encoding of specific memories, as well as subsequent memory performance (Figure 3b).

Importantly, in some of these studies, the animal is immobilized, and thus space plays no role in ongoing behavior or memory. Nevertheless, the role of the hippocampus in organizing events in time extends even to spatial memories and spatial representations. Thus, in rats performing a spatial alternation task in a T-maze, place cells fire differently depending on whether the animal is executing a left-turn or right-turn trial, even as animals traverse the portion of maze where these routes overlap (reviewed in Shapiro et al. 2006). Furthermore, the activation of these route-specific spatial representations predicts accurate memory performance (Robitsek et al. 2013).

Neural Network Representation of Schematic Organizations

Since the publication of the work of Piaget (1928) and Bartlett (1932), we have known that memories are not stored in isolation but rather are integrated into schematic organizations from which we can extract both direct and indirect associations that are linked via structural rules. The simplest organizations of schematic memories are the associative organizations discussed above (Schlichting& Preston 2016, Zeithamova et al. 2012). In this section, I consider more complex schematic organizations in which the links between elements are operations rather than simple associations.

In an experiment involving the integration of experiences in a naturalistic situation, subjects studied distinct animated scenes that partially overlapped in the inclusion of specific persons, objects, or scenes of a room, and could thus eventually be integrated into connected narratives that compose the flow of the scenes in time and space (Milivojevic et al. 2015). After the successful integration of the scenes, RSA identified new patterns of hippocampal activation that were more similar than the patterns observed before the integration, indicating a merging of cell assemblies associated with the building of knowledge that integrates the scenes (Milivojevic et al. 2015). In subjects performing a more complex weather prediction task, Kumaran et al. (2009) reported hippocampal activation associated with abstract knowledge about cue relationships that applied as well to novel stimuli that were linked by the same conceptual relations.

In a yet more complex task, Tavares et al. (2015) designed a role-playing game in which their subjects moved to a new town and sought a job and apartment. To accomplish this, the participants interacted with local people through different responses allowing them to comply with a character’s demand or make demands, thereby increasing or decreasing the power of the character, and engage or not engage in personal conversation and physical interaction, thereby increasing or decreasing affiliation with the character. Thus, the composite outcomes of these social interactions positioned each character along axes of power and affiliation, constituting a vector describing the subject’s relationship to each character in social space. Tavares et al. then scanned the subjects and found that the fMRI signal in the hippocampus correlated with the vector angle, indicating that the hippocampal network identified each character’s position in social space as an interaction of their power and affiliation relations. These findings indicate that the hippocampus plays an important role in the integration of distinct social episodes into a schematic organization along specific relevant dimensions.

In animals, RSA using measures of similarity in neural population coding can also be employed to characterize the organization of multiple memories within the hippocampus. McKenzie et al. (2014) trained rats on a context-dependent object–reward association task in which the rats shuttled between two spatial contexts, in each of which the same two object stimuli (A and B) were presented in one of two positions (Figure 4a). Object A was rewarded in one context and object B in the other, such that the rats were required to use the context to guide learning and retrieval of each object–reward association. Subsequently, the animals were trained on the same task with an additional pair of objects, C, which was rewarded in the same context as A, and D, which was rewarded in the same context as B; they were then tested with both pairs of objects. To characterize the structure of the neural network representation for all 16 distinct events (four objects in each of two positions within each of two contexts), similarities between neural population firing patterns in the hippocampus were measured for all pairwise comparisons between events and a hierarchical analysis was applied to iteratively cluster event representations and reveal the organization of the memories. This analysis revealed a highly systematic organization of the hippocampal representations of distinct events (McKenzie et al. 2014). Figure 4b illustrates the relationships between representations of each of the events (x axis) as linked (y axis) by specific task dimensions. At the top of this hierarchy, events that occur in different contexts are widely separated in representational space, indicated by anticorrelation between events that occur in different contexts, positioning context as the highest superordinate dimension. Within each context-based network, events are moderately separated by positions within a context, i.e., positions are subordinate to contexts. Next, within each position representation, events that involve different reward valences are associated by positive correlation, and within each reward valence, different objects are associated. Notably, RSA revealed an emergent network representation of the organization of memories that animals acquire in the task that could not be observed from single-neuron firing patterns.

Figure 4.

Representational similarity analysis (RSA) of related object memories in the hippocampus of rats. (a) Context-guided object–reward association task. Rats choose between objects A and B in either of two positions in each context. Note that object–reward associations are opposite in the two contexts. Objects were positioned as shown within each context or in the reverse positions in different trials. (b) RSA measuring hierarchical ordering of representational similarities (y axis) of the 16 object memories (x axis). The specific task dimensions are listed on the right. In trials of the dorsal hippocampus (top graph), the animals were tested with C and D objects as well as with A and B objects, allowing the distinction of reward valences and object identities, whereas in trials of the ventral and intermediate hippocampus (bottom graph) only A and B stimuli were used, so comparisons for identical trial types were made on odd and even numbered trials. Figure adapted with permission from McKenzie et al. (2016).

Further insight into the mechanisms by which new information is integrated into an existing schematic organization comes from consideration of the course of learning in this study. RSA on the data from an early stage of training, when only objects A and B were included, showed that the basic organization of those object representations was fully established prior to learning about C and D. When C and D were learned the next day, the hierarchical organization rapidly assimilated the new memories by elaborating only the lowest level of the hierarchy (McKenzie et al. 2014). Notably, within this organization, event representations that were closest together in representational space (A and C, B and D; Figure 4b) were never experienced in the same trial. However, the proximity of their representations in the organization is likely to support strong indirect associations between these items. These findings are consistent with observations from fMRI studies in humans that reveal an integration of new memories into preexisting network representations (Milivojevic et al. 2015, Zeithamova et al. 2012).

A Topography of Organizational Structure

Other studies have extended the observations on memory organization to suggest that there may exist an anatomical topography of the specificity of memory organization within the hippocampus. In the study described in the preceding section, the populations of neurons that encode specific events in particular places in each context were recorded from the dorsal hippocampus. In contrast to those firing patterns, another study showed that the neurons in the ventral hippocampus gradually acquire more generalized representations of events within one of the contexts and that neural ensembles in the ventral hippocampus outperform those in the dorsal hippocampus in discriminating between the contexts in which events occurred (Figure 5a; Komorowski et al. 2013). RSA of population firing patterns showed greater similarity of object and position representations in the hierarchical organization of ventral as compared with dorsal hippocampal networks (Figure 4b).

Figure 5.

Topography of specificity and generality of representations along the long axis of the hippocampus. (a) The size of the place fields of hippocampal neurons (as well as the specificity of object and position coding) was graded along the long axis of the hippocampus in rats performing a context-guided object–reward association task. (Top) Outline of the two contexts and a typical place field in each panel. Warmer colors indicate higher firing rates. Blue indicates the area of each context explored. (Bottom) Areas where place fields of different size are found in the dHPC, mHPC, or vHPC. (b) The representational similarities of different scales of association were graded along the long axis of the hippocampus in humans performing an associative inference task. The posterior hippocampus had the highest similarity for one direct association (AB), the middle hippocampus had the highest similarity for both direct associations (AB and BC), and the anterior hippocampus had the highest similarity for the full network of associations (AB, BC, and AC). Panel b adapted with permission from Collin et al. (2015). Abbreviations: dHPC, dorsal hippocampus; mHPC, middle hippocampus; vHPC, ventral hippocampus.

Consistent with these observations, studies using a version of RSA to analyze functional imaging patterns in humans have shown that, in a paradigm in which subjects build a narrative from overlapping story elements (as introduced in the preceding section), specific associations (AB and BC) were most strongly represented in the posterior hippocampus (equivalent to the dorsal hippocampus in rodents) and the representation of the full network was more strongly represented in the anterior hippocampus (equivalent to the ventral hippocampus in rats) (Figure 5b; Collin et al. 2015). Similarly, in the associative inference paradigm, RSA showed that the posterior hippocampus distinguishes specific learned associations (AB and BC) by using different representations for A and C, whereas the anterior hippocampus integrates these associations by using similarity of A and C representations (Schlichting et al. 2015). Furthermore, whereas the posterior hippocampus is differentially activated during retrieval of the specific details of events in autobiographical memories, the anterior hippocampus is differentially activated during the retrieval of the general context of those memories (Evensmoen et al. 2013).

These and related findings have led to the recent proposal that the dorsal-ventral (posterior-anterior in humans) axis of the hippocampus may contain a topography of more specific to more general features of memories (for reviews, see Poppenk et al. 2013, Strange et al. 2014). Notably, the outputs of the hippocampus to the prefrontal cortex arise in the ventral (anterior in humans) hippocampus, suggesting that these generalized representations provide the prefrontal cortex with information that characterizes the context of a set of related memories. This information may play a major role in prefrontal control of memory organization, as will be discussed in the following sections.

THE COGNITIVE CONTROL OF MEMORY BY THE PREFRONTAL CORTEX

The organization and retrieval of memories is not an automatic or passive product of experience but instead involves distinct control processes that actively guide the encoding and retrieval of memories. Among these active processes are prefrontal–hippocampal interactions, which may direct the structure and selective retrieval of information from memory organizations. In the following sections I consider evidence that interactions between the prefrontal cortex and hippocampus are essential to memory and that the role of the prefrontal cortex is to direct the encoding and retrieval of memory representations in the hippocampus.

The Prefrontal Cortex and Hippocampus Work Together in Support of Memory

There are strong anatomical connections between the prefrontal cortex and hippocampus consistent with the view that the prefrontal cortex and hippocampus operate interactively in support of memory (Figure 6a) (for a review, see Simons & Spiers 2003). In humans, functional imaging studies have demonstrated correlations between hippocampal and prefrontal activity in a variety of memory tasks (e.g., Brassen et al. 2006, Bunge et al. 2004, Dickerson et al. 2007, Zeithamova et al. 2012; for a review, see Ritchey et al. 2015). In monkeys, synchronization of theta and beta band oscillations is modulated by trial outcome in a paired associate learning task (Brincat & Miller 2015). In rats, several studies have shown that accurate memory performance in delayed nonmatching to sample is associated with synchronization of prefrontal neural ensembles with hippocampal theta activity (Benchenane et al. 2010, Hyman et al. 2005, Sigurdsson et al. 2010).

Figure 6.

Model of prefrontal–hippocampal interactions in memory. (a) The PFC receives direction projections from the vHPC (in rats; anterior HPC in humans) and projects indirectly (via the perirhinal and entorhinal cortices) to the HPC. In this model, when one is cued by context to recall memories, contextual cues are processed by the vHPC, which sends this information to the PFC, which then biases the retrieval of the context-appropriate memories in the dHPC. (b) Direction of the flow of information was determined by correlating the amplitude of the theta rhythm in the HPC and PFC across a range of time shifts between the two signals. These correlations reveal that during the exploration of the spatial context, information processing in the HPC leads that in the PFC, whereas during the sampling of the objects, the flow of information reverses, such that the PFC leads the HPC. Abbreviations: dHPC, dorsal hippocampus; HPC, hippocampus; PFC, prefrontal cortex; vHPC, ventral hippocampus.

Demonstrating the importance of these physiological interactions, lesions that disconnect the prefrontal cortex from the hippocampus impair object–place and object–order memory (Barker et al. 2007). Similarly, lesions that disconnect the prefrontal cortex from the hippocampus result in severe impairment in conditional visual discrimination learning in monkeys (Parker & Gaffan 1998). Taken together, these studies provide evidence from multiple approaches in humans, monkeys, and rats that the prefrontal and hippocampal regions interact in the service of memory. But what is the nature of the conversation between the prefrontal cortex and hippocampus reflected in the physiological and functional interactions between these areas?

The Prefrontal Cortex Performs a Specific Role in the Control of Memory

Considerable converging evidence indicates that the prefrontal cortex contributes to memory through cognitive or strategic control over memory retrieval processes within other brain areas (Anderson & Weaver 2009, Buckner & Wheeler 2001, Dobbins et al. 2002, Kuhl & Wagner 2009, Miller & Cohen 2001, Moscovitch 1992, Postle 2006, Ranganath & Blumenfeld 2008). In humans, evidence indicates that areas of the prefrontal cortex are involved in establishing the organization of relationships among memories and in monitoring retrieval, and functional imaging data have indicated that the memory impairment caused by prefrontal damage can be characterized as a deficit in the suppression of interfering memories. Consistent with this view, patients with prefrontal damage do not have severe impairments in standard tests of event memory. However, deficits resulting from prefrontal damage are apparent when memory for target information must be obtained under conditions of memory interference or distraction. For example, individuals with prefrontal damage successfully learn a set of paired associations (AB) but are severely impaired in learning new associates of original elements (AC), and the impairment is marked by intrusions of the original associations (Shimamura et al. 1995). Similarly, in prefrontal cortex–damaged patients learning two lists of unrelated associations, memory for one list is compromised by intrusions from the other, suggesting that the prefrontal cortex controls memory retrieval by selecting memories relevant to the current context and suppressing irrelevant memories (Depue 2012).

Although the specific homologies between human and animal prefrontal areas are not clear (Vertes 2006), there is substantial evidence that the role of the prefrontal cortex in the control of memory is conserved across mammalian species. Studies on monkeys and rats have extended the classic findings on prefrontal function in set switching determined via the Wisconsin Card Sorting test in humans, showing that prefrontal damage results in severe impairment in animals in switching between learned perceptual sets (Birrell & Brown 2000, Dias et al. 1996). Several studies have also shown that the rodent prefrontal cortex is critical to rule-guided switching between memory strategies (Brown & Bowman 2002, Durstewitz et al. 2010, Marquis et al. 2007, Ragozzino et al. 2003, Rich & Shapiro 2007).

Complementary studies on neuronal activity patterns support the idea that the prefrontal cortex acquires representations of behavioral contexts that determine appropriate memory retrieval. Miller and colleagues (reviewed in Miller 1999, Miller & Cohen 2001) have demonstrated in monkeys the importance of the prefrontal cortex in acquiring prefrontal neural representations that guide perceptions, actions, and cognitive rules. In rats, neuronal ensembles in the prefrontal cortex fire distinctly in different behavioral contexts (Hyman et al. 2012), and patterns of neural activity are altered following a change in contingencies (Durstewitz et al. 2010, Karlsson et al. 2012). In a study directly related to the strategy-switching experiments cited above, Rich & Shapiro (2009) observed that prefrontal neuronal activity patterns predict switching between remembering place and response strategies in the domain of spatial memory. These and other findings have led to a view that the hippocampus creates organizations of memories, whereas the prefrontal cortex extracts a common set of ongoing task rules that govern the selection of memories within the hippocampal organization.

The Mechanism of Prefrontal Control Is Suppression of Competing Memories

Additional evidence suggests that the prefrontal cortex employs contextual representations to control the retrieval of detailed memories in the hippocampus by suppression of context-inappropriate memories. For example, in a study that examined intrusion errors in recognition memory, researchers found that prefrontal damage resulted in a selective increase in false alarms from previous learning in recollection-like memory in rats performing a recognition task, similar to the observation of intrusion errors by humans in the AB versus AC problem described in the preceding section (Farovik et al. 2008). Further evidence for suppression of context-inappropriate memories comes from the studies described in the section Neural Network Representation of Schematic Organizations in which rats used each of two spatial contexts to guide retrieval of otherwise contradictory object–reward associations. In rats performing this task, neurons in the dorsal part of the hippocampus encode these memories as selective firing to specific objects in particular places in each spatial context (Figure 4b; Komorowski et al. 2009). However, when the prefrontal cortex is inactivated, dorsal hippocampal neurons indiscriminately retrieve both appropriate and inappropriate object memory representations (Navawongse & Eichenbaum 2013). This observation indicates that the hippocampus is capable of retrieving memories even in the absence of prefrontal input but that the role of the prefrontal cortex is to select the appropriate memory for that context by suppressing alternative representations.

Bidirectional Hippocampal–Prefrontal Interactions Support Context-Guided Memory

The ventral hippocampus projects directly to the prefrontal cortex (Swanson et al. 1978), providing a powerful and immediate route for hippocampal representations of meaningfully distinct spatial contexts to arrive in the prefrontal cortex. This observation, combined with the findings described in the preceding section, suggests a model of bidirectional hippocampal–prefrontal interactions that support memory encoding and context-dependent memory retrieval (Figure 6a). According to this model, closely related events that occur within a single context, as well as environmental cues that define the context, are processed by the ventral (anterior in humans) hippocampus as a collection of features and events that define the particular context in which those events occur. This context-defining information is sent via direct projections to the prefrontal cortex, where neural ensembles develop distinct representations that can distinguish contextual rules during the course of learning. When subjects are subsequently cued with the same context, ventral hippocampal signals carrying the contextual information are sent directly to the prefrontal cortex, which then engages the appropriate rules to support retrieval of the context-appropriate memory representations in the dorsal (posterior in humans) hippocampus by suppressing context-inappropriate memories. The dialog between the hippocampus and prefrontal cortex suggested by this model is supported by the recent observations of functional connectivity between the hippocampus and prefrontal cortex in rats performing the context-guided object memory task described in Figure 6b. When rats enter a spatial context, hippocampal networks send information about the context to the prefrontal cortex, whereas when the animal subsequently evaluates the object choices, prefrontal networks send information to the hippocampus, presumably to guide retrieval of the appropriate memories (Place et al. 2016).

Prefrontal Control Over the Development and Updating of Memory Organizations

In addition to its role in retrieving memories, cognitive control by the prefrontal cortex may also support the development and updating of memory organizations by guiding the integration of new memories into the organization of preexisting knowledge (Preston & Eichenbaum 2013). McClelland et al. (1995) proposed that new memories are initially represented within the hippocampus and subsequently become interleaved into the semantic organization of existing related memories in the neocortex. This interleaving process incorporates new memories and typically requires modification of the preexisting network structure to add the new memories, consistent with Piaget’s (1928) views on assimilation of new information and accommodation of the existing knowledge structure to integrate the new information.

Several studies have reported that the integration of memories into knowledge organizations relies on the prefrontal cortex. In particular, the prefrontal cortex plays an essential role in the acquisition of schematic organizations and in making inferences between indirectly related memories (DeVito et al. 2010). Tse et al. (2011) also showed that prefrontal areas are involved in rapid assimilation of new food–location associations into an organization of preexisting memories of other food locations in the same environment. Conversely, Richards et al. (2014) showed that prefrontal inactivation blocked sensitivity of hippocampal representations to reorganization when the subject was presented with conflicting spatial patterns. In humans, the prefrontal cortex is activated during assimilation of new memories within a schematic organization (Wendelken & Bunge 2010); is essential to transitive inference (Koscik & Tranel 2012); and is activated during assimilation of new memories, inference between indirectly related memories (Zeithamova & Preston 2010, Zeithamova et al. 2012), and updating of a memory organization (Milivojevic et al. 2015). Together, the findings in rodents and humans indicate that the prefrontal cortex plays an important role during integration of new information as well as memory retrieval from schematic organizations.

ORGANIZATION AND CONTROL OF MEMORY: CONCLUDING COMMENTS

Organization of Memories by the Hippocampus

The findings on memory organization discussed in this review share much in common with Tolman’s (1948) conception of a cognitive map as a systematic organization of information across multiple domains of life, supporting flexible expression of acquired knowledge in purposeful behavior. O’Keefe & Nadel’s (1978) model introduced the idea that cognitive maps are supported by the hippocampus, but later work has focused entirely on how the hippocampus specifically supports geographical maps as they are employed to navigate physical environments. However, Tolman’s conceptualization provides a framework for research that would reveal a more comprehensive understanding of hippocampal function in the cognitive maps that organize memories more generally, and several studies have provided evidence that the notion of cognitive maps extends to a broad range of dimensions (Buzsaki & Moser 2013, Eichenbaum & Cohen 2014, Milivojevic & Doeller 2013, Schiller et al. 2015). These findings have inspired a reconciliation of divergent views about the role of the hippocampus in spatial navigation versus memory by revealing the common mechanisms that underlie both functions (Eichenbaum & Cohen 2014, Schiller et al. 2015).

It is also important to consider that all of the forms of memory organization described in this review may be based simply on spatial and temporal contiguities that underlie organizational structures in general. In the 1990s, my colleagues and I proposed that the nature of hippocampal representation is fundamentally relational and can be envisioned as a network of memories—a memory space—that links conceptually distinct events to form a framework of relevant associational dimensions, including spatial and temporal relationships and, potentially, all relationships between events that we experience (Eichenbaum et al. 1992, 1999). Furthermore, we proposed that activation of a subset of elements within a relational network leads to activation of other elements, including those only indirectly connected with the original activated elements. This gives rise to an important property of hippocampus-dependent memory: the ability to use memory flexibly to guide performance in diverse situations outside repetition of the learning event, a property we called representational flexibility. These properties are readily apparent in the examples of associative, sequential, and schematic organization described in this review.

Within this framework, the studies on spatial firing characteristics of hippocampal region neurons in animals can be viewed as both a metaphor and an example of a memory space and can shed light on the mechanisms of this memory space. Thus, the findings on coding of location, direction, speed, and borders by hippocampal region neurons have led some to suggest that the hippocampal system reconciles path integration signals from movement through the representation of physical space with current viewpoints to support navigation (e.g., Cheung et al. 2012, Knierim et al. 2013, Moser et al. 2008); these concepts could apply as broadly to the flexible expression of organized knowledge in any memory organization. Within the framework proposed here, the same computations may reconcile thoughtful movement through a memory space with perception of current events in support of flexible prediction of succeeding events.

Control of Organization and Retrieval from the Memory Organization

The mechanisms of prefrontal control over memory organization could operate via the resolution of conflicts between new experiences and existing memory organization. When new experiences occur, they usually conflict in some way with preexisting associations. For example, in the associative inference paradigm (Figure 1a), the preexisting association between B and A is challenged when the subject learns that B is now also (or instead) associated with C. Success in integration of new memories into an already established organization requires some degree of reorganization of the existing structure. The prefrontal cortex could support memory integration by generating the most relevant established structure for reconciliation of the conflicting new information. The key to understanding hippocampal and prefrontal contributions to memory organization therefore lies in understanding how these regions support the representations of new events based on the degree to which those events relate to prior knowledge and how the conflicts in new and existing knowledge are reconciled (Preston & Eichenbaum 2013).

SUMMARY POINTS.

1. Two major processes that characterize declarative memory are the elaborate organization of networks of memories supported by the hippocampus and the control of encoding and retrieval of information in the organization by the prefrontal cortex.

2. The hippocampus plays an essential role in memory organization in humans and animals. Correspondingly, networks of neurons in the hippocampus encode associations between events in context, sequential associations that characterize episodes, and complex (e.g., hierarchical) organizations of related memories.

3. The prefrontal cortex supports the cognitive control of memory by developing representations that employ current contextual cues to select context-appropriate memory representations, primarily by suppressing context-inappropriate memories.

4. Context-appropriate retrieval may be supported by a dialog between the hippocampus and prefrontal cortex in which the ventral (in rodents; anterior in humans) hippocampus sends contextual information to the prefrontal cortex, which then identifies contextual rules that direct retrieval of specific memory representations in the hippocampus.

FUTURE ISSUES.

1. The full range of memory organization supported by the hippocampus is unknown. Current work highlights spatial and temporal organization, but other findings suggest that the hippocampus supports any systematic dimension of organization (Schiller et al. 2015). In particular, it remains to be determined whether spatial mapping and memory views of hippocampal function can be merged by identifying firing properties of hippocampal region neurons that code direction, speed, and other dimensions of movement through a memory organization.

2. The structure of representations of contexts and rules by networks of neurons in the prefrontal cortex is poorly understood. Current evidence shows distinct network patterns associated with ongoing rules, but the way in which features of rules are represented is unknown.

3. Although there is considerable evidence of a dialog between the hippocampus and prefrontal cortex, it remains unclear how network representations in these areas interact to influence one another. It has yet to be determined how a hippocampal context representation changes rule-related activity in the prefrontal cortex and how prefrontal rule-related activity influences hippocampal memory representations.