Time and space in the hippocampus

Abstract

It has been hypothesized that one of the functions of the hippocampus is to enable the learning of relationships between different stimuli experienced in the environment. These relationships might be spatial (“the bathroom is about 5 m down the hall from the bedroom”) or temporal (“the coffee is ready about 3 minutes after the button was pressed”). Critically, these spatial and temporal relationships may exist on a variety of scales from, say a few hundred milliseconds up to minutes. In order to learn consistent relationships between stimuli separated by a variety of spatial and temporal scales using synaptic plasticity that has a fixed temporal window extending at most a few hundred milliseconds, information about the spatial and temporal relationships of distant stimuli must be available to the hippocampus in the present. Hippocampal place cells and time cells seem well-suited to represent the spatial and temporal locations of distant stimuli in order to support learning of these relationships. We review a recent computational hypothesis that can be used to construct both spatial and temporal relationships. We suggest that there is a deep computational connection between spatial and temporal coding in the hippocampus and that both serve the overarching function of learning relationships between stimuli—constructing a “memory space.”

1. Introduction

The modern era of research on hippocampal function began with Scoville and Milner’s (1957) identification of the hippocampus as critical to memory in humans. Subsequent pioneering studies that sought the basis of hippocampal memory function in the firing properties of hippocampal neurons revealed the existence of “place cells”, neurons that fire when a rat occupies a particular location in its environment (O’Keefe & Dostrovsky, 1971). This observation, plus studies showing selective deficits in spatial learning following hippocampal damage, inspired the notion that the hippocampus supports memory by creating a cognitive map of the physical environment into which memories are encoded (O’Keefe & Nadel, 1978). Decades of research since then have sought to characterize the organization of the cognitive map. Recent research in this direction has focused on how the hippocampus and neighboring interconnected structures supports navigation through space (McNaughton, Battaglia, Jensen, Moser, & Moser, 2006; Moser, Kropff, & Moser, 2008), indicating this line of research is losing sight of the well established more general role of the hippocampal system in memory (Eichenbaum & Cohen, in press). Here we will review some of the key features of hippocampal place cells and suggest that they reflect a more general mechanism of temporal as well as spatial organization of events, and that this broader function is the basis of hippocampal function in memory.

1.1. Coding across space and time in the hippocampus

The hippocampal place code provides an exquisitely detailed representation of the animal’s location in allocentric coordinate system at multiple spatial scales (Wilson & McNaughton, 1993; Solstad, Boccara, Kropff, Moser, & Moser, 2008). Eichenbaum (2001) hypothesized that the spatial map in the hippocampus is a special case of a more general function in representing relationships among all of the task-relevant stimuli, including both spatial and non-spatial variables (see also Eichenbaum, Dudchenko, Wood, Shapiro, & Tanila, 1999). If the hippocampus captures relationships among stimuli separated by macroscopic quantities of space and time, and if plasticity depends on simultaneous activation within a fixed temporal window, there must be some way to bridge these gaps. Put another way, if the hippocampus is to facilitate learning of a relationship such as, “these nuts are stored 20 cm from the big oak tree” or “the shock will come 5 s after the tone” then it must have some facility to represent these relationships. At the location where the nuts are being stored, the hippocampus must have some ability to describe the distant location of the oak tree; at the time the shock arrives the hippocampus must have some way to represent that a tone was experienced a certain time in the past. Once the relationship of the distant stimulus is represented, it is relatively straightforward to associate that representation to the other stimulus—the nut in the spatial example or the shock in the temporal example. In this way, the spatial and temporal representations can be used to cue the response. For instance, when the animal is 15 cm from the oak tree, the similarity between the current spatial representation and the stored representation can cause activation of the representation for the nuts. Similarly, if one is looking for nuts, nuts may be used as a cue to recover the spatial representation coding for “20 cm from the oak tree.”

1.1.1. Spatial coding reflects spatial relationships to distant stimuli

These considerations suggest that the neural activity in the hippocampus at a particular time is not solely, nor even predominantly, caused by the stimuli available in the environment at that moment. For instance consider the experiment performed by Gothard and colleagues (Gothard, Skaggs, & McNaughton, 1996). They trained a rat to run back and forth on a linear track. Each of the journeys along the track started from a goal box. Across journeys the location of the box was moved relative to the track and the room coordinates. On journeys away from the box, place cells fired at a constant distance from the start box, moving across journeys relative to the room frame. Consider a place cell that fires 50 cm from the start box. Because the location of its firing is controlled by the box, this means that a stimulus that was experienced perhaps several seconds in the past—the box—controls firing of the cell in the present. Moreover, because the phenomenon also holds in the dark (Gothard, Hoffman, Battaglia, & McNaughton, 2001), we can be certain that the firing in this experiment is not dependent on the animal being able to see the goal box from its current location. Burgess and O’Keefe (1996) observed similar control of firing in a two-dimensional environment; as the shape of the enclosure changed, place fields in the center of the environment changed location to maintain their relationship to one or more of the walls of the environment (see also Hartley, Burgess, Lever, Cacucci, & O’Keefe, 2000; Barry et al., 2006; Lever, Burton, Jeewajee, O’Keefe, & Burgess, 2009).

These place cells are apparently not coding for a place per se but a spatial relationship relative to a landmark, or set of landmarks. That is, a place cell in the Gothard et al. (1996) experiment is not coding for a location, but rather something more like “50 cm from the goal box”—a spatial relationship. Note that without establishing stimulus control, simply observing that a place cell fires in a particular location within an environment provides no information about the stimulus and relationship (or relationships) that it is being used to represent (Figure 1).

Figure 1.

Firing of hippocampal cells must be controlled by events distant in space and time. Left: A place cell fires in a location within an environment with four landmarks, A B C D. It is unclear whether the cell fires because it is SE of A, NE of B, SW of C, NW of D, or any conjunction of those variables. In order to determine this, it would be necessary to systematically move the landmarks and establish control over the cell’s firing. Right: A cell fires during a temporal sequence of events A B C D E. It is not clear whether the cell fires because D has just been presented, because it is one time step after C, or because it is two time steps after B. If the sequence is consistent and well-learned it is also possible that it is firing because E is predicted. In order to determine which of these relationships is responsible for the cell firing, it would be necessary to present the stimuli many times in many different orders to establish stimulus control of the firing.

1.2. Temporal coding may reflect temporal relationships of temporally remote stimuli over long time scales

The principle that currently-available stimuli do not control hippocampal firing should also inform non-spatial firing as well. A large body of evidence suggests that the hippocampus constructs temporal and sequential relationships in addition to spatial relationships (e.g., Bunsey & Eichenbaum, 1996; Clark & Squire, 1998; Westmacott & Moscovitch, 2001). It is clear that neural activity supporting memory changes gradually over long periods of time. Although there are examples of some cells in the MTL that respond to non-spatial stimuli currently being perceived, such as the celebrated “Jennifer Aniston” cells (Quiroga, Reddy, Kreiman, Koch, & Fried, 2005), these cells coexist with other cells that show very slowly-changing activity. Figure 2 illustrates this phenomenon in recordings from human epileptic patients during performance of an item recognition experiment using pictures of faces and scenes (Howard, Viskontas, Shankar, & Fried, 2012; Viskontas, Knowlton, Steinmetz, & Fried, 2006). Note that the cells fire at an elevated level over periods of time much longer than the time any individual stimulus is presented. These cells are from human MTL; autocorrelated signals have been obtained from animal studies in a variety of brain regions (Manns, Howard, & Eichenbaum, 2007; Mankin et al., 2012; Hyman, Ma, Balaguer-Ballester, Durstewitz, & Seamans, 2012; Kim, Ghim, Lee, & Jung, 2013). While autocorrelation itself may result from any number of causes, including recording artifacts and stochastic noise, additional evidence suggests that gradually-changing firing could reflect, at least in part, memory for temporally remote events.

Figure 2.

Examples of neural firing correlated over long periods of time. Each panel gives the spikes fired over a recording session. Each row gives a different trial of a continuous recognition task, aligned to the presentation of the stimulus (vertical line). At the bottom a PSTH is shown. On the right of each panel, the cell’s firing rate for each trial is shown. Note the robust autocorrelation—the firing of each cell changes gradually over many many trials. The entire experiment takes about 300 s. a. This cell shows autocorrelated firing over a few trials. Each trial is approximately 4 s, so this constitutes a change over about a dozen seconds. b. This cell shows autocorrelation over about 10 trials, corresponding to several dozen seconds. c. This cell shows autocorrelation over a few dozen trials, corresponding to a few minutes. After Howard, et al. (2012).

1.2.1. Time cells code for when an event took place

If the hippocampal ensemble changes gradually over long periods of time, this creates the possibility that the stimulus that caused the cell to begin firing was experienced far in the past, in much the same way that the place code can reflect stimuli experienced remotely in time. Time cells provide an excellent illustration of this phenomenon. Pastalkova, Itskov, Amarasingham, and Buzsaki (2008) trained rodents to navigate around a maze, stopping on each lap to run for a period of time in a running wheel. As expected, some hippocampal neurons fired in a circumscribed set of locations along the maze. They also observed that while running on the wheel, some hippocampal neurons fired during circumscribed periods of time. Similar results have been observed when the animal is not running at all (MacDonald, Lepage, Eden, & Eichenbaum, 2011; MacDonald, Carrow, Place, & Eichenbaum, 2013) and when the animal is running on a treadmill with controlled velocity (Kraus, Robinson, White, Eichenbaum, & Hasselmo, 2013).

These “time cells” carry information about the timing of past events. Figure 3a shows the firing of a set of time cells during the delay interval of an object memory task. The set of cells that fire early in the interval are different than the set of cells that fire during the middle of the interval which are, in turn, different than the set of cells that fire near the end of the interval. The ensemble of neurons carries information about the time since the beginning of the delay interval. The delay interval in this experiment began with a discrete event—the closing of a door to the enclosure where the animal waited during the delay. In these recordings, the accuracy of the timing information decreases as the delay interval proceeds (Howard et al., 2014). This can be seen by the spread in the “time fields” later in the delay interval. Kraus et al. (2013) observed a similar spreading. In addition, they were able to show that the distribution of peaks was not uniform across the delay interval. In addition, time cells can show history-dependence in which they fire differentially based on preceding events (Gill, Mizumori, & Smith, 2011; Pastalkova et al., 2008).

Figure 3.

A relational memory code in the hippocampus could rely on temporal memory representations. a. Neurons in the rodent hippocampus during the delay of an object memory task. The cells are sorted according to their peak of firing. Across cells, the time into the delay interval can be reconstructed. After MacDonald, et al., (2011). b. Non-spatial “splitter cells” retain information about past events. Rats were presented with sequences of odors with overlapping subsequences such as M N A B C O P and W X A B C Y Z. Some cells responded differentially to the odors in the subsequence depending on which sequence they were presented in. This cell fires more to B and C when they were preceded by the other odors of sequence 1 than when they were preceded by the other odors of sequence 2. After Ginther et al., (2011).

1.2.2. Hippocampal coding for the identity of temporally distant stimuli

Time cells may also be able to code for the time since specific non-spatial stimuli were experienced. A few of the time cells in the MacDonald et al. (2011) experiment coded for the identity of the object presented during the delay. MacDonald et al. (2013) observed different sequences of cells firing during the delay after an odor was presented in a recognition task, although the overall firing rates were quite low. Although it is not yet completely clear to what extent time cells carry information about preceding stimuli, there is abundant evidence that firing in the MTL carries information about non-spatial stimuli that occurred many seconds in the past. Figure 3b shows a non-spatial “splitter cell” observed by (Ginther, Walsh, & Ramus, 2011). In this experiment, the rat was presented one of two series’ of odors with an overlapping subsequence. Let us denote sequence 1 as M N A B C O P and sequence 2 as W X A B C Y Z, with odors A B C being common to the two sequences. During the time when the overlapping odors are presented, the locally available information is identical; only the past history of odor presentation distinguishes the two sequences from one another. Some hippocampal neurons fired differentially during the overlapping sequence (e.g., Fig. 3b), reflecting a sensitivity to previous stimuli. Hsieh, Gruber, Jenkins, and Ranganath (2014) observed analogous findings from humans using fMRI. In addition, Yakovlev, Fusi, Berman, and Zohary (1998) observed gradually-decaying stimulus-specific firing that persisted across the intertrial interval of a DNMS task in monkey perirhinal cortex. In humans, (Howard et al., 2012) showed that the gradually-changing representation in the MTL (Figure 2) was indeed caused by the stimulus presented to the patient. This rules out the possibility that the autcorrelation is purely a recording artifact or stochastic noise; the gradually-changing representation is at least in part a signal that carries information about stimuli that were experienced in the past.

While the significance of slowly-changing firing, as shown in Figure 2 is not yet clear, these findings raise the intriguing possibility that the representation carries information about the time of past events. Representations in the MTL change gradually over at least many minutes (Manns et al., 2007; Hyman et al., 2012; Mankin et al., 2012; Howard et al., 2012). Thus far, time cells have not been observed firing over more than a few seconds. However, human and animal memory shows similar properties at a variety of time scales (Balsam & Gallistel, 2009; Glenberg et al., 1980; Howard, Youker, & Venkatadass, 2008; Lejeune & Wearden, 2006; Wearden & Lejeune, 2008; Lewis & Miall, 2009), suggesting that the slow changes in the neural representation could reflect a gradually-changing representation of history that supports behavior over a range of time scales (Howard & Eichenbaum, 2013; Howard, Shankar, Aue, & Criss, in press).

2. A computational approach to spatiotemporal context in the hippocampus

In order for the hippocampus to learn consistent spatial and temporal relationships, it must have the ability to represent temporal and spatial position of distant stimuli. Howard et al. (2014) described a computational mechanism to do just this. We start by describing, in a qualitative manner, how the model is able to represent temporal information, then generalize this ability to spatial relationships. After this description, we present a simulation illustrating the computational model’s ability to describe the continuity between temporal and spatial variables by describing experimental results from Kraus et al. (2013).

2.1. A method for computing scale-invariant spatial and temporal relationships

The goal of the computational model introduced by Shankar and Howard (2012, 2013) is to, at each moment, estimate the history of stimuli leading up to the present. The history is encoded as a conjunctive representation of what stimulus happened at each time in the past. This timeline is encoded by a set of neurons each of which are activated when a particular stimulus was experienced a certain time in the past. Presenting a stimulus causes a sequence of neurons to be activated as the stimulus recedes further into the history (Fig. 3a). The “time fields” of the neurons are spread out over a range of times, with time fields representing times further in the past showing more spread, consistent with experimental findings (Howard et al., 2014; Kraus et al., 2013; MacDonald et al., 2013). As a consequence of this spread, the model represents distant events with less temporal accuracy. It turns out that, in the model, the decrease in accuracy is precisely scale-invariant (Shankar & Howard, 2012), a property that is very useful in explaining behavioral properties of learning and memory (Balsam & Gallistel, 2009; Chater & Brown, 2008; Gallistel & Gibbon, 2000; Howard et al., in press).

Constructing a representation of temporal history with these properties is computationally non-trivial (Shankar, 2014). The output of the model is generated from an intermediate state. The intermediate state in the temporal case is a set of leaky integrators. Each cell in the intermediate state accepts the input present in the environment at that moment and then decays gradually as time passes; each of the cells accepting input from the same stimulus decay at different rates. The model can be generalized from coding time to space by changing the rate of decay to depend on the animal’s velocity in a particular direction (Howard et al., 2014). In this case, the intermediate state is a path integrator. A variable α controls the behavior of each of the cells. If α is a constant over time, then that cell is a leaky integrator and the output constructed from that intermediate state will be a temporal history. Cells in this representation code for the time since a stimulus was encoded. If α is set at each moment to be the velocity of the animal, then that cell is a path integrator and the output constructed from that intermediate state will be a spatial representation. Cells in the spatial representation code for the net distance traveled, along some direction, since a stimulus—here a landmark—was encountered in the environment. This framework can be extended further to also generate representations of sequential position as well as temporal and spatial relationships (Howard et al., 2014).

Because an animal moves around an environment in smooth trajectories, space and time are highly correlated with one another under normal circumstances. Consider a linear track. If an animal starts from one end and travels to the other with constant velocity, position along the track and time since the departure are perfectly confounded with one another. Observing a cell that fires during a circumscribed region of the track cannot determine whether that cell is a “place cell” or a “time cell.” In order to systematically dissociate position from time per se, it is necessary to establish experimental control of velocity.

2.2. Demonstration: The continuity between spatial and temporal coding

Kraus et al. (2013) established control over velocity by having animals run on a treadmill. They varied velocity across trials and examined firing as a function of both time and of distance traveled on the treadmill. They characterized the sensitivity of each cell to distance traveled compared to time since the treadmill started. Some cells responded predominantly to position; others predominantly to time. Critically, they found a continuity of sensitivity to these variables recording from the same population of cells; many cells showed some sensitivity to both time elapsed and distance traveled.

This continuity between spatial coding and temporal coding can be readily explained in the computational framework we have described. Figure 4 shows simulated results of the constant distance condition of the Kraus et al. (2013) experiment. We started the interval with an input, corresponding to the start of the treadmill. Then, the simulated animal ran on the treadmill at one of three speeds until a certain distance was traversed on the treadmill. The treadmill advanced at different speeds. The cells in the intermediate state were allowed to evolve with their value of α set according to α = a + b s, where s is just the speed on that trial. The activity of a representative cell in the output representation is shown at each moment of trials run at the three speeds as a function of time (left) or distance traveled along the treadmill (right).

Figure 4.

A continuity between spatial and temporal coding. We simulated results from the Kraus et al., (2013) experiment using the computational framework introduced in Howard et al., (2014). In the simulated experiment, the rat ran on a treadmill at a constant speed (0.75, 1, or 1.5) each trial. The trials continued for different durations until 30 units of distance had been traversed. In each panel we plot the activity of the cell as a function of time (left) and as a function of distance traveled (right). a. A “time cell” that codes for time since the treadmill started. We set a close to 1 and b close to zero (with a slight offset so the different lines would be visible). Note that the response of the cell is almost completely controlled by time (left) whereas the activity as a function of position varies greatly depending on the speed (right). b. A cell showing coding intermediate between a temporal and spatial coding. Here a = 0.5 and b = 0.5. c. A “distance cell” that codes for elapsed distance since the treadmill started. We set a close to 0 and b close to 1 (with a slight offset so the different lines would be visible). Note that the response of the cell is almost completely controlled by position (right) whereas the activity as a function of time varies greatly depending on the speed (left).

Figure 4a shows the results for a cell whose intermediate representation was dominated by temporal information, with a set close to 1 and b set close to zero. The firing of this cell is controlled by the time since the treadmill running started and does not depend directly on the distance traveled. It is fair to call the cell in Figure 4a a time cell. In contrast, the cell in Figure 4c is dominated by spatial information, with a set close to zero and b set close to 1. This cell fires at different times depending on the speed of the trial (left) but its firing is well controlled by the distance traveled on that trial (right). Interestingly, the model a b also admits hybrid cells that are partially modulated by temporal information and partially modulated by spatial information. The cell in Figure 4b shows a cell with a = b = 0.5. This cell is neither a pure time cell nor a pure distance cell, but is intermediate between the two. By choosing different values of a and b we can generate a cell with any degree of relative sensitivity to temporal and spatial information. The continuity between temporal and spatial information observed by Kraus et al. (2013) amounts to being able to set a and b to a wide variety of values.

2.3 Exploiting spatial and temporal representations to behaviorally express relationships among stimuli

The computational framework described here is able to construct representations of distant stimuli. Here we discuss how one can use this spatiotemporal representation to build relationships between different stimuli that can be expressed behaviorally. This approach uses two forms of association which we refer to context-to-item binding and item-to-context binding. Each of these associations are assumed to depend on synaptic plasticity although the mechanisms for the two forms of association may be quite different.

Association is a longstanding theme in computational models of memory performance (e.g., Murdock, 1982; Raaijmakers & Shiffrin, 1980; Humphreys, Bain, & Pike, 1989). The idea is that a relationship between two stimuli, say A and B, is stored as a pattern of synaptic weights joining the representations of the two stimuli. The synapses joining the active units of A and the active units of B are strengthened during learning. When A is repeated as a probe, the active units in the representation of A cause the pattern in B to be reinstated via the synapses that were strengthened during learning. In the brain, associations could be instantiated by much more complex relationships than simple Hebbian association and the pathways supporting association may be much more detailed than a single set of weights that direction connect two sets of neurons. Nonetheless, this associationist framework provides a simple model to understand how a representation of spatiotemporal context could be used to express behavioral relationships between different stimuli.

Context-to-item bindings provide one way that behavioral associations can be supported by synaptic plasticity. Suppose that associations can be formed between a representation of spatiotemporal context and the currently-available stimulus—an item. To be more concrete, let the task be to learn that A predicts B not immediately, but at a delay of 10 s, as might be required in trace conditioning. Note that a simple association between the representations of A and B is insufficient to describe this temporal relationship. However, note that a temporal representation that codes for information about the time of occurrence of A could be used to support trace conditioning. During study, a series of stimulus-specific time cells is activated by presentation of A. When B is presented 10 s later, only the currently-active cells can be associated to the representation of B. When A is repeated, the same sequence of cells is activated. The cells firing 1 or 2 s after A have not been associated with the representation of B. As time passes, approaching 10 s, the overlap between the state of the temporal representation and the set of cells associated to B grows, resulting in increased activation of B at the appropriate time (see also Shankar & Howard, 2012; Howard & Eichenbaum, 2013).

In addition to the behavioral effects of a representation of spatiotemporal context on the activation of item representations, we can also consider the possibility that item representations can reinstate previous states of spatiotemporal context. The ability of items to recover previously encoded states of context is a common feature of several quantitative models of memory performance (Dennis & Humphreys, 2001; Howard & Kahana, 2002). This provides a concrete definition of the “jump back in time” used to describe the phenomenological experience of episodic memory (e.g., Tulving, 1983). Although not yet definitive, a growing body of evidence from human single unit recordings, ECoG data and fMRI suggest that memory retrieval is accompanied by the reinstatement of a gradually-changing neural representation (DuBrow & Davachi, 2014; Howard et al., 2012; Manning, Polyn, Litt, Baltuch, & Kahana, 2011; Zeithamova, Dominick, & Preston, 2012), a qualitative prediction of these models. The mechanism by which contextual reinstatement is accomplished is not typically specified by mathematical models that describe behavior and may be quite complex (Howard, Fotedar, Datey, & Hasselmo, 2005).

If it exists, this hypothesized contextual retrieval mechanism provides a mechanism for learning behavioral associations between stimuli that are not presented close together in time. Consider a learning task in which subjects learn pairs of stimuli that include overlapping items, for instance the pairs A–B and B–C (e.g., Bunsey & Eichenbaum, 1996; Howard, Jing, Rao, Provyn, & Datey, 2009; Primoff, 1938; Slamecka, 1976; Zeithamova et al., 2012). In a representative experiment, many pairs are learned in a random order so that there is no consistent temporal relationship between, say, A and C. Nonetheless they share a higher-order relationship by virture of having both been experienced in proximity to B. Contextual retrieval makes this possible. Suppose that A–B is learned and then, much later B–C is studied. If during study of B–C, the stimulus B can reinstate its previous context, which includes information about the time of presentation of A, then the context in which C is studied includes information about A. If A is later repeated as a cue, this will tend to activate C resulting in an association between A and C. Because the representation of spatiotemporal context discussed here can capture spatial and temporal relationships, the mechanism of contextual reinstatement can account for the results of temporal mapping experiments in which subjects not only learn that A and C go together, but incorporate specific temporal relationships (Arcediano, Escobar, & Miller, 2003; Cole, Barnet, & Miller, 1995; Savastano & Miller, 1998). For instance, if the subject initially learns that A predicts a shock at a delay of 5 s and that A predicts B at a delay of four seconds, the subject will behave as if B predicts the shock at a delay of 1 s.

To summarize, reinstatement of a spatiotemporal context representation coupled with simple associations between the contextual representation and item representations could endow the hippocampal memory system with the ability to flexibly express a rich set of relationships between stimuli. Even though the elementary operations are simple, if contextual reinstatement is under strategic control, then a broad variety of task-appropriate relationships could be retrieved.

3. Discussion

This paper has reviewed an integrated computational hypothesis for spatial and temporal coding in the hippocampus. This approach is a quantitative implementation of longstanding ideas about hippocampal coding. Depending on the inputs to the model and the form of modulation, the method can construct both temporal and spatial representations. These representations can be used to store spatial and temporal relationships among stimuli manifest in behavior. A model of flexible behavioral expression of relationships, requires that this representation representation is augmented with mechanisms for storing context-to-item associations and for reinstating prior states of spatiotemporal context. In this way, relationships between stimuli separated in space and time could be extracted to flexibly inform behavior.

3.1 Spatial and temporal input streams to the hippocampus

Information about experiences arrives in the hippocampus via two main cortical pathways (Van Essen, Felleman, DeYoe, Olavarria, & Knierim, 1990; Eichenbaum, Yonelinas, & Ranganath, 2007). The origins of these pathways are the well described “what” and “where” streams of sensory processing that represent, respectively, the identity and spatial location of perceptual objects (Mishkin & Ungerleider, 1982). These streams are best described in the visual system but also incorporate information from all of the other sensory modalities (Felleman & Van Essen, 1991; Burwell & Amaral, 1998; Suzuki & Amaral, 1994). These anatomical observations strongly suggest that a major function of the hippocampus is to integrate information about the nature of events and the spatial context in which they occur (Davachi, 2006; Eichenbaum et al., 2007). Several studies have now identified major end points of the what stream in the perirhinal cortex and lateral entorhinal cortex, and of the where stream in the parahippocampal cortex and medial entorhinal cortex, as specialized for processing event and spatial information (Eichenbaum et al., 2007). In addition, many studies have now shown that hippocampal networks integrate event and spatial contextual information (e.g., Komorowski, Manns, & Eichenbaum, 2009; McKenzie et al., 2014, reviewed in Davachi, 2006; Eichenbaum et al., 2007). Since there is now a wealth of evidence that the hippocampus also plays a key role in the temporal organization of events in memory (Eichenbaum, 2013), a fundamental question is how temporal coding is generated from information provided by the what and where streams (Eichenbaum, in press).

3.2 Extracting temporal and spatial regularities to construct semantic “maps”

Given that the hippocampus has access to a code that spatial and temporal relationships between stimuli that were experienced at distant times and places, semantic relationships can be learned by extracting regularities between stimuli experienced with consistent spatial and temporal relationships. Cognitive science has generated specific computational models of semantic memory (e.g., Griffiths, Steyvers, & Tenenbaum, 2007; Howard, Shankar, & Jagadisan, 2011; Landauer & Dumais, 1997) that describe mechanisms by which the temporal contexts of isolated experience could be integrated to form efficient networks of semantic memories. Neuroimaging work (Schapiro, Kustner, & Turk-Browne, 2012, see also Miyashita, 1988) makes it clear that MTL representations of stimuli come to reflect their temporal contexts, even when there are no explicit task demands to learn the relationships (see also Bunsey & Eichenbaum, 1996).

Human subjects can learn semantic relationships between stimuli by simply observing the contexts in which they appear. Consider being exposed to the novel word FLOOB in the sentence “The baker reached into the oven and pulled out the FLOOB.” Knowledge about other words that fit into the context lets us know that FLOOB is more similar to “bread” or “cake” than to “democracy” or “spaceship.” The idea of learning from contexts has been formalized in computational models of semantic learning. These models observe large amounts of naturally occuring text and then use statistical techniques of varying sophistication to construct representations of the words that capture their contextual relationships (e.g. Griffiths et al., 2007; Landauer & Dumais, 1997; Shankar, Jagadisan, & Howard, 2009).

Neural evidence shows that long-term memory representations of stimuli in the MTL also reflects the temporal context in which the stimuli were experienced. For instance, Miyashita (1988) found neurons in inferotemporal (IT) cortex that responded to sample stimuli in a working memory task in which the animal was required to remember a sample stimulus during a delay interval. The sample stimuli were presented in the same order on each of several recording sessions. Remarkably, a cell that responded to one sample stimulus was more likely to respond to other sample stimuli presented close in the list despite the fact that the stimuli were separated by dozens of seconds during learning. This tendency to code stimuli experienced close together in time was subsequently observed in perirhinal cortex (Naya, Yoshida, & Miyashita, 2001; Erickson & Desimone, 1999; Messinger, Squire, Zola, & Albright, 2001) and appears to depend on connections from MTL regions back to IT cortex (Higuchi & Miyashita, 1996; Miyashita, Kameyama, Hasegawa, & Fukushima, 1998; Naya et al., 2001).

Schapiro et al. (2012) showed that the neural representations of visual stimuli in the MTL, as measured by representational similarity analysis, came to reflect their temporal context. At the beginning of a session, subjects were presented with a set of fractal images. Then the subjects viewed the images during a training session while performing a cover task that required them to detecting a slight alteration in some of the images. During training, some pairs of non-target images were repeatedly presented in sequence. Critically, pairs were assigned randomly from the pool. After training, the images were presented in a random order. After training, the pattern of BOLD response across hippocampal voxels was more similar for members of the same pair than for images that were not paired. This effect held throughout the subregions of the hippocampus.

Schapiro, Rogers, Cordova, Turk-Browne, and Botvinick (2013) showed that the ability to learn stimulus representations from temporal context extends to more complex networks of relationships. They found that stimuli were presented sequentially during presentation and shared common neighbors became more similar to one another than stimuli that were simply paired during presentation but whose temporal neighbors did not overlap to the same extent. The fact that hippocampal representations reflect these relatively subtle aspects of the statistical structure of experience suggests that the hippocampus learns relational networks from temporally structured experience. Time cells could play an essential role in representing the temporal structure of experience so that it can be extracted into lasting stimulus representations that reflect temporal structure.

Conclusions

We suggest that a general function of the hippocampus is to construct “memory spaces” that capture reliable spatial and temporal relationships between stimuli. In order to construct these networks within a fixed window of synaptic plasticity, the hippocampus must be able to, at each moment, have access to information about the location and temporal distance from stimuli that were experienced in the past. Place cells and time cells are ideally suited to provide this capacity. We briefly reviewed computational model that codes for spatial and temporal representations and showed that it describes a continuity between temporal and spatial representation, consistent with that observed experimentally (Kraus et al., 2013). This representation could be used to construct a learning model that encodes temporal and spatial relationships into a persistent semantic memory. This hypothesis is at least roughly consistent with neurophysiological studies of long-term memory representations and recent fMRI studies of learning of temporal contexts.