On the integration of space, time, and memory

Abstract

The hippocampus is famous for mapping locations in spatially organized environments and several recent studies have shown that hippocampal networks also map moments in temporally organized experiences. Here I consider how space and time are integrated in the representation of memories. The brain pathways for spatial and temporal cognition involve overlapping and interacting systems that converge on the hippocampal region. There is evidence that spatial and temporal aspects of memory are processed somewhat differently in the circuitry of hippocampal subregions but become fully integrated within CA1 neuronal networks as independent, multiplexed representations of space and time. Hippocampal networks also map memories across a broad range of abstract relations among events, suggesting that the findings on spatial and temporal organization reflect a generalized mechanism for organizing memories.

“There is no difference between time and any of the three dimensions of space except that our consciousness moves along it.” – H.G. Wells in The Time Machine

Introduction

The hippocampus has long been regarded as critical to memory (Clark & Squire, 2013) as well as to supporting the brain’s representation of space (Moser et al., 2008). A potential link between these characterizations is that the hippocampus organizes memories in space, which is a prominent feature of memory that depends on the hippocampus (Eichenbaum et al., 1999). In addition, memory for specific experiences (episodic memory) is characterized by an organization of events in time (Tulving, 1972), and several recent findings have revealed temporally organized hippocampal neuronal activity patterns that support memory (Dragoi & Buzsaki, 2006; Pastalkova et al., 2008; MacDonald et al., 2011; Wikenheiser & Redish, 2015; Cai et al., 2016; reviewed in Eichenbaum, 2014). Combining these lines of evidence, one possible accounting of hippocampal function is the organization of memories in space and time (Eichenbaum, 2017). A key question in pursuing this hypothesis is how neuronal networks within the hippocampus accomplish the combination of spatial and temporal organization.

In our everyday lives we typically conceive of space and time as separate dimensions of experience, but we often combine them in our expression of episodic memories. If I asked about your morning, you likely could recap the full episode as it unfolded in time and across places where successive events occurred. This perspective reflects a common view that episodic memory involves embedding our record of events within a unified representation of “spatiotemporal context” (e.g., Copara et al., 2014). Also, in formal applications of physics and cosmology, considerations of space straightforwardly include time as a “fourth dimension”. Most famously, a key component of Einstein’s special relativity theory is that time dilates with speed across reference frames, and this key observation forms the basis for our modern conception of “spacetime” as a unification of spatial and temporal dimensions. Here I will consider how the brain processes space and time, focusing on whether the brain pathways and mechanisms for spatial and temporal processing reflect distinct spatial and temporal codings or instead whether they reflect a unified representation of “spacetime” in which memories are localized.

Brain pathways for spatial and temporal cognition

The pioneering studies that distinguished “what” and “where” streams of visual processing identified a dedicated brain pathway for spatial cognition and action separate from a different pathway that supports the perception of specific objects (Mishkin & Ungerleider, 1982; Gooddale & Milner, 1992). Decades of research have delineated details of the “where” stream as a succession of cortical areas that extends to the parahippocampal cortex (in primates; called postrhinal cortex in rodents) and the medial component of the entorhinal cortex, which are parts of the hippocampal region (Felleman & Van Essen, 1991; Suzuki & Amaral, 1994; Burwell & Amaral, 1998). Numerous functional imaging studies in humans have observed selective activation of the parahippocampal and medial entorhinal cortex when subjects recall scenes in which specific items were studied, thus identifying these areas as representing spatial elements of memories (reviewed in Eichenbaum et al., 2007; see below).

Studies on the perception of elapsed time (called “interval timing”) describe a widespread brain system that involves partially distinct pathways and mechanisms from those of the “where” stream (Buhusi & Meck, 2005; Meck et al., 2008). There are many models of the mechanisms of timing (Mauk & Buonomano, 2004), and one prominent model suggests that the capacity for interval timing involves interactions between multiple oscillatory patterns in prefrontal and parietal cortical areas, resulting in unique patterns of activation at different times that are integrated in the striatum (Mattel & Meck, 2000; Lustig et al., 2005).

In support of this model, several studies have shown that timing depends on the prefrontal and parietal cortex and striatum and that neurons in these areas signal elapsed time. Thus, damage to the medial prefrontal cortex severely impairs interval timing in rats, and neurons in the medial prefrontal cortex signal elapsed time in rats performing an interval discrimination task (Kim et al. 2013; Tiganj et al., 2015). Several studies have also provided compelling evidence that the parietal cortex is also involved in timing. In monkeys trained to report whether the duration of a test light was longer or shorter than a remembered “standard”, neurons in the lateral parietal area signal changes in judgments about elapsed time (Leon & Shadlen, 2003; Janssen & Shadlen, 2005). Functional imaging studies in humans have shown that the parietal-temporal junction is activated when subjects judge the temporal order of stimuli that are presented in rapid succession (Davis et al., 2009). Conversely, when subjects are trained to judge the temporal order of two rapidly presented visual stimuli, TMS over the parietal-temporal junction impairs judgments about their temporal order (Woo et al., 2009). Also, mental self-projection both to earlier times in life and to specific locations in space engages the parietal cortex within partially distinct cortical networks (Gauthier & van Wassenhoive, 2016).

Similarly, several studies have shown that damage or dysfunction of the striatum results in impairments in interval timing judgments in rats (reviewed in Howard et al., 2015), and conversely, striatal neurons fire at sequential moments during memory delays in rats performing a fixed interval lever pressing task (Mello et al 2015) and in a spatial working memory task (Akhlaghpour et al. 2016). Also, striatal neuronal activity predicts judgments of time in rats performing a duration categorization task (Gouvea et al., 2015), and neural ensembles in the striatum signal elapsed time in monkeys performing an interval-timing task (Adler et al., 2012).

Additional studies that characterize interactions among areas report that overlapping cortical networks process memories for spatial and temporal information in extended events. In experiments where human subjects travel through a city in virtual reality, memory for the spatial location and memory for temporal order of events both engage a prefrontal–parietal cortical network along with medial temporal areas. However, interactions between areas associated with remembering where or when specific events occur operate at different network oscillation frequencies identified by phase synchronization in EEG patterns as a measure of network connectivity. In one study subjects performed a working memory task where they were required to judge the order or location of stimulus presentations on a screen (Watrous et al. 2013). Time-frequency analysis of the EEG revealed enhanced power of left frontal theta (5–8 Hz), posterior alpha (9–12 Hz), and left posterior beta (14–28 Hz) during the delay period of correct temporal order trials compared to correct spatial trials. In another study, gamma (30–50 Hz) power at right lateral frontal sites was increased during the delay period of spatial working memory trials as compared to temporal working memory trials (Roberts et al. 2013). These observations of differences in oscillatory patterns associated with spatial and temporal memory success suggest distinct mechanisms for the maintenance of temporal and spatial information in working memory.

Similarly, fMRI studies on humans performing the virtual city task have shown that the hippocampus, prefrontal cortex, precuneus, and visual cortex all serve as “hubs” of high network connectivity. However, within the larger network, the operation of overlapping sub-networks for spatial and temporal memory retrieval are distinguished by higher connectivity within posterior and anterior brain areas, respectively (Schedlbauer et al., 2014). The combined EEG and fMRI findings thus provide parallel evidence of sub-networks for spatial and temporal processing connected via medial temporal areas as a hub for spatial-temporal integration (Figure 1; also see Hsieh & Ranaganth, 2015). Furthermore, taken together, these observations support the notion that spatial memory and temporal memory are mediated by overlapping and interacting brain systems that converge on the hippocampal region.

Figure 1.

Schematic outline of the spatial and temporal information processing systems in the brain. Spatial and temporal perception and cognition are performed in widespread overlapping cortical networks that connect with the hippocampus via the parahippocampal cortex (PHC), which is a major cortical input to the medial entorhinal cortex (MEC). MEC then sends merged spatial-temporal information to hippocampal areas CA3 and CA1. While some studies suggest differential representation of space and time within hippocampal circuitry that could be separately supported by subcortical and cortical pathways, respectively, information about events in space and time are fully integrated in CA1.

Spatial and temporal memory processing in parahippocampal cortical areas

There is substantial additional evidence that the parahippocampal cortex and medial entorhinal cortex support memory for spatial and temporal context. Functional imaging studies have reported that the parahippocampal cortex is activated when human subjects view spatial scenes (Epstein & Kanwisher, 1998; Epstein et al., 2007) as well as pictures of objects that have strong associations with a spatial context (e.g., a refrigerator) compared with objects that have no strong contextual association (a pencil; Bar & Aminoff, 2003). In a further study, Aminoff et al. (2007) trained human subjects on spatial and temporal contextual associations by having subjects repeatedly view a set of meaningless visual patterns either in the same spatial arrangement, always together but in different spatial arrangements, or individually. They then used fMRI to compare to level of activation for items trained individually to those when items were trained within a spatial and temporal context. The activation patterns within parahippocampal cortex partially distinguished a parahippocampal area that activated in response to items that were associated by spatial arrangement from another parahippocampal area that activated for items that co-occurred in time without spatial regularity.

Experiments that explicitly tested memory for spatial and temporal context or a spatial arrangement of object cues have shown that the parahippocampal cortex activates when human subjects remember either spatial or temporal context. In fMRI studies of spatial context, Davachi et al., (2003) reported that the parahippocampal area is activated when subjects recall a spatial scene with which object stimuli were associated. Libby et al. (2014) tested subjects on memory for objects in specific locations in an array and used multivoxel pattern analysis to determine the extent of object and location coding in hippocampal areas. They found that, whereas activity patterns in perirhinal cortex carried information about individual objects, activity patterns in the parahippocampal cortex carried information about the configuration of spatial locations that was to be remembered, although the patterns of activation in these areas was not predictive of memory success per se. Another fMRI study reported that the medial entorhinal cortex activates associated with familiarity of object locations, as contrasted with the observation that lateral entorhinal cortex activates associated with familiarity of object identities (Reagh & Yassa, 2014).

In a study on the temporal organization of memories, Hsieh et al. (2014) trained human subjects on sequences of objects, then used multivoxel pattern analysis to measure the similarity of activation patterns for the same objects in different temporal sequences. They found that, whereas the perirhinal cortex activated in similar multivoxel patterns for specific objects across sequences, the parahippocampal cortex activated in similar patterns for temporal positions across sequences, indicating a representation of temporal organization. Another fMRI study showed that the parahippocampal cortex also activates during the retrieval of the temporal order of a series of scenes in a movie as compared to a control condition where subjects logically inferred the order of scenes from the same movie (Lehn et al., 2009).

Additional evidence from studies on rodents indicates that space and time are integrated in the medial entorhinal cortex. Lipton et al. (2008) recorded from medial entorhinal neurons in rats alternating between paths in a T-maze where left-turn and right-turn routes through the maze overlapped just before the critical choice in the task. They found that medial entorhinal neurons had spatially specific firing patterns, and these patterns differed for left-turn and right-turn trajectories including sections of the maze the rats traversed on both trajectories. Thus, the spatial firing patterns of medial entorhinal neurons discriminated paths depending on the trajectory to a goal that defines the temporal context of that episode (see also Frank et al., 2000). Furthermore, Kraus et al. (2015) found that medial entorhinal “grid cells”, which have highly specific spatial firing patterns during open field exploration, also fire at sequential moments while running in place for specific periods on a treadmill, and the activity of these cells signaled both elapsed time and distance run on the treadmill. These results indicate that the same neurons that code locations in space are the cells that code moments in a temporally structured experience, although individual neurons differed in the extent to which they coded time and distance (see below). Taken together, the findings on humans and animals provide strong evidence indicating that the role of parahippocampal and medial entorhinal cortical areas in memory extends equally to the representation of spatial and temporal organization.

Are time and space separated or integrated within the hippocampus?

Given that space and time are represented together in major cortical afferents to the hippocampus, it might be expected that these dimensions be also merged throughout the circuitry of the hippocampus itself. Evidence from functional imaging studies in humans and lesion and recording studies in animals provide different perspectives on whether space and time are processed distinctly or fully integrated within the hippocampal subdivisions. Hippocampal circuitry involves sequential and parallel stages of information processing, such that the entorhinal cortex projects to all hippocampal subdivisions and intrinsic pathways involve successive projections from CA3 to CA2 and CA1, and from CA2 to CA1 (Figure 1; Amaral & Lavenex, 2006; Dudek et al., 2016). Outputs of this circuitry are from CA3 to subcortical areas, and from CA1 back to the entorhinal cortex and other cortical areas both directly and indirectly via the subiculum. Thus, because CA3 and CA1 have independent inputs and outputs, it is possible that each of these areas make separate contributions to spatial or temporal processing in the absence of the other, as has been reported in some of the studies described below.

Functional imaging studies in humans

There is a long history of studies showing hippocampal activation in humans associated with memories for events in spatial context (reviewed in Eichenbaum et al., 2007). One particularly compelling demonstration involved testing human subjects on memory for objects in specific locations in an array. Multivoxel pattern analysis revealed that hippocampal activity patterns predicted accurate memory for particular object–location relationships (Libby et al., 2014). In recent years, several studies have also reported activation of the hippocampus associated with memory for the temporal organization of memories (Hsieh et al., 2014; Lehn et al., 2009; see also Ezzyat & Davachi, 2014; DuBrow & Davachi, 2016; review in Eichenbaum, 2014). For example, Lehn et al. (2009) let subjects watch a novel movie and later, during fMRI, asked them to rearrange and “replay” scenes from the movie in correct order. To identify areas specifically involved in retrieval of temporal order, they used a control condition where subjects logically inferred the order of scenes from the same movie. Strong hippocampal activation was specifically related to retrieval of temporal order and positively correlated with accuracy of sequence recall. Also, Hsieh & Ranganath (2014) used multivoxel pattern similarity analysis of fMRI data during retrieval of learned object sequences to systematically investigate hippocampal coding of object and temporal context information. Hippocampal activity patterns carried information about the temporal positions of objects in learned sequences, but not about objects or temporal positions in random sequences. In addition, hippocampal activation patterns differentiated between overlapping object sequences and between temporally adjacent objects that belonged to distinct sequence contexts.

To explore potential differences among hippocampal subdivisions and whether space and time are represented by the same or different neural ensemble activity patterns, Kyle et al. (2015) employed high-resolution fMRI on human subjects during memory retrieval after playing a virtual reality game where they visited stores in a particular order within a specific spatial layout. During retrieval participants made independent judgments either about near or far intervals within the spatial layout or about temporal sequence. Across both near and far intervals, retrieving spatial layout and temporal order information resulted in comparable levels of activation in the hippocampus that was not preferentially localized to a specific subfield. However, although no specific subfields were differentially recruited for spatial or temporal order retrieval, multivariate pattern similarity analysis indicated that correct near judgments vs. correct far judgments differed in their patterns of activity for spatial versus temporal order judgments. These findings indicate that while space and time are both processed throughout mthe hippocampus, the two domains are represented by distinct neural network patterns.

Additional evidence that the hippocampus simultaneously processes spatial and temporal organization comes from a study by Deuker et al. (2016) where they trained subjects to traverse spatiotemporal trajectories in a large-scale virtual city. During testing of memories for the distances and timing between key events, subject-specific neural similarity in the hippocampus scaled with the remembered proximity of events both in space and time. Notably, unlike in the Kyle et al. (2015) report, in this study subjects’ judgments about space and time were correlated and the integrated spatial and temporal patterns did predict space and time judgments. However, there were also individual influences of spatial and temporal coding when the other variable was removed in the analysis, confirming a degree of independence between the neural ensemble patterns that represent space and time.

Lesion studies in rats

Studies on rats also report that the hippocampus plays a role in both spatial and temporal memory processing, but some of these studies identified differences between hippocampal subdivisions. Kesner and colleagues distinguished spatial and temporal memory processing between hippocampal areas CA3 and CA1, respectively. In one experiment, rats learned associations between a particular object or odor and its location in an open field (Gilbert and Kesner, 2003). Selective lesions of CA3 impaired acquisition of object-place and odor-place associations, whereas CA1 lesions did not. Indeed, in the case of odor-place associations, CA3 lesioned animals showed no learning, whereas animals with CA1 lesions performed normally. In another experiment rats learned to associate a pair of objects separated by a temporal gap (Kesner et al., 2005). Thus if object A was presented before a delay, then odor 1, but not odor 2, was associated with reward. Conversely, if object B was presented first then odor 2, but not odor 1, had the reward. Rats with selective CA1 lesions showed no sign of acquiring the associations, whereas rats with CA3 lesions acquired the task just as rapidly as normal control animals. Thus, the pattern of results in the two experiments was complementary: CA3 was critical to object-spatial associations whereas CA1 was critical to associations between objects across time.

Farovik et al. (2010) also reported a selective role for CA1 in temporal processing using an odor paired-associate task. In the study phase, rats were presented with a list of ten odor pairs wherein the odors within each pair were separated by a 3 or 10 sec gap. In a subsequent memory test, animals were presented with the same ten odor pairs and were required to distinguish pairs where the elements of a pair were presented in the same order as during study from pairs where the odors were presented in the reverse order. When the interval between odors within a pair during study was brief (3 sec), bilateral dorsal CA3 lesions severely disrupted memory for their order, whereas dorsal CA1 lesions did not affect performance. However, when the inter-item interval was extended to 10 sec, CA1 lesions, as well as CA3 lesions, severely disrupted performance. These findings confirm the Kesner et al. (2005) observation that CA1 is essential to bridging long temporal delays. However, the findings also suggest that, contrary to Kesner et al. (2005), in a the task demands memory for multiple temporal associations, CA3 may be required to associate sequential events across even brief periods. Thus, CA3 and CA1 may both perform temporal processing but play different roles that are distinguished by the duration of time that must be bridged between key events.

Recording studies in rats

Some of the findings on the firing patterns of hippocampal neurons are generally consistent with the observations on selective CA3 and CA1 lesions. Mankin et al. (2012, 2015) monitored activity patterns of CA3, CA1, and CA2 neurons as rats foraged for food in multiple open field environments and these recordings were repeated using the same environments across hours and days. Robust CA3 spatial firing patterns distinguished different environments and the pattern for each environment was stable across long periods. In contrast, CA2 spatial firing patterns only poorly distinguished environments but became progressively dissimilar across time. CA1 firing patterns robustly distinguished environments and became progressively different across time. These findings suggest a separation of selective spatial coding in CA3, selective temporal coding in CA2, and the combination of spatial and temporal coding in CA1. Consistent with the findings from foraging tasks, Ito et al. (2015) observed that CA3 cells maintain the same spatial firing patterns as rats traverse distinct paths through a maze whereas CA1 neurons distinguish the two paths. These findings can be interpreted as showing that CA3 neurons code only for spatial location whereas CA1 neurons code a sequence of locations traversed within the context of the ongoing left-turn or right-turn episode. In general, these findings suggest that CA3 may process spatial information only, CA2 time, and space and time are merged in CA1.

On the other hand, in contrast to the findings that focused on active foraging behavior, a recent study indicated that CA2 neurons do have robust spatial firing patterns, albeit only when animals are immobile in between foraging episodes (Kay et al., 2016). Also in rats running in place on a treadmill (see below) and traversing alleyways in a maze, elapsed time and spatial location are signaled equivalently among the same populations of both CA1 and CA3 neurons (Salz et al., 2016). And, in contrast to the findings of Ito et al. (2015), Bahar et al. (2012) reported equivalent representation of distinct paths through overlapping sections of a maze in CA1 and CA3. Therefore, the findings from lesion and recording studies both suggest that spatial and temporal coding may be segregated or combined across hippocampal subdivisions depending on how spatial and temporal processing is engaged during different behaviors.

Space and time support a gradual evolution of contextual representation in area CA1

While aspects of space and time may be processed separately through the hippocampal region, the findings are largely consistent in showing that spatial and temporal coding are merged within the same neural ensembles at the final stage of hippocampal information processing in CA1. The nature of spatial and temporal representation in CA1 has been studied in detail for two different scales of time, (1) over periods of hours to days - that is, in “macrotime” – over which neuronal representations for entire episodes are compared, and (2) over periods of seconds - that is, in “microtime” - for which the organization of spatial and temporal representation within a repeated individual experience is mapped.

Organizing distinct memories and places in macrotime

Manns et al., (2007) described a gradually changing temporal context signal embodied as a population firing pattern that evolves over an hour or more during which rats were tested on successive trials that assessed memory for unique sequences of events. Over the course of the entire testing session, the ensemble-firing pattern continuously changed, such that individual cells waxed and waned in activity within the ensemble, suggesting that a representation of temporal context gradually evolves over long periods. In addition, within the course of each memory trial where stimuli were presented 10 seconds apart in either of two locations. Events that occurred in different locations were distinctly coded and these spatial representations were stable over the course of a trial. In addition, the ensemble-firing pattern gradually changed over the course of each trial and the extent of change in the ensemble-firing pattern predicted successful memory on that trial. These findings indicate that hippocampal neuronal ensembles gradually change over macrotime, and the evolving change even within the period of a minute-long experience is sufficient to contribute to memory for the order of events on each trial.

The study by Mankin et al. (2012) introduced above extended the period of gradual evolution of hippocampal spatial firing patterns over several hours and days. They recorded from CA1 neurons as rats explored different, highly familiar environments repeatedly at hours apart over days and found a gradual change in the population spatial representation over at least 30 hours, suggesting a mechanism for distinguishing identical foraging episodes that occurred at different times. Using optical imaging of hundreds of CA1 neurons simultaneously, Ziv et al. (2013) quantified the changes in spatial firing patterns as involving a substantial fraction of the cells that altered their spatial activity patterns each day, while maintaining a subset of cells with a stable spatial representation. A further study using this approach to compare spatial mapping patterns in two environments across days showed that the spatial mappings shared common elements within each day regardless of the spatial context in which they occurred, whereas temporally remote experiences have distinct “timestamps”, neurons that fire differently in the same contexts across days, and timestamp cells involve both place cells and cells that do not code for location (Figure 2a; Rubin et al., 2015). Thus, distinctions in these mappings across time and space are intertwined within a CA1 population that involves mixed selectivity for space and time.

Figure 2.

Idealized CA1 mappings of spatial and temporal information by mixed selectivity in different behavioral paradigms. a. Place cell maps during foraging behavior in two environments (distinguished by solid and dashed outlines). Some neurons (dark red) code for space only and some (dark blue) for time only. Other neurons signal both space and time by variations in firing rates in the two environments or on different days, respectively. b. Place cell maps during T-maze alternation. Some neurons (dark blue) fire at specific locations only on left turn trials and others (dark red) only on right-turn trials. Other cells have higher (blue/red) or lower (light blue/pink) firing rates that differentiate left and right trials, respectively. Yet other cells (black) fire equivalently on both trajectories. c. Time and distance maps during treadmill running. Some cells (dark blue) fire strongly associated with time and not distance, and others (dark red) fire strongly associated with distance and not time. Other cells (dark & light blue/pink and red) differentially signal time and distance by firing rates, and yet other cells (black) equivalently signal time and distance. Dark red = spatial; dark blue = temporal; lighter blue = rate coding of time > space; lighter red = rate coding of space > time. Black = equal coding of space and time.

The studies described just above involved foraging behavior and did not include tests of memory. Using a contextual fear conditioning paradigm, Cai et al., (2016) showed that the gradual evolution of CA1 ensemble firing patterns links memories that occur within relatively short time intervals and, conversely, distinguishes memories that occur far apart in time. They observed that CA1 firing patterns recorded as animals explored different environments had correlated activity patterns across two different environments that were experienced a few hours apart, but ensemble patterns were poorly correlated between environments that were experienced several days apart. Furthermore, when the animal was shocked in one environment and then tested for generalization of the acquired fear memory to the other environment, generalization occurred between environments that had similar ensemble firing patterns but not between environments that had less similar ensemble patterns. These findings suggest that the linking and separation of memories for spatial environments may be supported by a gradually changing network representation of time.

Mapping of spatially and temporally structured episodes in microtime

A different pattern of spatial and temporal coding occurs in the hippocampus over brief periods when experiences are repeated many times. The well-known phenomenon of place cells observed when animals forage for long periods in an environment can be characterized as a mapping of the spatially structured context by a parsing the environment into place fields (the “cognitive map”; O’Keefe & Nadel, 1978). However, contrary to a stable map of space, in a variety of linear track and maze tasks where animals repeat the same route many times, hippocampal representations of distinct paths are embodied as sequential place cell activations that are unique to a specific path through space and time. For example, in rats performing T-maze and plus-maze tasks, partially distinct place cell sequences are activated during the course of left-turn and right-turn paths through the maze, including different spatial firing patterns where the animal traverses segments of the maze that overlap between the two routes (Wood et al., 2000; Frank et al., 2000; Ferbinteanu et al., 2003), and Ainge et al. (2007) described place cell sequences that similarly distinguished four multi-branched paths. Importantly, the representations of these paths involves a combination of cells that fire at a specific place only on the left-turn or the right-turn path, cells that fire at different rates at the same specific locations on the two paths, and cells that have the same spatial firing pattern on both paths, thus characterized as mixed selectivity in the neural population for places and for the temporal context of the overall path to a specific goal (Figure 2b) or for different motivational contexts (Kennedy & Shapiro, 2009). These observations of spatial firing patterns that are unique to the sequence of events that define a specific episode indicate a merging of spatial and temporal coding.

These studies on path-specific ensemble activity patterns do not make clear whether the temporal flow of the spatial firing sequences is a secondary feature of the orderliness of places traversed, or whether there is an underlying representation of time per se that binds the sequential place representations that compose a specific episode. This question has been addressed in studies that examine temporal firing patterns of hippocampal neurons during periods when the rat is not running through the T-maze. Pastalkova et al. (2008) trained rats on a T-maze alternation task where, in between each trial, the animal ran in a running wheel for 10 sec. They reported that CA1 neurons fired at sequential moments during the period of wheel running – we call these neurons “time cells” – and other neurons fired at sequential locations while traversing the maze, and the combined population activity spanned the full period of each trial. Furthermore, CA1 population activity differentiated left-turn and right-turn trials both during wheel running and on the section of the maze that was common to both trial types. Thus, the CA1 population included both “time cells” that fired at sequential moments of running in place on the wheel and place cells that fired as the animals traversed sequential locations on the maze, and both types of cells differentiated the left-turn and right-turn episodes.

A limitation in interpreting the time cell firing patterns was that elapsed time and distance run were confounded during wheel-running – the longer the run, the farther the distance traveled. To address this confound, Kraus et al (2013) used an experimenter-controlled treadmill to force running at different speeds in between alternations, allowing them to disentangle whether neural activity is more related to time or distance. By comparing firing patterns across runs with different running speeds, they found that CA1 time cell activity patterns had mixed selectivity for elapsed time and distance traveled and the population of cells varied in the relative coding of time versus distance but were hardly influenced by small changes in position on the treadmill. The finding of temporal coding independent of place coding and separate from distance coding suggests that the representation of spatial episodes is not a simple consequence of the order of locations traversed, but rather a population representation of spatial events ordered within a temporal framework (Figure 2c).

In the Kraus et al. (2013) experiment, both time cells and place cells were observed within the same population of simultaneously recorded neurons. Indeed, some of the same cells showed time-specific activity when the rats were running on the treadmill and place-specific firing patterns when rats traversed other segments of the maze. Furthermore the spatial and temporal firing patterns of these cells are similar in CA1 and CA3 with the exception that resolution temporal coding diminishes as time passes, that is, cells that fire later fire longer (sometimes called “scalar coding”; Figure 2c; Kraus et al., 2013; Salz et al., 2016). These findings emphasize the mixture of space and time coding in the same neural population even as the two dimensions involve different scales.

Several other studies have identified temporal coding and similarities in temporal and spatial coding by hippocampal neurons in different species and different behavioral paradigms. Additional work by multiple investigators has reported that hippocampal time cells develop with repeated experiences in a specific temporal structure in a variety of memory tasks in rats, monkeys, and humans (Gelbard-Sagiv et al., 2008; MacDonald et al., 2011, 2013; Modi et al., 2014; Naya & Suzuki, 2011; Paz et al., 2010; reviewed in Eichenbaum, 2014). The characteristics of time cells closely parallel the properties of place cells observed in other studies. For example, when a salient spatial cue in a familiar environment is altered, place cells either retain the same firing pattern associated with remaining spatial cues or ‘re-map’, that is, alter their spatial firing pattern or cease or begin firing (e.g., Shapiro et al., 1997). Similarly, when the duration of the interval between cues in an object-odor association task is altered, some time cells maintain firing at a specific time relative to the beginning or end of the interval and many other cells ‘re-time’ — that is, fire at a different moment or cease firing (MacDonald et al. 2011). The development of a new mapping of place fields occurs gradually following a brief silencing of activity (e.g., Wilson & McNaughton, 1993, Frank et al., 2004), and similarly, when the duration of a familiar temporal period is changed, hippocampal neurons re-time after a brief period of silence (MacDonald et al., 2011). Thus the partial reorganization of time cell firing patterns when a critical temporal cue is altered closely parallels the partial reorganization of place cells when spatial cues are altered. These findings further extend the prevalence and similarly of properties of spatial and temporal coding within the same neural population.

Also, both place cells and time cells encode additional relevant dimensions of experience. Several studies have reported that place cells encode a range of spatial features (e.g., distance, direction; Gothard et al., 1996; Ravassard et al., 2013) and specific non-spatial stimuli (e.g., an odor in a particular place; Komorowski et al., 2009) and behavioral actions (e.g., a learned “jump” escape response; Lenck-Santini et al., 2008). Similarly, as described above, multiple studies using different behavioral paradigms have reported that time cells encode place, distance, and ongoing behavior (MacDonald et al., 2011; 2013; Kraus et al., 2013). Thus, it is most accurate to think of the CA1 network as multidimensional, encoding time and place along with other relevant spatial and non-spatial features of ongoing experience (Mckenzie et al., 2014).

Organizing memories in macrotime and microtime

Experiments on macrotime focus on variation of a representation when an experience re-occurs after a long time whereas experiments on microtime focus on the stability of representations over many rapidly repeated experiences, so one should be cautious in comparing these phenomena directly. Nevertheless, it is tempting to envision the two phenomena as reflecting the same fundamental neural coding mechanism. In macrotime, population firing patterns that gradually evolve over minutes to days reflect changes in spatial coding and timestamps that are common to temporally neighboring experiences. In microtime, when experiences reoccur within a stable spatial or temporal structure, place cells and time cells emerge within the hippocampus, parsing the dimensions of space and time into stable place fields and time fields, respectively. The mechanisms of macrotime and microtime might have a common basis if repetition of the first events in a familiar temporally structured experience reinitiates the original network state, which then recapitulates its evolution over time – this might sharpen and enhance each cell’s activity pattern and stabilize the sequence of activations to constitute a reliable time cell and place cell sequence for a specific memory.

Space and time in the hippocampus: underlying mechanisms and implications

There are multiple views on the possible mechanisms by which space and time are integrated within the hippocampus, and each view suggests important implications about the fundamental role of the hippocampus in the representation of memories. Notably, each of these models is based on a basic temporal organization, onto which spatial and other events are mapped. However, the models differ in whether temporal sequences are generated internally from sequences of cell activations or externally by a sequence of experienced events, and on the nature of underlying mechanisms that organize the mapping (Eichenbaum, 2014).

Internally generated sequences

It has been suggested that the underlying mechanism of temporal representations in the brain is based on internally-generated network activity sequences (Buzsaki, 2013). As such, some have argued that neither time nor space is represented explicitly, and spatial and temporal coding are apparent only as a reflection of orderliness of neural assembly firing patterns (Pezzulo et al., 2014: Friston & Buzsaki, 2016). That sequences exist without spatial coding per se is observed when hippocampal place cells “replay” a former spatial firing sequence during subsequent off-line states in sleep or during immobility outside the same locations (Carr et al., 2011). In addition, temporal patterning has been observed in the “preplay” of firing sequences prior to the incorporation of spatial tags for each element of a spatial sequence (Dragoi & Tonegawa, 2013) and temporal sequencing can persist even when spatial firing is disrupted in an animal model of neurodegenerative disease (Cheng & Ji, 2013).

These observations indicate that intrinsic network patterns contribute to representating the sequential order of events in memories by attaching successive events to the network sequence. However, when rats forage in random paths through open environments, place cells fire at the same location regardless of the current trajectory sequence, so path-independent firing patterns by internally generated sequences would have to add a mechanism for associating overlapping locations that were encoded in different sequences. In addition, the scalar property of time cell sequences features a decrease in number and expansion of firing duration of successively active time cells (Kraus et al., 2013; Salz et al., 2016). Internally generated sequences would have to incorporate a mechanism for nonlinear scaling of time fields differently than for the linear progression of place fields across space.

Temporal Context Model

The findings discussed above suggest that the representation of memories in space and time involves a gradual evolution of network activity that links neighboring events and distinguishes events that are separated in space or time. A popular conception for this mechanism is the Temporal Context Model, which posits that a leaky integrator of inputs carries information about of the flow of events in experience, such that the hippocampal representation at any moment is the sum of influences by current cues and influences by preceding cues in temporally declining strength (Howard et al., 2014; Shankar & Howard, 2012). This model was originally conceived to explain contiguity effects in human free recall, but has been expanded as an account of neural mechanisms of memory supported by the hippocampus in animals as well as humans (Howard & Kahana, 1999; Howard et al., 2005). Howard et al. (2014) modeled both spatial and temporal organization in the hippocampal region using a unified mathematical framework that computes functions of both spatial location and time as special cases of a more general computation where experience unfolding in time is encoded via a set of leaky integrators.

Manns et al. (2007) confirmed the model, showing that hippocampal ensemble representations are more similar between temporally adjacent events in a novel sequence and grow more dissimilar with the separation of events within and between sequences. Moreover, the change in this temporal context signal within an episode was associated with successful memory of the order of events, thus revealing that hippocampal neuronal populations form a gradually changing representation of temporal context that supports for memory for the order of events. The same model, extended to longer periods of time is consistent with the findings on the evolution and “timestamping” of hippocampal spatial representations over hours and days and for the findings on behavioral generalization between near-occurring events and discrimination of events that are separated in time (Cai et al., 2016). The temporal context model also accounts for distinct representations of sequences of places traversed that compose a specific spatial episode (Wood et al., 2000; Frank et al., 2000).

Unlike the internally generated sequences model, the temporal context model integrates information from multiple cortical networks that could have different coding scales, and so evades the problem of different scales of representation for time and space. Also, because the inputs are external, diverse trajectories that share locations will integrate neighboring location information across overlapping routes, generating a trajectory-independent spatial map during random foraging in open fields. Conversely, in tasks that involve stereotyped routes, the model predicts distinct representations of different routes, as observed when a task switches from random foraging to movement patterns that are stereotyped (Markus et al., 1995) as well as in T-maze alternation (Wood et al., 2000; Frank et al., 2000).

A likely source of temporal context information for the hippocampus is the medial entorhinal cortex. The grid cells of medial entorhinal cortex that constitute a mapping of spatial context also signal elapsed time in rats during treadmill running (Kraus et al., 2015). Conversely, optogenetic inactivation of the medial entorhinal cortex disrupts time cell firing sequences in the hippocampus (Robinson et al., 2017).

Multiplexed maps of space and time

While the models discussed above are attractive as mechanisms that could underlie the integration of spatial and temporal coding under a unitary mechanism, it should be emphasized that space and time are not tightly correlated in the activity patterns of single neurons, as might be expected if each neuron coded for a unique coordinate in a unified spacetime framework. Instead, several lines of evidence introduced above suggest that space and time are coded independently, albeit via a multiplexing of spatial and temporal representation within the same neural population: (1) In animals performing a task in which they both run through the arms of a maze where place cells can be identified and run in place on a treadmill where time cells can be identified, the likelihood that a cell codes time and place is precisely that expected if the two dimensions are coded independently (Salz et al., 2016). (2) Over long periods (days), the evolution of hippocampal cell assemblies involves distinct elements that “timestamp” memories independent of their representation of space and elements that code space independent of time (Figure 2a; Rubin et al., 2015). (3) The distinctive representations of different trajectories through a T-maze involve a combination of cells that fire uniquely during the left-turn or right-turn path (thereby strongly distinguishing temporal contexts), other cells that have the same firing patterns during the two paths (coding only space), and yet other cells that fire at different rates in the same locations of the two paths (mixed selectivity for temporal context and space; Figure 2b; Wood et al., 2000, Robitsek et al., 2013). (4) During treadmill running, the coding of time and distance by single neurons includes a full distribution of strengths of spatial and temporal coding, indicating independence of the representations (Figure 2c; Kraus et al., 2013). (5) In analyses of multivoxel similarity of fMRI signals, the representations of space and time in memory differ (Kyle et al., 2015; Deuker et al. (2016). These findings converge in indicating that spatial and temporal representations do not reflect the same fundamental feature coding, nor are they inextricably linked, such that each dimension is assigned independently to events as they unfold together during the same experience in time and space. In other words, the hippocampus does not unify its representation of space and time – as embodied in the notion of “spacetime” – but rather represents these dimensions independently. However, findings on both single neurons and fMRI show that spatial and temporal coding are mixed within the same neuronal population, suggesting the integration of space and time occurs as a multiplexing of information in the spatial and temporal domains.

Emerging evidence of maps beyond space and time

A mechanism that could incorporate the broad range of mixed selectivity of CA1 neurons is that space and time are examples of a fundamental hippocampal code for representing continuous dimensions of experience, using elements that parse the dimensions into units that connect neighboring representations of any kind of events. Consistent with integration in the Temporal Context Model, fundamental plasticity mechanisms of highly interconnected hippocampal networks to integrate events that neighbor in space and time to generate a continuous mapping of adjacencies that reflects proximity in space (place cells), time (time cells), or both conjointly (a specific path). From this perspective, it becomes possible to imagine that the same hippocampal network can develop maps of arbitrary spaces by adding neurons that also code for specific events, and these events become part of an integration over time and space (Howard et al., 2014). Particularly compelling cases involve the learning of temporal statistics of transitions between any set of neighboring events (Schapiro et al., 2014). For example, in one experiment, subjects learned regularities in transitions between clusters of arbitrary visual patterns that were highly interconnected as compared to a single connection between clusters, with the regularity of transitions between directly interconnected items equalized (Schapiro et al., 2016). Pattern similarity analysis on hippocampal fMRI multivoxel activations reflected the community structure of interconnected clusters and less connectivity at cluster boundaries, paralleling the findings on representations of environments across long periods (Rubin et al., 2015; Cai et al., 2016).

The statistical learning paradigm is particularly illustrative in suggesting a large range of organizational structures that could be mapped in the hippocampus by regularities of diverse dimensions of experience. Indeed, there are now several additional examples of hippocampal mapping of a broad range of continuous dimensions of experience. Tavares et al. (2015) reported that hippocampal networks represent a social space in humans playing a virtual reality game where they acquire social relations in dimensions of power and affiliation. Similarly, Constantinescu et al. (2016) discovered a grid-cell-like mapping in entorhinal cortex for a learned arbitrary mapping of two dimensions of physical features of stick-figure bird species.

Several studies in animals have also revealed engagement of the hippocampus in mapping of arbitrary continuous dimensions by the hippocampus. Aronov et al., (2017) recorded from hippocampal and medial entorhinal neurons in rats performing a task where they press a lever to experience a pure tone that ramps upward in pitch, and then to receive reward they release the lever within a specific frequency zone. They observed that neurons in both areas acquired auditory receptive fields that successively mapped the relevant continuous frequency space. Parallel to this physiological evidence, lesion studies on rodents, monkeys, and humans have shown that the hippocampus is essential for mapping of a variety of learned associational and hierarchical organizations of arbitrarily related stimuli (e.g., Bunsey & Eichenbaum, 1996; Dusek & Eichenbaum, 1997, 1998; Buckmaster et al., 2004; Preston et al., 2004; Shohamy & Wagner, 2008; Zalesak &Heckers, 2009; Collin et al., 2015; see Schiller et al., 2015).

The large range of organizations of information supported by hippocampal networks share in common that elements are linked when they occur together, and these linkages can be extended across elements never experienced together but experienced with common associates. The resulting combination of direct and indirect associations form the structure of a “memory space” organized by continuous dimensions of experience to support the representation of relations among elements of memories and among memories (i.e., relational representation) and flexibility of expression of declarative memories (Howard et al., 2005; Deuker et al., 2016; Eichenbaum, 2017).

Conclusions

The evidence presented here suggests that space and time are initially processed by overlapping brain networks and coded in different scales; then spatial and temporal signals are integrated within the hippocampal region to create a framework for the spatial-temporal organization of memory. Although movement through space and time are intrinsically coupled, spatial and temporal coding can be observed differentially in hippocampal regions under some behavioral demands or combined by individual neurons that code both dimensions into the overall population representation. Notably, different from a representation of unified coordinates of “spacetime”, the hippocampal network multiplexes spatial and temporal representations via mixed selectivity for independently coded space and time. This observation is consistent with emerging evidence that hippocampal networks also map a broad range of other continuous dimensions of experience that organize knowledge via mixed selectivity of neuronal elements in the same population. The Temporal Context Model, as applied to a hippocampal network with a high range of mixed selectivity for diverse inputs, suggests a simple mechanism for associating neighboring events that constitute the integration of space, time, and other dimensions that characterize abstract associational spaces.

In Brief.

The hippocampus maps the spatiotemporal context of experience, suggesting a unified representation of “spacetime”. However, a review of evidence on activity patterns in the hippocampus suggests, instead, hippocampal networks maintain independent, multiplexed representations of space and time, and more.