Space and time: The hippocampus as a sequence generator

Abstract

Neural computations are often compared to instrument-measured distance or duration, and such relationships are interpreted by a human observer. However, neural circuits do not depend on human-made instruments but perform computations relative to an internally defined rate-of-change. While neuronal correlations with external measures, such as distance or duration, can be observed in spike rates or other measures of neuronal activity, what matters for the brain is how such activity patterns are utilized by downstream neural observers. We suggest that hippocampal operations can be described by sequential activity of neuronal assemblies and their internally defined rate of change without resorting to the concept of space or time.

Space, time, and memory in the hippocampal system

Perhaps the most important computation attributed to the hippocampal system is episodic memory [1]. Episodic memory refers to an important concept that describes first-person experienced events within the context of space and time (“what happened to me where and when?” [2]. How such personal memories are created and recreated is a subject for intense research. If the brain had to store a separate representation of every individual experience of our lifetime, that is, every combination of ‘what’, ‘where’ and ‘when’, the list would be extraordinarily long and would require an extraordinarily large storage capacity. Recalling an episode from such a long list would be complicated and time consuming. Borges’ fictitious character Funes the Memorius had an impeccable memory and could recall every single moment of his previous day’s activity, but it took him another full day to do that [3]. An alternative solution is to store the ‘what’, ‘where’, and ‘when’ components separately and re-create the original episode by re-embedding the ‘what’ into the ordinal structure of ‘where’ and the ‘when’. Such separation of memory components is appealing. By assuming independent spatial and temporal frameworks and adding them to the coding of ‘what’, it might appear that neuroscience has identified a road map for uncovering the neurophysiological mechanisms of episodic memory and understanding the global function of the hippocampal system (Box 1).

Box 1. Evolving views of space and time in the hippocampus.

Two key episodes provided critical insights into the operation of the hippocampus and associated structures: the discovery of amnesia in the patient H.M. after bilateral removal of the hippocampus [1] and the discovery of place cells in rats [14]. Yet, how these observations relate to each other and the many other functions attributed to the hippocampus have remained a puzzle [5]. There is a general agreement that the hippocampus is essential for the utilizing spatial relations of an environment and for the ability to remember specific personal events [2]. Episodic memory is defined as first-person experienced events in the context of space and time (“what happened to me where and when?”; [2]). Based on animal research, O’Keefe & Nadel (1978) conceptualized a hippocampal cognitive map and suggested that “the hippocampus is the core of a neural memory system providing an objective spatial framework within which the items and events of an organism’s experience are located and interrelated.” Thus, the cognitive map may provide the needed spatial scaffolding for episodes. Recently, Howard Eichenbaum argued that the episodic memory framework also needs a temporal context and dubbed hippocampal and entorhinal neurons ‘time cells’ [21,74,126,127]; “time cells may play a role in episodic memory by tagging when events occur in time, just as place cells map where events occur in space.” [5] According to this framework, a fundamental goal of neuroscience is to uncover the neuronal ‘representations’ of space and time into which things and events can be embedded.

In this Opinion piece, we challenge this general framework on both conceptual and experimental grounds. The current conceptual framework in neuroscience is based on the space and time ideas of classical physics. However, in contemporary physics “there is no longer space which ‘contains’ the world, and there is no time ‘in which’ events occur” [4]. We suggest neuroscience requires a similar update in paradigm. When the concepts of space and time are scrutinized, they turn out to be mere human-invented terms conveniently classifying events of the world, rather than independent entities. At the experimental level, we consider that conceptualizing the hippocampal system as a device that computes space and time fails to account for many experimental observations [5] because the hippocampus may be ‘blind’ regarding the modality of its inputs. Whatever information is presented to it, from whichever parts of the neocortex, it activates the same computational algorithms. Thus, the specific terms that we tend to assign to generic hippocampal computation may reflect largely the experimental conditions and the engaged neocortical inputs rather than an internal computation of space or time.

Space and time as organizing coordinates versus spacetime in physics

The concepts of space and time may be regarded as axioms of the universe, independent from each other and everything else. They are also the fundamental organizers of our ideas [6]. Their appeal goes back to the Newtonian framework, in which events take place in a large ‘theater’ or ‘container’ and unfold on a timeline. However, the concepts of space and time are dimensionless and unmeasurable, so they cannot be studied directly. Modern science transformed these abstract concepts by replacing them with their measurable variants: distance and duration. The ever-increasing prevalence and precision of measuring instruments changed the human concept of space and time. In today’s industrialized world it would be hard to imagine and organize human life without clocks even though a few closed human societies have survived until this day without developing linguistic structures that relate to time. Yet, these people understand sequences of events [7].

The alleged independence of space and time has been debated by linguists and physicists alike. In our everyday conversations, these dimensions are often used interchangeably, “The restaurant is a five-minute walk from here”. A unit of distance is defined by time; a light-second is the distance traveled by light in one second. The longitudes of the earth (i.e., distances) are known as time zones. Today, we find our way with the help of GPS, which has no meter metric. It computes position by determining the time interval (phase difference) that signals take to reach the receiver from multiple satellites.

Even in classical mechanics, this independence of space and time has begun to be questioned. If time is a medium through which things travel, the definition of time requires that something must move across space. Thus, time is chronicled by matter. In the 20th century, the classic ‘container’ perspectives of space and time were replaced by a spacetime model of general relativity, where the time axis is a fourth dimension. In general relativity theory, past and future are completely symmetric. The concept of now is irrelevant.

These discussions within physics and between physics and philosophy [8] have had little impact on neuroscience. Research on space and time in the brain continued in the tradition of classical physics where space and time maintain their independence from each other and everything else.

Conceptual issues

Perpetual change of activity is a fundamental rule in the brain. In our research practice, we compare the evolution of neuronal firing patterns, cell assembly sequences or other measures, against the units of instruments and often find reliable correlations. Thus, the experimenter interprets both the meaning of the neuronal responses, as measured against instruments, and the meaning of the units measured by those instruments without independent grounding. This comparison produces the illusion that the brain computes or ‘represents’ time, but what we call time is simply a set of rules that govern change. This problematic practice - interpreting both the meaning of brain responses, as measured against instruments, and the meaning of the units of those instruments - might be a fundamental confound in our current experimental approach. The observer will not find meaning in the responses outside of the variables that the instruments measure. If distance and duration are defined and understood with the help of human-made measuring instruments, one may wonder how space and time are computed and comprehended by other animals who cannot read those instruments [9].

We should recognize the difference between the spatial and temporal properties of the process of representing something and the representations of space and time. Even if neuronal activity is reliably correlated with the spatiotemporal succession of events (‘temporal sequence of representations’), such correlations do not necessarily mean that neuronal activity computes duration (‘representation of temporal sequences’; [10-12]). In other words, we must not conflate the description of events with their subjective interpretation. Neither instruments nor brains create space or time. Time does not mean anything to a clock and ticking has no intrinsic meaning without an observer. Because humans have defined the units of rods and clocks, this process inevitably defines our notion of space and time.

The classical physics-based framework has generated two largely independent neuroscience fields addressing problems of space and time with separate literatures [9,13]. One can argue that the laws of physics do not apply to ‘psychological’ time. Lived space and time may be fundamentally different from the microscopic and astronomic worlds ruled by relativity and quantum physics. Alternatively, one can adopt the attitude that resolution of the relationship between space and time is a task for neuroscience: what do space and time mean for the brain? Perhaps a good place to start is discussing the neuronal mechanisms of the hippocampus and its allies.

The anatomical organization of the hippocampus renders it blind to modalities

The hippocampus and its allied structures have been suggested to ‘encode’ a spatial map [14,15] and, more recently, to ‘represent’ time [16,17]. These theories lead to the postulates of place cells in the hippocampus [18], grid cells in the entorhinal cortex [19] and time cells in both structures [20,21], apparently paving the path towards understanding the neuronal mechanisms of episodic memory [2,16,17]. To critically evaluate this framework, it is instructive to inspect the anatomical substrates that are thought to ‘represent’ space and time.

The hippocampal-entorhinal system has a topographically organized bidirectional connectivity with the large neocortex. In a rodent, a major fraction of the neocortex processes sensory inputs and generates motor outputs. In contrast, in primates the largest fraction of neocortex is dedicated to computing more complex functions (Figure 1). Therefore, the non-sensory inputs to the hippocampus have progressively increased in parallel with the enlargement of the neocortex during mammalian evolution. The septal (dorsal) and midtemporal thirds of the rodent hippocampus receive visuospatial and other sensory inputs from the dorsal entorhinal cortex, whereas the temporal pole communicates mainly with hypothalamic, amygdalar, and prefrontal areas [22,23].

Figure 1. Homologous regions of the hippocampus across species.

The ventral quadrant of the rodent hippocampus is disproportionally enlarged in primates to keep up with the increasingly larger share of higher-order neocortex. Only the relatively small tail part of the primate hippocampus communicates with visuospatial areas. This tail is homologous to the rodent dorsal-intermediate hippocampus. The species’ differing connections to and from the segments of the septo-temporal axis are shown. Most recordings and manipulations in the rodent brain have been done in the dorsal hippocampus. Adapted from [29], Image: Debbie Maizels Springer Nature.

In the primate brain, the ventral pole forms the uncus and body and becomes disproportionally enlarged to keep up with the large fraction of the non-allied neocortex. Only the relatively small tail part of the primate hippocampus communicates with visuospatial areas and this tail is homologous to the rodent dorsal hippocampus. In contrast, the remaining major part of the primate hippocampus receives and sends neuronal signals to the non-allied (also called higher-order or associational) neocortex. In humans, differences in function have been demonstrated between the left and right hippocampi as well, commensurate with the hemispheric specialization of the neocortex [24]. Taking anatomy as a guide, sensory ‘representations’ might be only a part-time job for the hippocampus. Since the brain does not have sensors for either space or time, it is not clear how distance and durations are measured by the brain and ‘modeled’ by the hippocampal system.

In contrast to the modularly organized neocortex, the hippocampus can be viewed as a single giant module with an enormously large recurrent excitatory system [25,26], even if one acknowledges the relatively gradual anatomical, physiological and molecular changes from the septal to the temporal pole [22,27–29]. The strongly connected graph of the intrahippocampal circuit serves to combine and mix neuronal messages from wide areas of the neocortex, independent of their modalities and origin. This structural organization and the coordinated theta oscillation across the entire hippocampal-entorhinal system [30–32] suggest that that same general computation is performed on all incoming signals to the hippocampus, largely irrespective of their neocortical source. Whether a particular experimental observation implies that the hippocampus is computing space, time, memory, planning, abstract concepts, or other relationships [33-35] may depend on the experimental design and the cortical input rather than on hippocampal computation per se. This dimensionless, relational computation in the hippocampus is perhaps not so different from Howard Eichenbaum’s idea of the “representation of multidimensional inter-event relationships in memory space” [17,36,37] although it may well be that ‘multidimensionality’ is predetermined through anatomy and connectivity (Figure 1). However, even if we assume that these seemingly discrete representations are calculated by dedicated neurons and circuits, what needs to be addressed is whether these allegedly separate information channels can be differentially interpreted by downstream reader mechanisms.

Equivalence of distance and duration in the hippocampus

Classical physics already revealed that distance and duration are linked through speed: knowledge of any two variables can identify the third, so one is always redundant. Let’s examine their relationship in the hippocampus.

Place fields of hippocampal neurons can be relatively large, from 20 up to 300 centimeters compared to a typical body size of ~20 cm in rats. So, place fields from several neurons can overlap with each other for multiple theta cycles (Figure 2; [38–41]). During each theta cycle, approximately seven gamma cycles occur, and nested within each of them is a cell assembly, whose activity correlates with a given spatial position [42–45]. The spike sequence of neuronal assemblies within each theta cycle predicts the sequence of past and upcoming locations in the rat’s path. As the rat moves, new place cells join the existing assembly in the theta cycle, while spikes of the place cells whose field the animal just left disappear. Neurons with upcoming place fields fire at the late phase of the theta cycle, whereas neurons with recently passed place fields ride on the early phases. Thus, as the animal moves forward, a particular neuron’s sequence position moves from the ascending (late) phase to the descending (early) phase of the theta cycle (known as ‘phase precession’; [46]; Figure 2). This ‘one-in, one-out’ shifting membership keeps the number of cell assemblies within theta cycles relatively constant [39]. In larger environments the place fields become larger and so are their distances from each other [40,47,48]. Therefore, the distance-duration ‘compression’ ratio is larger in larger environments, and the spatial resolution is poorer. In other words, the hippocampus thus can ‘zoom out’ or ‘zoom in’ depending on the size of the environment [40,41]. Rather than an exception, this form of sequenced activity appears to be a common organizing principal across brain regions thought to carry different types of information [49–52].

Figure 2. Distance-duration-speed relationships in the hippocampus.

A. Spiking activity of two place cells and local field potential theta rhythm during maze walking. Temporal duration T is the time needed for the rat to run the distance between the peaks of the two place fields (behavioral time scale). τ, time offset between the two neurons within the theta cycle (“theta time scale”). B. Idealized overlapping place fields of three place cells with identical theta oscillation frequency, illustrating the relationship between T and τ. The relationship between distances of place fields and time offsets (τ) shows a linear relationship within the theta time scale (100–160 ms). C. Movement trajectories of a rat through a place field on two trials with different speeds. D. Spikes of one place cell and the corresponding theta rhythm of the same two trials as in C. Horizontal double arrows indicate the time it took for the rat to run through the place field. E. The number of spikes within the neuron’s place field is similar on slow and fast run trials. Trials are sorted by velocity from slowest to fastest. F. In addition to current position, the start and goal positions can be also read out from the theta phase of spike. Left, the decoded probability (high probability: red) of the rat occupying each track position (y-axis) is calculated at each phase of theta (x-axis, white sine wave). In each subplot, the range of the white sine wave demarks the rat’s actual position. Generally, there is a high probability of the rat occupying its actual position. Theta sequencing can be visualized by diagonal streaks of high probability that begin at the START position on the falling phase of theta and finish at the END position at the rising phase. Right, the same data averaged across all positions actually occupied by the rat. Note that theta sequences are bookended by representations of the linear track START and END positions at the falling and rising phases, respectively. G. Place cell assemblies in the hippocampus are organized by theta oscillations (“hippocampus time”). Each row of dots is a trial of the spiking activity of ten place cells (indicated by different colors). Top panel: Trials are shown with reference to distance through the maze. Middle panel: Trials are shown against elapsed time from start. Bottom panel: Trials are shown as phase-locked activity of neurons in successive theta cycles. A-E, after [58]; F, after [128]; G, courtesy of Carina Curto and Eva Pastalkova.

Therefore, from the experimenter-viewed correlations between positions in the environment and spike-theta phase data, one may conclude that the hippocampus measures distances [47]. However, environmental stimuli alone cannot be sufficient to provide a metric needed for a spatial map. To calibrate distances the animal must move to acquire meaning of the relationships among landmarks [53]. In the absence of movement, place fields fail to emerge or become very large [54,55].

In contrast to the abstract space and time, the brain can directly sense speed and head direction and combine them to determine velocity [53,56,57]. The velocity signal can be conveyed to the hippocampus from the vestibular system, muscle afferents, or optic flow [53], and the distance traveled can be calculated as the product of the number of theta oscillation cycles (in lieu of duration) and the speed gain [58].

Access to running speed information is critical for maintaining a reliable relationship between spike phases of place cells relative to upcoming and previous positions. When the rat traverses the place field of a neuron in one second in one trial and then in half a second during another trial, the place cell will be active for 8 and 4 theta cycles, respectively (assuming 8 Hz theta frequency; Figure 2). The number of spikes within the place field varies relatively little even if the rat’s velocity changes. For this reason, the number of spikes per theta wave doubles. Due to this firing rate gain, which reflects stronger excitation of the neuron as velocity increases, the magnitude of the cycle-to-cycle theta phase shift increases proportionally. As a result, the velocity-gain compensates for the shorter time spent in the place field, leaving the relationship between spike phase and spatial position relatively invariant (Figure 2; [41,58]). Speed-sensitivity of hippocampal neurons decreases from the septal to the temporal pole and, as a result, place fields become correspondingly larger [29,59,60].

Thus, as in physics, the brain is capable of equating distances and durations through velocity- dependent modulation [12,61]. Time computation, as measured from single neurons or assemblies, can be dynamically warped relative to clock time. The ensuing compression or decompression may be brought about by their dynamically changing and state-dependent sensitivity to speed signals. In the absence of movement, mechanisms underlying the subjective term ‘attention’ can function as gain and enhance critical aspects of the encoded or recalled content of mental travel. One can hypothesize, therefore, that attention is a brain-internalized version of speed which can affect the rate of change in sequentially organized activity [9,62].

The hippocampus as a sequence generator

What is the implication of the above-postulated space-time equivalence for the brain? Can space and time assume each other’s role? Below, we put forward the hypothesis that a generic function of the hippocampus is to generate perpetually changing sequences, without the need to refer to the concepts of space or time.

Externally imposed and internally generated cell assembly sequences

Ample experimental evidence illustrates that hippocampal neurons can respond to landmarks [47,63] and signals from the body [53], leading to the tacit assumption that place cell sequences are brought about by changes of external inputs [63,64] as the animal explores its environment. Under this framework, the cognitive map theory would predict that a small set of place cells should fire continuously and at the same exact phase of the theta cycle as long the animal’s head and body are not displaced. In contrast, if assembly sequences are generated by internal mechanisms, neuronal activity should change continually. A key mechanism in sequential organization of hippocampal cell assemblies is the theta oscillation [65].

The theta rhythm is a self-organized pattern in the septo-hippocampal system and provides a mechanism to concatenate neuronal assemblies in appropriate sequences (Figure 2G). In the absence of coordinated theta oscillations, the compressed sequences of neuronal trajectories are strongly compromised, accompanied by behavioral impairment in hippocampus-dependent tasks [65–70]. Since the theta rhythm is an internal ‘timer’ whose units can shrink and expand, depending on brain states, it does not faithfully emulate the passage of clock time (Figure 2G). This ability to warp clock time (and therefore space) may explain why the BOLD signal in human studies shows differences between hippocampal representation of objective spatial distances compared to remembered distances [71].

An early experiment demonstrating that hippocampal sequential activity could be internally referenced to theta oscillations is shown in Figure 3A. Rats were trained to alternate between the left and right arms of a modified T maze and were required to run in a centrally placed wheel for 10 to 20 s at about the same speed while facing the same direction during the delay period between trials. Since both environmental cues and body-derived cues were kept constant by this design, the animal could not use external signals to help maintain the choice information. Instead, it had to rely on the memory of the previously chosen correct arm.

Figure 3. Internally generated assembly sequences in a memory task.

A. Center: The rat was required to run in a running wheel during the delay between trials while remembering its last choice between the left and right arms of the maze. Dots represent spike occurrences of simultaneously recorded hippocampal neurons. Left: normalized firing rate trajectory of neurons during wheel running, ordered by the latency of their peak firing rates during left trials (each line is a single neuron). Right: normalized firing rates of the same neurons during right trials. The experimenter can easily tell the difference between future left or right choices just looking at the neuronal assembly vector any time during wheel running. B. Mean firing rates, each normalized to their own in- task maximum rate (blue- red = 0–1), of all recorded BF neurons (y- axis) across all phases of the task (x- axis). Green, blue, purple, and red arrows demark, respectively, the light flash, nose- poke, plate- cross and stop/reward phases of the task. C. Same as B, for neurons recorded in posterior parietal cortex. D. Firing patterns of C A1 pyramidal neurons during preferred (first column) and non-preferred (second column) choice trials. Third column shows segments with significantly higher discharge rates in choice-L (red) or choice-R (blue) trials E. Tuning curves of goal-distance cells in the bat hippocampus. Note that task-phase intervals have different, and variable clock durations and organize neurons to internal needs rather than to external (clock) time. A, after [72]; B-E, reproduced from [34,82,83].

When enough neurons are recorded simultaneously, the entire journey in the wheel is associated with a neuronal trajectory of perpetually changing cell assemblies. Pyramidal neurons fire at specific times during wheel running on each trial or at the same specific distances run from the beginning of each trial. As there is no displacement of the animal while running in the wheel, these active neurons do not meet the criteria for place cells. Instead, the evolving neuronal trajectory must reflect some cognitive content. In support of this hypothesis, the neuronal trajectories are unique to left or right choice trials, indicating that the initial condition − set by the reward or the planned action − determines the pattern of the neuronal trajectory and reflects future choices with high accuracy, including errors (Figure 3A; [72]). Importantly, the neuronal assembly content of successive theta cycles shifts the same way during wheel running in the memory task as during ambulation while exploring the maze. Overall, these findings show that even though prominent spatial or ideothetic signals can affect the firing patterns of place cells, they are not necessary to induce cell assembly sequences. In fact, the default dynamic of the hippocampal system is perpetual change. Even during sleep, the hippocampus repeatedly generates neuronal sequences [73].

Place cells and time cells?

Although distance run is not a relevant parameter in the wheel running memory task (Figure 3A), both distance and the elapsed time can be computed from either the evolution of the firing rate vectors of neuronal assemblies or their theta phase vectors [20]. In other words, the same neuronal data can be related to distance or time [16,71], depending on which measuring instrument is being used for comparison. Extensive data sets obtained during maze behavior and forced treadmill running or walking on a ball demonstrate that evolving neuronal assemblies obtained in the hippocampus, prefrontal cortex, entorhinal cortex, and parietal cortex can be used to derive run distance or run duration [74–77]. It is primarily the units of the measuring instruments that determine whether the experimenter interprets the neurophysiological data as space or time and refers to them as place or time cells.

In many behavioral situations an organism must operate with a level of flexibility, achieving goals that vary widely across spatial and temporal scales. An optimal strategy for such tasks could be to ‘event-normalize’ the distance or duration intervals between behaviorally relevant stimuli. Such task-specific changing intervals may not have much to do with clock time but may be important in concatenating sequential neuronal events, where the rate of change varies by task demands or internal variables (Figure 3B-E). In support of this hypothesis, task phase-formed cell assemblies have been described in the basal ganglia [78,79], basal forebrain [80], parietal cortex [81,82], and hippocampus [34,83].

Another example of the clock time-independence of neuronal sequences is the variable speed at which such sequences can play out. The same succession of neural events can occur across at least three orders of magnitude in temporal scale. During locomotor behaviors, place cells will fire in behavioral sequences that change on the order of 1–2 seconds (determined by the animals running speed). Simultaneously, the same trajectories are played out at the 100–200 millisecond temporal scale during theta sequences, which are themselves temporally adaptive to running speed and goal distance ([41,84]; Figure 2F). During sleep and other consummatory behaviors in association with hippocampal sharp wave ripples, these ripple sequences can ‘replay’ previous experiences on the temporal scale of 10–20 milliseconds. This is not unique to spatial trajectories and the hippocampus, as head direction assemblies in the thalamus and task-related sequences in the neocortex can be reactivated during non-REM sleep at several-fold compression [85,86]. Thus individual networks and their dynamics do not have a single ‘clock speed’ that can be directly related to the outside world. Rather, neural dynamics are flexible and adaptive, changing the tempo of computation as factors (i.e. behavior or brain state) demand it.

Because the brain has no sensors for directly detecting distance or duration, neither space nor time can ‘cause’ any change in the brain. Therefore, distance and duration cannot be derived from first principles. Instead, these concepts are inevitably inferential and depend on human-made instruments [9]. Although one can often find a correlation between the evolution of some brain measure and units of a clock and can conveniently place measured parameters on a timeline, such correlation may be interpretable only to the experimenter who has access to both the brain measure and instrument-calibrated duration. Yet, such correlation does not warrant the conclusion that brain circuits either sense or compute time.

Time tracking or successive neuronal computation

It is hard to justify a hypothesis according to which there are dedicated neuronal networks whose sole function is to compute time. When scrutinized, neuronal assemblies always turn out to compute something else as well, in addition to tracking duration. For example, neurons in the parietal cortex have been suggested to compute time [87–90]. At the same time, the firing patterns of these neurons can be locked to multiple reference frames; including egocentric actions [91], objects [92,93], and learned routes [81]. With such a diverse set of coordinate systems in mind, it seems that the ability to decode ‘time’ from such signals is merely the consequence of a system capable of carrying “a frame of reference defined by any arbitrary set of points” [94]. In this vein, and similar to the hippocampus [34], basal forebrain neurons operate at a level of abstraction from distances and durations in the physical world [52]. While the action potential timing in these structures may correlate with distance covered and duration elapsed, the number of neurons and the strength of their correlations increases when referenced to task-phase as opposed to units of instruments.

Sequence generation

Instead of a timer, one may consider the hippocampus as a general-purpose sequence generator that carries content-limited ordinal structure, and tiles the gaps between events or places to be linked [11]. Navigation in real or mental space is, by its nature, a succession of events. Perhaps that is all the hippocampal system does: producing content-limited cell assembly sequences without encoding the details of particular events and predicting the best options based on prior experience [95]. Howard Eichenbaum’s classic “transitive inference” study also points to the critical importance of neuronal order [96,97]. Rats were trained to discriminate between pairs of odors that overlap with one another (i.e., odor “A” > odor “B”; odor “B” > odor “C”, etc.). After learning the paired associations, the rat could ‘build’ a sequence, which enabled it to infer the relationships between pairs of odors that were not previously learned (e.g., odor “A” > odor “C”). Rats with hippocampal lesions learned the individual discriminations but could not correctly deduce the relationship between a novel pair (odor “A” > odor “C”).

Whether in reference to rod or clock units, sequences in the hippocampal system may simply point to items (what) stored in the neocortex in the same order as they were experienced during learning. Similar theories have suggested that this division of labor is analogous to the role of a librarian (hippocampus, pointing to the desired items) in a library (neocortex, where semantic knowledge is stored; [98–102]. A key difference between our current perspective and the indexing theory [98] is the ordering of events. Rather than a single index that ‘points’ to cortical modules, the ‘unit’ in our view is a sequence of indices (a ‘multiplexed pointer’) such that the hippocampal system is responsible for concatenating neocortical information chunks into sequences for both encoding and retrieval (Figure 4). With this view, covariation in activity between modalities leads to hippocampal sequence formation that can be decoupled from individual modalities, such that sequences would correlate best with these covariates, which are often distance or duration with current experimental paradigms. However, the demonstrated ability for such sequences to warp relative to externally measured distance and duration suggests that this internal ‘representation’ of space or time would be a highly inaccurate reflection of the external world. With the proposed sequencing theory episodic memory is simply an ordered sequence of translationally invariant ‘whats’ and their related context (another ‘what’), with no explicit internal ‘representation’ of space or time.

Figure 4. Hippocampal sequencing hypothesis.

Indices that point to cortical, and subcortical, modules for different inputs are sequenced by hippocampal activity patterns, thus preserving the ordinal structure over which experience occurs.

Support from human studies

The sequence generator hypothesis of the hippocampus is supported by both lesion and imaging data. As in experimental animal studies, most studies in humans have separately investigated the neural correlates of space [24,103–108] or time [16,17,109-112], the postulated contextual ingredients of episodic memory.

Hippocampus-damaged amnesic patients have much less of a problem with estimating and recalling distances and durations than with remembering the sequential order of events in which they occurred [103,113]. Similarly, rats with hippocampal damage show impaired memory for the order of odor stimuli, in contrast to their preserved capacity to recognize odors that had recently occurred [114].

Human imaging experiments have also provided support for a sequence generator function of the hippocampus, without explicit maps of space or time. A recent experiment contrasted elapsed duration between presentations of objects embedded in a sequence. The findings demonstrated that hippocampal activity patterns carried information about the rank positions of objects in learned but not random sequences, even when the temporal positions of the objects were the same [111]. Another recent study used GPS and camera timestamps of snapshots of events taken automatically by participants who wore a camera over the course of four weeks. Subsequently, when participants were scanned with fMRI while recalling their real-life experiences, their hippocampal pattern of activation was similar to spatial and temporal distances over various scales of magnitude [115]. In a virtual reality study, subjects navigated through streets of a virtual town and were asked to recall whether two objects were close or far. In this experiment as well, neural similarity scaled with the proximity of event memories in both spatial and temporal dimensions, supporting the notion of a common hippocampal coding mechanism for space and time [71].

An independent series of studies examined how experience can be understood through the sequences of events [116]. It was found that sequence memory is more accurate for items presented in the same context compared with events separated by boundaries [117] and that items appearing in the same context are later rated as having appeared more closely together [110] even though actual distance was the same. Higher pattern similarity of BOLD activity was observed during task trials for items which were subjectively judged as ‘close’ as compared to items which were later judged as ‘far’ even though the item pairs were separated by the same number of intervening trials in the task [110]. Overall, the behavioral, lesion, and imaging experiments support the hypothesis that encoding and preserving the sequential order of experiences is a critical function of the hippocampus. Learning the sequential order of events allows us to form predictions about the impending future and plan upcoming actions accordingly.

Transduction of hippocampal assembly sequences to action

Classification of hippocampal neurons as place cells, time cells, or both by the experimenter may be irrelevant for the brain. What is relevant is how downstream reader mechanisms classify hippocampal messages. Short of a demonstration that separate deciphering methods exist for distance and duration, we cannot conclude that hippocampus ‘codes’ for either space or time. To demonstrate the utility of the hypothetical cognitive map, we need to examine how such information is read out by downstream reader mechanisms to guide the animal’s actions.

Because hippocampal neurons do not have direct projections to the muscles, their computation must be transferred elsewhere to inform action selection mechanisms. Candidate output structures include the entorhinal cortex, subiculum and retrosplenial cortex, prefrontal cortex, and the septal nuclei [25]. Of these, the lateral septum is the most likely conduit [118]. Of all targets, it receives the densest hippocampal projections. While the sole corticofugal projection from the hippocampus is the CA1 region, neurons in the lateral septum receive both CA1, and CA3 inputs [23]. The lateral septum also has direct projections to the motivational centers of the hypothalamus [119,120], midline thalamus [121], and brainstem [23]; potentially allowing for the selection of action programs. If the theta-paced sequence of cell assemblies in the hippocampus is relevant for such a hypothetical map-to-action transformation, then the lateral septum should be able to read this syntactic structure. However, previous recording studies have found only heavily degraded spatial information in the firing rates of lateral septal neurons [122–125].

A crucial aspect of this line of reasoning is that the neuronal reader mechanism in the lateral septum should not be referenced to the external world, but to the sequentially ordered activity of its inputs. In this case, the format of neural activity in the lateral septum may be different from the format of spatial correlations within the hippocampus. Indeed, spiking of lateral septal neurons shows a highly reliable correlation between their theta phase and the animal’s position. This is a pure ‘phase code’ because firing rates of lateral septal neurons carry virtually no information about the place the animal occupies and requires phase information from the hippocampus to be observed (Figure 5). Within a single theta cycle, action potentials from lateral septal neuronal assemblies correlate with both the current position and the distance from the beginning and to the end of the learned trajectory through space. This rate-independent phase code is not inherited from the phase or rate information of individual upstream hippocampal place cells but reflects the computation of a continuously changing relationship between CA1 and CA3 place cell assemblies [118]. From the perspective of lateral septal neurons, theta sequences of CA3 and CA1 neuronal assemblies are the ‘unit’ of information transmission out of the hippocampus. Assuming similar transformation mechanisms in downstream hypothalamic and brainstem circuits, such an ‘action selector’ mechanism can quickly identify the source of information and thus mediate action from a higher order map.

Figure 5. Rate-independent position-phase correlations of lateral septal neurons.

A. Neuron in the lateral septum shows phase precession relative to CA1 theta phase. Colored dots indicate the occurrence of action potentials from a single lateral septal neuron while a rat traverses a circular track (~3.2 meters; N = 32 trials). Colors map onto C A1 theta oscillation phases. B. Reconstruction of the rat’s position (mean squared error, MSE) from the firing rate (red) or from a spike phase- position relationship (blue) relative to control (shuffled). Reproduced from [118]

Concluding Remarks

Space and time, as organizing containers of classical physics and neuroscience research, will continue to generate a wealth of research. There is, of course, nothing wrong with using measuring instruments and relating our observations to their units. Formulation of spacetime by physics did not discredit clocks. Quite the contrary, physics and all other sciences rely more and more on the precision of measuring instruments to calibrate experimental observations. It would be unimaginable to perform contemporary neuroscience research without referring to spatial and temporal units. However, declaring from such comparisons that some brain region or mechanism ‘represents’ space and time or calculates distance and duration is another matter. In many cases, we may simply not have found the correct transformation rules that can more effectively explain neural firing dynamics than distance or duration [52] (Outstanding Questions Box).

To meaningfully interpret the significance of any neuronal pattern, it is critical to examine how such patterns are utilized by downstream neural observer mechanisms. It is inconsequential to call sets of hippocampal neurons place cells or time cells without a demonstration of their differential impact on their downstream partner neurons. We illustrated this strategy here by demonstrating that transformation of activity patterns of hippocampal neurons, conceptualized as a cognitive map, into a pure phase-code in the lateral septum depends on theta oscillation-guided neuronal sequences. Studying transformation rules between brain structures does not require resorting to concepts of space or time. In fact, this transformation likely exists for all sequences produced in the hippocampus or other structures, independent of whether we assign these sequences to space, time, or other cognitive constructs.

Highlights.

• We propose that the hippocampus performs a general yet singular algorithm: producing sequential content-free structure to access and organize sensory experiences distributed across cortical modules

• Neural ‘representations’ can be referenced to many frames, and direct comparisons across frames of reference can be used when trying to identify underlying neural computations

• Neural dynamics and transformations can be described without resorting to the concepts of space and time

• Future research should focus on transformation rules between structures, rather than on tuning. A neuronal observer-centered approach that compares two internal variables may be more fruitful than correlating an external signal with neuronal patterns.

Outstanding Questions Box.

• Is it justifiable to attribute dual roles to space and time: spacetime of physics and orthogonal space and time coordinates for brain computation?

• If the concepts of space and time depend on their instrument-measured variants of distance and duration, how can non-human animals conceptualize or utilize these concepts?

• Can we describe neuronal computation without referring to an external spatial and temporal framework?

• Is it necessary to assume an internally generated temporal framework to define neuronal computation?

• Is there an active behavior for which the hippocampus does not produce reliable task- related sequential activity?

• Population activity in any brain structure can track time as measured by the clock. Are there neurons and networks whose sole function is to track time or is it the case that evolving neuronal activity always computes something else (as well)?

• If there are dedicated place cells, time cells, tone-tracking cells, goal cells or reward cells in the hippocampus, who reads them? Postulating such dedicated functions is only useful for the brain if downstream classifiers of such alleged unique functions exist.

• The dominant view of memory consolidation assumes that experience is transiently stored in the hippocampus and transferred to the neocortex over time. This would require veridical coding of waking information in the hippocampus. Alternatively, is it possible that waking information is always stored in the neocortex and the critical role of the hippocampus is to organize orderly sequences of events?

Glossary

Behavioral sequence

A sequence of neural activity that transitions at the rate at which organisms behave and experience the environment

Distance

The shortest route between two positions. This definition makes the relationship between positions and distance circular

Duration

The interval between two time points. This definition makes the relationship between time points (e.g., birth of Christ and today) and duration circular

Gamma rhythm/oscillation

A 30–120 Hz rhythm local field patterns. Gamma oscillation is signature of the competition between excitatory and inhibitory neurons. It reflects a transient balance (oscillating winning and losing) in any brain circuit with these neuron types. The amplitude of gamma oscillation is often modulated by the phase of slower oscillations (such as theta)

Grid cells

Neurons in the medial entorhinal cortex and the parasubiculum, whose increased firing at the apexes of imaginary equilateral triangles on the floor of the environment mark the animal’s position are referred to as grid cells

Hippocampal-entorhinal system

Components of systems do not work in isolation but in cooperation. The hippocampus and entorhinal cortex often works together as a unit, complemented by contribution of the subicular complex.

Place cells

Pyramidal neurons and granule cells in the hippocampus, which become consistently activated at certain parts of the environment are defined as place cells

Ripple sequence

A sequence of neural activity that correlates with the experience of the organism but occurs on a ~10–20 fold compressed timescale. Examples of this are, in some instances, referred to as ‘replay’. See[129–131]

Rod and clock units

All measuring instruments have units, which are arbitrarily defined by its designer. The intervals of these units are always relational and always refer to a measure provided by another instrument (e.g., a period of a pendulum can be calibrated by another faster clock) or natural event (e.g., Lunar cycle, heart rate, circumference of the world)

Theta rhythm/oscillation

A 4 to 9 Hz oscillatory pattern in the local field of the hippocampus when the animal is engaged in exploratory behaviors or REM sleep. It reflects coherent, rhythmic fluctuation of the transmembrane potential in many neurons

Theta sequence

A sequence of neural activity that correlates with the experience of the organism but occurs on a ~10 fold compressed timescale. Examples of this are often referred to as ‘look-ahead’. See [38,41,84]

Time cells

In the hippocampus and entorhinal cortex, principal cells which generate clusters of spikes in the absence of the animal’s displacement (e.g., running in a wheel or on a treadmill) are called time cells. Firing rates of single neurons in multiple brain structures often increase as a function of some experimental variable (e.g., delay) to some threshold. In principle, accumulation of action potentials of a neuron or neuronal population can be used to track time