Cellular dynamical mechanisms for encoding the time and place of events along spatiotemporal trajectories in episodic memory

Abstract

Understanding the mechanisms of episodic memory requires linking behavioural data and lesion effects to data on the dynamics of cellular membrane potentials and population interactions within these brain regions. Linking behavior to specific membrane channels and neurochemicals has implications for therapeutic applications. Lesions of the hippocampus, entorhinal cortex and subcortical nuclei impair episodic memory function in humans and animals, and unit recording data from these regions in behaving animals indicate episodic memory processes. Intracellular recording in these regions demonstrates specific cellular properties including resonance, membrane potential oscillations and bistable persistent spiking that could underlie the encoding and retrieval of episodic trajectories. A model presented here shows how intrinsic dynamical properties of neurons could mediate the encoding of episodic memories as complex spatiotemporal trajectories. The dynamics of neurons allow encoding and retrieval of unique episodic trajectories in multiple continuous dimensions including temporal intervals, personal location, the spatial coordinates and sensory features of perceived objects and generated actions, and associations between these elements. The model also addresses how cellular dynamics could underlie unit firing data suggesting mechanisms for coding continuous dimensions of space, time, sensation and action.

INTRODUCTION

As humans, most of us have personal experience of the phenomenon of episodic memory. We have rich recollections of sequences of events from our recent or remote past that play out in our minds as if we were reliving the experience. For example, I can remember going into the kitchen this morning to prepare my breakfast, and sitting in the dining room eating it. I remember individual movements involved in getting my cereal, and sitting at the table, with distinct memories of where I was facing and where my family members were. Definitions of episodic memory by Tulving describe this experience as the capacity for mental time travel and autonoetic awareness [42, 176, 177]. The process of mental time travel goes beyond forming associations of single items with a single, static behavioural context of location and time, and involves encoding of a full spatiotemporal trajectory. This article will focus on models of the cellular dynamics of episodic memory involving the capacity to relive a sequence of events as a spatiotemporal trajectory with an explicit sense of position in continuous space and duration in continuous time, and an explicit re-experience of factors such as head direction and the direction of movements.

Some researchers have tried to argue that episodic memory and mental time travel are a purely human capacity [165], whereas others argue that animals have this capacity [28, 36]. The personal experience of animals is beyond experimental test, but behavioural data provides a compelling argument that many of the capacities for episodic memory shown in humans can be found in animals [36]. In addition, electrophysiological recordings from animals show phenomena that support the existence of mental time travel along previously experienced trajectories [93, 109]. In addition, many of the qualitative anatomical and physiological features of neural circuits observed in human cortical structures are also found in other mammals [65, 66]. Thus, it is reasonable to suppose that the cellular dynamics mediating episodic memory in humans are also present in animals, even if species differ in the quantity of data on the manifestations of episodic memory. This article will review some of the available behavioural and physiological data indicating the presence of episodic memory in animals, and describe a model of how specific cellular dynamics may be involved in episodic memory. The article will focus primarily on the function of the oscillatory dynamics of membrane potentials and populations, with less focus on the modification of synaptic connections that have been extensively reviewed in other work [12].

Anatomical circuits for episodic memory

The first question is where we should look for the cellular dynamics of episodic memory function? Human data provides some answers about the specific anatomical structures involved in episodic memory (Figure 1A). Considerable attention has focused on the impairments of episodic memory caused in patient HM by the bilateral removal of the anterior hippocampus, the entire entorhinal cortex, and portions of other parahippocampal cortices [30]. Patient HM showed striking deficits in quantitative memory tests that test recall of discrete items from an episode, such as the recall of information from paragraphs, or the free recall of words from a list, or cued recall of paired associates [29, 147]. Other patients with bilateral damage to the hippocampus and parahippocampal cortices also show impairments in these quantitative tests [61, 134]. These lesions indicate the anatomical locus of mechanisms for episodic memory but do not provide physiological mechanisms.

Figure 1. HUMAN AND RAT ANATOMY.

A. Schematic representation of the medial temporal lobe in humans, illustrating structures that are associated with human episodic memory function. The entorhinal cortex receives convergent input from a range of neocortical association cortices. Superficial layers of entorhinal cortex project into the hippocampal formation, which projects back to deep layers of entorhinal cortex. B. Schematic representation of analogous structures in the rat showing entorhinal cortex input to the dentate gyrus (DG) and hippocampal regions CA3 and CA1, as well as connections between the hippocampus and the septum via the fornix (which also contains connections with the mammillary bodies and anterior thalamus). Output from region CA1 reaches the postsubiculum (ps) which projects to entorhinal cortex. Cellular dynamical mechanisms relevant to episodic memory function have been studied extensively with cellular neurophysiological techniques in rats and non-human primates.

Specific behavioural scoring methods have been developed to show impairments of the richness of detail in human episodic recollection [106]. These techniques show significant reductions in the recall of internal details from an episodic memory after lesions of the hippocampus and parahippocampal cortices [96, 162]. As shown in Figure 1, the corresponding structures exist in the rat, allowing detailed experiments on physiological dynamics of neurons in these regions that could mediate episodic memory function. Lesions of the medial temporal cortices also cause significant impairments in the description of future or imagined episodes [68, 141]. In contrast to the impairment of episodic memory, hippocampal and parahippocampal lesions have less effect on long-term semantic memory [162], or simple tests of working memory for familiar items such as digit span [29, 61]. These data indicate the importance of neural circuits in the hippocampus and parahippocampal cortices for the performance of episodic memory. Other lesion data in humans indicates impairments of episodic memory associated with lesions of the anterior thalamus, the mammillary nuclei and the medial prefrontal cortex [3].

Extensive animal research has addressed the memory function of these structures, as reviewed in other articles in this volume. As with the human research, the early studies in non-human primates focused on memory for discrete items within episodes. Studies testing memory for trial unique objects in delayed non-match to sample tasks showed impairments after hippocampal lesions [195, 196] entorhinal lesions [105] and perirhinal and parahippocampal lesions [197]. Other studies in non-human primates have tested memory for associations between visual stimuli and specific spatial locations, indicating that lesions of the fornix impair the construction of a snapshot memory for the spatial location of visual features [52] or associations with responses [53].

Episodic memory in rats

Some tasks used in rats indicate a role for the hippocampus and associated structures such as the entorhinal cortex in episodic memory for complex spatiotemporal trajectories (Figure 1B). For example, the 8-arm radial maze task requires that rats visit 8 different arms without making an error by repeating an arm entry, and the number of arm re-entries is increased by fornix lesions [87, 131]. The rat could avoid the error of repeating an arm entry by sampling each arm and testing for recall of a previous trajectory into that arm on the same day, thereby using a strategy dependent on episodic memory. However, the task could also be performed by avoiding arms with strong familiarity from the same day.

Similarly, the delayed spatial alternation task requires that a rat alternates left and right arm responses, and rat performance is impaired by fornix lesions [4] and entorhinal lesions [8]. Spatial alternation could be performed by episodic retrieval of the most recent trajectory at the choice point [76, 193], but could also be performed by persistent neural activity holding the most recent response in working memory [193]. The Morris water maze has been used extensively to test rat spatial memory, both with a single fixed platform location [119] and with a platform location that changes between days [159]. Impairments of this task appear with lesions of the dorsal entorhinal lesions [160], postsubiculum [171] and lesions of the fornix [44]. A study comparing a fixed starting location to changing starting locations [44] indicates that this task puts demands on the capacity for planning a future trajectory from different starting locations to a goal location.

Recent experiments in rats have focused on potential episodic memory function by testing the specific requirement for memory of what, where and when. For example, rats have been tested for their change in investigation time to objects that were presented at different times and moved to new locations during a recognition period [36]. This task effectively tests memory for what, where and when, but could be performed based on retrieval of discrete single time snapshots of object and location – a single element in an episodic memory. Another test in rats indicates the retrieval of full trajectories. In this task, rats learn trajectories from a central choice point to two different hidden objects in an E-shaped maze [39, 40]. Then the rats are familiarized with one of the objects and must subsequently make a choice to visit the less familiar object. The trajectory followed from the choice point in this task appears to depend on actual episodic retrieval of a prior trajectory, rather than familiarity of cues or even a single snapshot memory [39, 40]. Performance in this task is impaired by fornix lesions. Thus, rat behavioural data support a role for the hippocampus and associated structures in the retrieval of episodic spatiotemporal trajectories for memory-guided behaviour.

Physiological data indicating episodic memory in animals

Another question is whether neural activity recorded at the cellular level in behaving animals indicates mechanisms of episodic memory? In fact, rat physiological data provides a rich source of additional support for the existence of episodic memory in animals. In particular, unit recording data indicates the encoding and retrieval of spatiotemporal trajectories. These data indicate that rat neural circuits can selectively encode the timing of spatial locations within a sequence of events within a trial, and can also selectively encode and discriminate between the timing of events at different spatial locations encountered on different trials.

The strongest evidence for episodic retrieval involves replay of spiking activity in region CA3 of the hippocampus during performance of a tone-cued alternation task [93]. In this task, rats hear a tone that indicates the appropriate direction of response at a later choice point. In this task, hippocampal neurons show selective spatial firing as place cells in different locations in the task, allowing statistical determination of the primary location coded by each neuron. At early stages of learning the task, when the rat is more hesitant at the choice point and turns between different possible response directions at the choice point, the neural activity shows sequential temporal reactivation of neurons coding spatial locations along individual trajectories to the left or right [93]. This indicates the retrieval of these encoded trajectories, and indicates the precise temporal distinction between sequential places visited on one trajectory, as well as indicating the separation of trajectories encountered at longer temporal intervals (the left versus right trajectories).

Other physiological studies analyzed the spiking activity of place cells in region CA1 that fire sequentially as a rat runs back and forth between reward locations at each end of a linear track [33, 37, 47]. During the period of time when the rat is at the end of the track, hippocampal place cells show forward or reversed replay of the sequence of hippocampal place cells that spiked during a preceding run along the linear track, further indicating the selective spatiotemporal retrieval of encoded trajectories, and the temporal separation of distinct episodes.

Other physiological support for episodic memory in rats comes from work on replay of episodes during sleep. Early studies showed reactivation in region CA1 of previously experienced neural ensembles during slow wave sleep [132, 154, 185]. Later studies showed that this activity maintains the spatiotemporal structure of experienced episodes. Hippocampal place cells sequentially activated during waking on a linear track appear to fire with the same sequential relationship during the ripple events in slow wave sleep [122, 154]. Perhaps the most striking replay phenomena concerns hippocampal spiking activity during long periods of waking behaviour on a circular track that are replayed with a similar time scale in association with theta rhythm activity during REM sleep [109]. This replay might be episodic, or could be based on a representation created over multiple learning experiences. This REM sleep replay is temporally structured, showing that the replay occurs at a time scale similar to waking, with time intervals of spiking activity as well as theta rhythmicity that correspond to the time intervals that the rat spent in particular portions of the behavioural task [109]. This indicates that neural circuits in the rat not only encode the order of events, but the time interval of events in an episode.

Synaptic modification and episodic memory function

As summarized above, behavioural and physiological data support the existence of episodic memory in animals, including rats. This raises the further question: What cellular processes in neurons provide the mechanisms for episodic memory? Most cellular work has focused on mechanisms of synaptic modification referred to as long-term potentiation (LTP) and long-term depression (LTD) or as spike-timing dependent plasticity (STDP). In fact, the synapses arising from entorhinal cortex and terminating in the dentate gyrus of the hippocampus (see Figure 1B) were the focus of the first experimental studies of LTP [13]. Subsequently many studies of LTP have focused on this pathway or the synapses of the Schaffer collaterals from CA3 terminating in stratum radiatum of region CA1 [12]. In particular, studies have shown that LTP in the hippocampus has Hebbian properties, depending on the temporal juxtaposition of presynaptic and postsynaptic activity. This was first shown in extracellular studies of the dentate gyrus [107, 114] and then in intracellular studies in CA1 [62, 94, 183]. Early models of hippocampal memory function used Hebbian modification for encoding of static associations between features of items [111, 115, 130, 174] or between items and context [80, 126].

Studies of the requirements for timing of pre and post-synaptic spikes show that presynaptic spikes should precede postsynaptic spikes by less than 40 msec [10, 83, 84, 108], as described with the term spike-timing dependent synaptic plasticity (STDP). STDP has been shown with intracellular recording in many cortical structures [10, 110], and its role in cortical function has been modelled extensively [60, 88].

The Hebbian properties of LTP are consistent with the role of the NMDA receptor in mediating induction of LTP and STDP. The NMDA receptor blocker 2-amino-5-phosphonovaleric acid (APV) prevents the induction of Hebbian LTP and STDP in hippocampal regions [12]. Behavioral studies have demonstrated that infusion of APV during encoding periods slows the learning of a fixed location of a hidden platform in the Morris water maze [119] and strongly impairs learning of a new platform location on each day [120, 159] as well as increasing errors in the 8-arm radial maze [26]. These data indicate that NMDA-dependent Hebbian synaptic modification may be necessary for encoding of spatiotemporal trajectories in episodic memory.

However, is Hebbian STDP sufficient as a cellular mechanism for episodic memory function? Many models have shown how Hebbian properties of STDP could allow the encoding and retrieval of sequences of discrete neural activity [90, 92, 115, 120, 175, 180]. In these models, STDP can mediate chaining of associations between sequentially activated discrete populations, potentially allowing a population of neurons activated at location A to activate a population activated at location B, that can then activate a population at location C.

However, such a chaining mechanism is not sufficient to account for memory of sequences in humans and animals, as a number of studies have shown that recall of sequences can occur despite omissions or transpositions of individual stimuli that would prevent a mechanism based on chaining [22, 81, 137, 173]. In addition, the very short timescale of STDP raises problems for the formation of sequential associations between behavioural items separated by intervals many times longer than the time window for STDP [90, 92, 99, 116]. In addition, humans can remember different temporal intervals between events, but the chaining mechanism using STDP cannot retrieve different time intervals between events because STDP depends upon synaptic strengthening with a brief fixed time window, and the retrieval interval depends upon synaptic transmission with an even faster fixed time course. A further problem concerns the issue of an episodic representation of a continuously varying dimension, such as movement through space or the passage of time, or the expansion of a balloon or fading between different colors in a film. Continuous dimensions are difficult to represent with synaptic links between discrete neural populations. Thus, Hebbian synaptic modification is clearly important, but models based on Hebbian synaptic modification alone have difficulty encoding continuous dimensions such as time and space, and need to be supplemented by mechanisms for coding changes in continuous dimensions within an episode.

Possible intrinsic cellular mechanisms for episodic memory

As described above, models based on synaptic modification alone suffer the problems of chaining and from requiring a discrete and fixed representation of dimensions of time or space or sensory features that are continuous in nature. This indicates a need for further cellular mechanisms that can mediate encoding of continuous dimensions of time, space and sensory features.

Electrophysiological data from the entorhinal cortex indicate cellular mechanisms that could complement synaptic modification for encoding and retrieval of episodic trajectories. These intrinsic cellular mechanisms have been demonstrated using intracellular sharp electrode or whole cell patch recording in entorhinal cortex neurons. Figures 2A and 2B illustrate important intrinsic properties of entorhinal neurons that could contribute to the coding of episodic memory.

Figure 2. Intrinsic properties of entorhinal neurons.

A. Whole cell patch recording in slice preparations from Giocomo and Hasselmo [55] shows that layer II entorhinal stellate cells generate subthreshold membrane potential oscillations in between the generation of action potentials. Blowup focuses on subthreshold oscillations. C. Whole cell patch recording from Yoshida, Fransen and Hasselmo [188] in the presence of cholinergic or mGluR agonists shows that layer III pyramidal cells exhibit persistent spiking that is maintained after the initial induction by a square pulse current injection.

Membrane potential oscillations

One intrinsic feature of neurons that could contribute to episodic coding of continuous dimensions are the subthreshold membrane potential oscillations that appear when entorhinal layer II stellate cells are depolarized near firing threshold [6, 7, 56, 57]. Figure 2A shows an example of subthreshold oscillations [55] with an amplitude of a few millivolts. These oscillations can influence the timing of spikes [48, 133, 138] and may contribute to network theta frequency oscillations in entorhinal cortex [1, 5, 118] and hippocampus [23, 69].

The oscillations in superficial layers may be due to a hyperpolarization activated cation current or h-current [38]. Membrane potential oscillations show differences in frequency along the dorsal to ventral axis of the medial entorhinal cortex [56, 57] that may result from differences in the h current time constant along the dorsal to ventral axis [55]. Membrane potential oscillations appear less prominently in pyramidal cells of superficial layers [6], but are observed in layer V pyramidal cells, where they may be caused by M-current [187]. The layer V membrane potential oscillations also show a gradient in frequency from dorsal to ventral medial entorhinal cortex [54]. Membrane potential oscillations are not as prominent in neurons of the lateral entorhinal cortex [166].

Persistent spiking

The cellular property of persistent spiking may also contribute to the episodic coding of continuous dimensions. Even during the pharmacological blockade of all excitatory and inhibitory synaptic transmission, neurons in the entorhinal cortex demonstrate the capacity to display persistent spiking. Persistent spiking refers to the capacity of neurons to show bistability, in which a neuron showing no spiking can transition to stable persistent spiking activity after a transient depolarizing current injection or transient repetitive synaptic input [41, 49, 98, 167, 188]. In contrast, cortical neurons without persistent spiking will spike only during current injection but will stop after termination of the current injection.

An example of persistent spiking is illustrated in Figure 2B. Some pyramidal neurons in layer II of medial entorhinal cortex show stable persistent spiking whereas others show spiking that self-terminates over periods of many seconds [98]. Pyramidal cells in layer III show stable persistent spiking that can last for two minutes or more [188]. Pyramidal neurons in layer V of entorhinal cortex can maintain stable persistent spiking at different graded frequencies for many minutes [41]. Both bistable and graded persistent spiking appear to be due to muscarinic or metabotropic glutamate activation of a calcium-sensitive non-specific cation current [49, 148, 188]. Graded persistent firing could allow these neurons to integrate synaptic input over extended periods. Persistent firing has also been shown in layer III of lateral entorhinal cortex [167].

The cellular phenomena described above appear in entorhinal neurons even in the presence of synaptic blockers. However, similar phenomena may arise due to circuit mechanisms, as described further below. The dynamics of interacting populations of neurons can result in network oscillations in the theta frequency range [32, 34] or in the gamma frequency range [15, 27, 182]. In addition, persistent spiking can be obtained due to the effect of excitatory and inhibitory synaptic feedback that drives neurons into stable attractor states [78, 184].

Modeling how cellular mechanisms could underlie episodic memory

The cellular mechanisms described here could provide a mechanism for encoding changes in continuous dimension such as time, space and sensory features, and for episodic retrieval of these changes in continuous dimensions. The coding of continuous dimensions for episodic memory could involve either a rate code, in which the firing rate of a neuron varies in a continuous manner, or could involve a phase code, in which the firing time of a neuron relative to a baseline oscillation changes in a continuous manner. A phase code has the advantage that the continuous representation can be coded by single spikes occurring at specific times, rather than requiring multiple spikes for rate coding.

The rhythmic cellular properties of entorhinal neurons described above could allow continuous dimensions to be coded in the form of phase. For example, a pair of neurons showing persistent spiking [41, 167, 188, 189] might have a single baseline frequency, as shown in Figure 3A. A synaptic input to one of these persistent spiking neurons slightly increases the firing frequency for a period of time, moving it progressively out of phase with the spiking phase of the neuron that stays at baseline frequency. In this manner, the relative phase of persistent spiking in the pair of neurons can integrate the synaptic input to one of these neurons.

Figure 3. RELATIVE PHASE CODE.

A. Schematic representation of the coding of excitatory input by a shift in relative phase of two persistent spiking neurons. On the left, the two persistent spiking neurons fire in phase with each other at the same baseline frequency. In the center, an excitatory input drives the lower neuron to a higher spiking frequency for a period of time, shifting its firing phase relative to the baseline neuron. On the right, in the absence of further input, the relative phase of firing maintains a representation of the magnitude and duration of the excitatory input. B. Coding of changes in two spatial dimensions. On the upper left, three neurons fire in phase. Three different movements from this location have different effects. Movement to the right (in the x dimension) shifts the phase of one neuron (x) relative to the baseline (b). Movement downward (in the y dimension) shifts the phase of another neuron (y) relative to baseline. Diagonal movement to the lower right (in both dimensions) shifts the phase of both neurons relative to baseline.

This framework can allow a group of neurons to encode continuous dimensions based on differences in relative phase. For example, as shown in Figure 3B, a two-dimensional environment can be coded by progressive shifts in phase of three persistent spiking neurons. The phase of the baseline neuron is illustrated by the spikes next to the letter “b”. The phase of the neuron coding the spatial dimension x is illustrated by the spikes next to the letter “x”, and the phase of the neuron coding the spatial dimension y is illustrated by the spikes next to the letter “y”. Imagine that movement in the x dimension shifts the firing frequency of the x neuron. As shown in Figure 3B, this results in the x neuron coding a shift in location along the x dimension by a shift in phase relative to the baseline b. Similarly, movement in the y dimension shifts the frequency of the y neuron, resulting in a shift in the spiking phase of y relative to baseline. Diagonal movement shifts the relative phase of both neurons. Thus, the firing phase of these three neurons can code two dimensions. Addition of neurons with a smaller frequency change with input and therefore a slower phase shift allows coding at a different spatial resolutions. These same mechanisms can be applied to the relative phase of subthreshold membrane potential oscillations [54, 57].

This mechanism of coding spatial state by relative phase was initially proposed by Burgess in a model of grid cell firing properties [18, 20, 127]. The essential feature of this model is that the phase of neurons shifted by velocity determines their oscillatory interference, resulting in spiking when neurons are in phase and the absence of spiking when cells are out of phase [18, 20, 77]. This model was developed for oscillatory interactions, but has been modified to use persistent spiking cells [71]. As shown in Figure 4, different populations of persistent spiking neurons with the same baseline firing frequency can drive the activity of a simulated grid cell [71]. Alternately, the interactions of membrane potential oscillations at different frequencies could drive grid cell firing [20, 57, 77].

Figure 4. GRID CELL MODEL.

Mechanism for interaction of persistent firing cells to cause grid cell firing. A. Spiking activity over time of three different groups of persistent firing neurons. Here, each group consists of three persistent spiking cells firing with a baseline frequency of 3 Hz with different phases. Cells receive input from head direction (HD) cells with 0 degree preferred angle for Group 1, 120 degree angle for Group 2, and 240 degree angle for Group 3. Grid cell firing arises from the convergent spiking of the three groups of persistent firing neurons. When all three persistent firing groups fire in synchrony, the grid cell will fire (dots). B. Grid cell spiking (dots) occurs only when all of the persistent firing neurons fire at the same phase, resulting in a typical grid cell firing pattern. Gray line indicates rat trajectory from experimental data (Hafting et al., 2005).

These models of grid cells require a velocity input that drives the shift in frequency of neurons. This is neurophysiologically realistic, as a velocity signal could be constructed as a vector from neurons responding on the basis of head direction [150, 169, 170, 172] along with neurons that respond to translational speed [128, 152]. This model based on oscillatory interference accounts for an impressive range of cellular neurophysiological data, including the pattern of grid cell firing as well as the phenomenon of theta phase precession observed in hippocampal neurons [20, 104, 129] as well as in entorhinal neurons [63]. The model predicted the dorsal to ventral difference in frequency of membrane potential oscillations that was shown experimentally in slices [57], and also correctly predicted differences in intrinsic spiking frequency measured with unit recording in behaving rats [89].

The example presented here focused on the linear coding of spatial dimensions, but the same properties could be applied to other dimensions. For example, a change in brightness could drive firing frequency to cause a phase shift coding the state of brightness, or a change in color could drive a phase shift to code the state of color. Similarly, cells responding to angular velocity [149, 151] could drive a phase shift that codes the shift in visual angle of objects in the visual field. Even complex actions such as expansion could be coded by a population of neurons. The population coding expansion would drive a change in firing frequency in neurons coding the width of an object such as a balloon, causing a progressive change in relative phase that corresponds to the change in width and height of the balloon.

One question about phase coding concerns the capacity to encode dimensions beyond the scale coded by the phase of a single oscillatory cycle. However, this problem can be avoided by using different rates of phase change in different oscillations, which results in different scales of interference. Neurophysiological data on membrane potential oscillations indicates that neurons at different positions along the dorsal to ventral axis of medial entorhinal cortex may respond with different magnitudes of frequency change in response to the same depolarizing signal [54, 56, 57]. In the grid cell model, this results in differences in the size and spacing of grid cell firing fields at different dorsal to ventral anatomical positions consistent with neurophysiological data [17, 57, 64, 140]. The interaction of coding at different spatial scales can effectively code very large ranges according to the least common denominator of interactions [59] and could drive place cells with firing fields of different sizes in the hippocampus [72, 97]. This raises the intriguing possibility that anatomical differences in intrinsic frequencies in other structures such as prefrontal cortex and piriform cortex could underlie differences in the scale of coding for different behaviours [72].

The phase code mechanism presented here has useful features. One feature is that interactions can be positive or negative dependent not on the pattern of connectivity between neurons, but dependent upon their relative phase. This could provide important context effects for cognitive processes. For example, initial conditions could set the phase of a population coding context to be synchronous with neurons coding current state, which would then result in firing of an output population. In contrast, if the initial conditions set a different phase in the population coding context, then the two populations will not drive output, and could even prevent output based on feedback inhibition. Thus, a contextual cue can have a positive or negative influence on a gated output, by a changing relative phase rather than changing from excitatory to inhibitory synaptic connectivity.

The phase relationships of neurons could undergo complex interactions, in which the influence of one neuron on another depends upon dynamics analogous to the ‘mod’ function of their relative phase, such that only the difference in relative phase influences the new spike time. This resembles the mod function commonly used in algorithms for random number generators, and could potentially provide an oscillatory mechanism for the stochastic properties of neural firing observed in many systems.

Episodic memories as spatiotemporal trajectories through multiple sensory dimensions

As noted above, episodic memories can be described as spatiotemporal trajectories through multiple dimensions. The definition of episodic memory already includes a definition of what, where and when. Thus, an event is defined in terms of its spatial coordinates (that can be defined by a two or three dimensional spatial state vector x) and its temporal coordinate (defined by a specific time t). But the term episode or even event does not just mean a static snapshot. The definition of an event includes some action or transition that could be simple (e.g. I saw a sign -- where the action is the movement of eyes or attention to the sign), or as complex as a conversation or purchase of an item. These events involve some transition in state over time (described by the derivative of the state vector - dx/dt). The definition of episode usually includes more than one event, and therefore usually includes more than one action.

A common type of episodic memory concerns a transition between locations over time, such as I drove to the store, or I went to see a film. These can be described as a spatiotemporal trajectory, as shown in the examples in Figure 5, where the movement through space takes place over varying time intervals. In Figure 5A, an agent moves at different speeds in a straight direction along one dimension (y) in a two dimensional space and experiences events at different locations and times. Figure 5B shows the same episode plotted in both time and space as a gray line, with a black line showing the retrieved spatiotemporal trajectory, indicating that the model described here can retrieve both the spatial location and the relative time intervals of events. Figure 5C shows a more complex episode involving movement in two dimensions, and Figure 5D shows the same episode plotted as a spatiotemporal trajectory during encoding (gray) and during retrieval (black).

Figure 5. SPATIOTEMPORAL TRAJECTORIES.

Examples of the encoding and retrieval of episodic spatiotemporal trajectories. The trajectories experienced during encoding are shown as light gray lines, and the events at different times and locations are shown as gray symbols. Black lines and symbols show the retrieval of trajectories and events generated by internal dynamics in the network model. A. Spatial plot of an episode with movement in the y dimension and 5 events. B. Spatiotemporal plot of the same episode in B showing varying speeds in different segments during encoding resulting in long time intervals between nearby events, and showing effective retrieval of events at correct coordinates in both space and time [74]. C. Spatial plot of an episode with movement in two spatial dimensions and five events. D. Spatiotemporal plot of the encoding and retrieval of the episode in C showing effective retrieval of both the time and location of events despite the spatial overlap of the trajectory [74].

Current data does not yet describe the temporal and spatial resolution at which a human can encode and retrieve an episode. The autobiographical memory tests described above [96, 106, 162] use discrete details that could be extracted from a sequence of snapshots equally as well as from a continuous movie-like replay. However, episodes usually involve actions that mediate transitions between points in time and space.

Episodic memories are not just defined in the spatial and temporal dimensions of an agent. The events within an episode could also occur with the agent in a single location,. But in order to be an episode it requires some action, which requires some type of transition. This could be a transition over time by another agent (for example if you watch someone cook a meal, or ride a bike) resulting in an episodic memory in which the relevant dimensions pertain not to the location of the encoding agent, but the location of an observed agent. The transition could also be more abstract, such as listening to a story (in which the actions are conceptual), or watching the sky grow dark (in which the action is a change in a non-spatial sensory dimension - the level of brightness). All of these transitions can be described by a multidimensional state space that can add dimensions dependent upon the relevant events being observed.

Episodes can be encoded by associating states with actions

What cellular mechanisms allow encoding and retrieval of a complete episode? The above description is based on a circuit model that does not just use associations between discrete states, but uses continuous representations of the states in an episode and the actions associated with individual events [73, 74]. The states can be represented by phase coding [20, 57, 71, 127], as described above. This contrasts with an alternative not explored here in which the state could be represented by other circuit mechanisms such as attractor dynamics resulting in grid cells [50, 113].

Episodes can be encoded by having the cellular mechanisms for representing states and actions interact with synaptic mechanisms for forming associations (e.g. spike timing dependent plasticity). During encoding, each individual state within an episode is associated with specific actions at that state via Hebbian synaptic modification between the population of neurons coding the state (e.g. location) and the population coding the action (e.g. velocity). During retrieval of the episode, neural activity spreads from an individual state into the associated action. For example, spiking of place cells representing spatial state (location) could activate transmission at previously strengthened synapses to cause activity in speed-modulated head direction cells representing a stored direction of movement at a particular state (i.e. at the corner, I went Northwest). Persistent spiking in the speed modulated head direction cells [189] could hold this activity to allow these cells to progressively update the grid cell activity to update the spatial state, and thereby update the place cell activity to activate a new associated action.

Review of a cellular model of episodic memory

The association of states with actions has been used in a network model of entorhinal cortex, hippocampus and postsubiculum [73, 74], as shown in Figure 6. During encoding in this model, the current location of an animal is represented by the relative phase of individual oscillations or persistent spiking in grid cells in entorhinal cortex layer II and III [63, 121]. The grid cell firing then drives firing of place cells and context-dependent cells in the hippocampus [19, 51, 113]. During encoding, Hebbian synaptic modification associates the pattern of place cell firing with cells coding velocity based on the pattern of head direction cell firing in postsubiculum [14, 172] and entorhinal cortex layer V [140] and cells coding translational speed [149, 152]. As described above, the head direction cells coupled with cells responding to translational speed could provide a velocity signal at each point on a trajectory. This velocity signal could drive a relative phase code for continuous space based on intrinsic cellular properties that drives the spiking activity of grid cells. The grid cells could then drive spiking activity in a population of hippocampal place cells, as shown in a number of models [73, 136, 156].

Figure 6. MODEL OF EPISODIC MEMORY.

Model of encoding and replay of trajectories. A. During encoding, behavior drives the activity of head direction cells that drive the activity of grid cells in entorhinal cortex layers II and III. The grid cells drive place cell firing in the hippocampus. Links between state (place) and action (speed and head direction) are made by strengthening synapses between place cells and head direction cells W_PH. B. During retrieval, the activity of place cells activates head direction cells coding the velocity from that state which then activates the next encoded location. C-D. The model simulates temporally structured replay of spiking activity of place cells during REM sleep. C shows place cell spiking activity during waking driven by movement through the environment. D shows spiking activity during simulated REM replay showing the same time intervals as during waking.

During encoding, the model experiences single presentations of spatiotemporal trajectories, such as the example trajectories plotted in light gray in Figure 5. During encoding, the activity of head direction cells is driven by afferent input of self-motion cues that update the grid cell activity that then drives the place cell activity. The trajectories are stored by Hebbian modification of synaptic connections between hippocampal place cells in region CA1 and cells coding velocity in the postsubiculum or entorhinal cortex layer V [73]. During retrieval, a particular pattern of hippocampal cell spiking activity causes transmission at previously modified synapses to cause activity in cells of the postsubiculum and deep entorhinal cortex that code head direction and translational speed. The spread of activity from place cells to head direction cells retrieves the action previously associated with a particular location (i.e. at the corner, I went Northwest). The cells coding head direction and speed essentially code velocity. The pattern of activity coding velocity updates the activity of grid cells, which drive a new set of place cells, which drive a new pattern of activity coding velocity.

As shown by the thick black lines in Figure 5, the model effectively encodes and retrieves individual spatiotemporal trajectories, with explicit representation of both the spatial location and the time duration at specific locations. The thick black lines representing trajectory retrieval in Figure 5 are calculated from an inverse transform of the change in relative phase of the entorhinal neurons during retrieval, when no behavioural input is present [74]. Retrieval of the spatiotemporal trajectory involves sequential activation of neurons in the component structures, and therefore retrieval replays the neural activity present in the hippocampus during encoding, thereby effectively simulating the temporally structured replay activity seen during REM sleep [109], as shown in the model output in Figure 6 [73].

During retrieval, the performance of the network is greatly enhanced if the activity representing action at each location is maintained until a new retrieved action updates this activity. This avoids the problem of the action representation being lost when there is no new update based on hippocampal neural activity, which would cause the spatial and temporal intervals between different events to become distorted. The maintenance of spiking activity to represent actions could be provided by the persistent spiking activity shown in slice preparations of the postsubiculum [189], which is a region showing robust head direction activity in behaving animals [149, 170, 172].

The head direction system plays a vital role in this model of episodic memory, providing a representation of both overall orientation and direction of movement during both encoding and retrieval. Loss of components of this system could cause severe impairments in the mechanisms of mental time travel along spatiotemporal trajectories. This could underlie the impairments of episodic memory associated with lesions of the anterior thalamus and mammillary bodies [3]. Head direction cells are found in the lateral mammillary nucleus [158] and the anterior thalamus [58, 157, 190]. The anterior thalamus also shows selective firing during viewing of familiar stimuli in a recognition task [135]. Lesions of the anterior thalamus abolish head direction responses in postsubiculum [58], and could thereby remove access to this information for encoding and retrieval of spatiotemporal trajectories. The presence of theta rhythmic neurons in anterior thalamic and mammillary nuclei close to head direction cells [178] suggests that rhythmic firing could code the distance and time intervals between head direction changes.

In the model described above, the spatiotemporal trajectory is encoded in entirety during one period of time, and retrieved in entirety during a separate time period. However, the encoding and retrieval of segments of the trajectory could occur in an interleaved manner if there were changes in the influence on the postsubiculum alternating between external input (rat velocity) and internal retrieval (hippocampal input to postsubiculum). Rhythmic changes in hippocampal retrieval driving postsubiculum could underlie the place by head direction cells found in postsubiculum [25]. This would allow separate phases that alternate between encoding current trajectory input and retreival of previously experienced trajectory segments [75, 125].

Input determines coding of place, length or time

The nature of the input regulating neural frequency determines the information coded by relative phase in the model. In contrast to the coding with velocity input described above, the input of speed could allow coding of length, and the representation of continuous time intervals could arise from interference of oscillations at slightly different fixed frequencies. The mechanism using velocity input can code Euclidean space, but has difficulty with the coding of overlapping trajectories, and with coding of the temporal duration of stationary periods. These properties can be provided by the additional role of cells in which the membrane potential oscillations or persistent spiking do not depend on velocity, but respond only to speed input, thereby coding the arc length of the trajectory [70]. Alternately, the coding of time can be provided by oscillatory interference of cells that keep the same frequency over time, causing relative phase to directly code continuous temporal intervals instead of continuous space [77]. Both the arc length code and the temporal interval code can overcome the problem of spatiotemporal trajectories that overlap in the same spatial location at different times. The integration of velocity cannot differentiate two visits to the same spatial location, but the integration of speed gives arc length of the trajectory, which differs for an early visit versus a later visit to the same location. Similarly, a fixed oscillation frequency gives a change in relative phase over time that provides a different temporal code for an early visit versus a late visit to the same spatial location. The fixed oscillation frequency will also allow relative phase to code the temporal duration spent stationary at a single location [73, 74, 77]. Thus, depending on what input influences frequency, relative phase can encode Euclidean distance, arc length or temporal intervals. The phase reset of temporal oscillations regulated by velocity could also provide context-dependent activity in the grid cell model driven by velocity [71]. The continuous representation of time presented here resembles the oscillatory codes for encoding word order in immediate serial recall models [16, 21], or the temporal context model used to model conditional response probability in free recall [85, 86]. In addition, this use of oscillations resembles the use of oscillations for encoding temporal intervals in models of the timing of behavioural responses [112, 117].

The simulated versions of the model above use sequential activation of place cells or arc length cells and are still vulnerable to the problem of chaining models. However, as noted above, the timing mechanisms can run concurrently in multiple persistent spiking cells to provide redundancy and avoid the chaining problems. In this framework, new items or features are proposed to activate a subpopulation of persistent spiking cells with stable frequency that shift in spiking phase relative to other persistent spiking cells. Downstream cells could respond to these cells by spiking when the shifting phases are close in time, allowing generation of spiking that codes time intervals from the onset of persistent spiking. This allows redundant coding of the timing of items or events in a sequence such that retrieval of each could depend on multiple prior onset cues.

A general model of episodic memory

A general model of episodic memory would include a wide range of possible dimensions, each of which could cue subsequent dimensions. In this framework, the initial sensory state of an organism would involve a pattern of neural activity in a population of neurons, potentially using phase coding to represent the initial dimensions. As the agent moves through the environment, velocity input could update the phase code of location, and angular velocity input could update the representation of head direction. When a new object is encountered, the match between the input pattern and previously modified synaptic connectivity could drive a new population of persistent spiking neurons over threshold. Once over threshold, the newly activated population of neurons would show persistent rhythmic spiking with specific phases relative to other neurons that depend on the strength of a feature of the initial input, thereby representing the initial dimensions of the object, such as spatial location, object orientation, object size or object color. Any changes in the features of an object would cause synaptic input that would drive shifts in frequency of the neurons coding the object, to alter the relative phase representing that feature of the object. For example, if the object were turned by 30 degrees in orientation, this could cause synaptic input coding the rotation velocity of the object that would shift the phase of spiking representing orientation to a different phase of spiking relative to other spiking activity. The associations of events and items could involve interactions of spatial codes in medial entorhinal cortex with item representations in lateral entorhinal cortex, which shows less spatial specificity [43, 67] but shows object responses.

The time intervals of the episode could be encoded by neurons with fixed frequency differences that progressively shift in phase relative to each other, resulting in a precise code for relative time intervals. For example, the population of neurons activated by the first object might include some neurons with slight frequency differences that result in a progressive change in relative phase such that different neurons are synchronized at different time points. The synchronized firing could drive other neurons at specific time intervals, coding time since the first object appeared. Now imagine that a second object appears. The neurons driven by synchronous activity due to phase shifts coding the temporal interval since the first object will strengthen synapses with neurons coding this second object, and with later objects. In this framework, each new object or dimension starts interference mechanisms that time the intervals to subsequent objects. All of these timing mechanisms run concurrently, so that any object can cue any subsequent object. This avoids the problem of the synaptic chaining framework in which only one preceding object can cue the next object, and avoids the problem of the fixed short time frame dependent on synaptic transmission. This modeling could be formalized mathematically as analogous to splines [102, 143], in which the state representation provides knots and the actions provide the knot vectors.

The state representation of space in a given environment will be shared between episodic trajectories. This might result in the appearance of place cells with stable spatial coordinates in a given environment [127] that could be combined with other state representations to be linked to multiple different spatiotemporal trajectories. In some cases, behavioural tasks with similar spatial location in different behavioral context might functionally require distinct context-dependent responses such as those that appear in neurophysiological data [45, 103, 155, 186]. In these cases, context-dependent firing could be driven by oscillatory interference coding arc length [70] or by the interval since reset due to previous reward or sensory stimulation [71]. In some cases, the context-dependent firing of neurons appears to depend not on the actual trajectory followed by the rat, but on the set of possible trajectories that a rat could follow within a certain configuration of barriers [35]. This might be due to scaling of the contribution of individual head direction angles based on the possible movement along that dimension, possibly analogous to the computation of all possible paths in a given environment [46].

Network dynamics might enhance cellular phase code

A phase code could also arise from oscillatory dynamics involving feedback interactions between excitatory neurons and inhibitory cortical interneurons. Numerous studies have shown that circuits of excitatory neurons interacting with inhibitory interneurons can cause oscillatory dynamics at gamma frequency [27, 182]. This could allow phase coding of memories relative to gamma oscillations in neocortical structures [153]. More complex network level dynamical interactions can cause oscillatory dynamics at theta frequency [31, 32, 34, 101, 133, 139]. Circuits that generate synchronous rhythmic activity of neurons have the potential for generating phasic firing of neurons at different phase relationships. If external depolarizing input causes a shift in frequency of one oscillation, then this will cause shifts in relative phase of spiking in different groups of neurons, providing a phase code as described above.

One problem that confronts the models of grid cells based on intrinsic mechanisms concerns the effect of phase noise. As seen in Figure 2B, membrane potential oscillations show high variability in oscillation period, and persistent spiking activity shows variability in spiking phase. Simulations with this level of variability show a rapid loss of coding accuracy [54, 181, 191]. However, these effects of noise could be reduced by network interactions. Experimental data shows that individual stellate cells receiving input from a dynamic clamp replicating excitatory interactions with other stellate cells will synchronize [123, 124]. Thus, stellate cells firing rhythmically in response to external input will shift into phase with each other due to recurrent excitatory coupling. This synchronization on the population level should be able to overcome the independent variability of the intrinsic mechanisms for membrane potential oscillations or persistent spiking. Simulations have demonstrated that network dynamics can maintain synchrony despite noise within individual neurons [191].

A phase code could involve an interaction of persistent firing cells and cells showing membrane potential oscillations. Intrinsic persistent spiking cells in medial entorhinal cortex [188] or postsubiculum [189] could drive the stellate cells in layer II that would have weak excitatory interactions sufficient for synchronization but not strong enough to change the overall frequency of the circuit [1, 124, 138]. A similar interaction could occur between persistent spiking cells in layer III of entorhinal cortex and local circuits in region CA1 that generate synchronization through interactions of pyramidal cells, and two types of interneurons: fast spiking cells (FS) and oriens-lacunosum-moleculare (OLM) cells [124, 139]. These CA1 circuits could interact with entorhinal circuits because of the topographic relationship between entorhinal projections to CA1 and the return projections from CA1 to deep layers of entorhinal cortex [168].

The cholinergic modulation of intrinsic properties could influence the generation of oscillations. Cholinergic modulation has been shown to enhance theta rhythm oscillations in the hippocampus [11, 100]. On a single cell level, cholinergic modulation lowers the resonance frequency of entorhinal stellate cells [82]. By reducing neuronal intrinsic frequencies, acetylcholine could cause an increase in the size and spacing of grid cell firing fields observed in novel environments [9]. Microdialysis shows increases in cortical acetylcholine levels in novel environments [2].

Interaction of memory systems

Previous modeling work demonstrates that tasks performed using episodic retrieval of spatiotemporal trajectories would also require a role for working memory or semantic memory. Working memory could underlie the human capacity for immediate recall of sequential verbal information using mechanisms that may depend upon phase codes or rate codes for temporal order [22, 81, 91]. This provides an important mechanism that could contribute to encoding of episodic memory. Working memory for multiple items based on persistent activity could be used to directly solve behavioural tasks [193, 194], or to provide input or output for an episodic store based on synaptic modification [192]. In support of this, human imaging data shows that persistent activity in the absence of a stimulus is correlated with the subsequent memory for that stimulus at a later time [144, 145] and shows load effects dependent on number of items held during a delay [146]. The mechanisms of persistent spiking might play an important role in the neural activity present in the hippocampus and parahippocampal cortices during working memory for novel stimuli [79, 164] and during the encoding of novel information into long term memory [95, 163, 179].

The new modeling framework presented here provides potential mechanisms for simultaneously modeling the interaction of memory systems such as working memory and episodic memory. For example, the active maintenance of phase in multiple different neurons can mediate working memory for the value of spatial locations or features on many different dimensions. However, once this working memory for state causes activity to spread across previously modified synapses to activate previously associated actions, then the working memory has cued retrieval of episodic memory. The new retrieved state would be held in working memory. Thus, this framework uses an ongoing interaction of working memory with episodic memory for memory-guided behaviour. Mathematical analysis shows how this interaction of memory systems can disambiguate individual states in behavioural tasks [192]. Reinforcement learning mechanisms can be used to guide the encoding and retrieval of episodic spatiotemporal trajectories to guide behaviour [193].

In general, working memory for state can interact with synaptic retrieval of actions guiding transitions to previously learned outcome states. This same mechanism can be used to mentally project novel trajectories through state and action space that allows mental time travel through imaginary or future locations. Instead of temporal intervals due to interference or a recurrent loop driving the retrieval of a previous trajectory, the actions along the trajectory could be determined by prefrontal input to the cells coding velocity. For example, to imagine arbitrary movement through a familiar house, semantic memory could activate memory of the front hallway, and prefrontal cortex could generate a representation of action (going forward). These cells could then drive the phase code of grid cells to progressively update place cell populations representing different locations, and thereby activate associations with items at particular locations. At the end of the imagined hallway, the prefrontal cortex could generate an action to go left or go right. This new action would then update the phase code of grid cells to drive place cells representing a location in a different room, and associations with items in that room. This resembles the overall framework used in previous simulations of interactions of prefrontal cortex with medial temporal and parietal cortices [24], but the model presented here focuses on understanding the role of specific cellular intrinsic properties mediating coding of both time and space. The same cellular mechanisms described here may underlie the role of parahippocampal and hippocampal structures in encoding and mental time travel during retrieval of episodic memory [161] as well as the mental time travel involving imagination of future experiences [142].