Reversed and forward buffering of behavioral spike sequences enables retrospective and prospective retrieval in hippocampal regions CA3 and CA1

Abstract

We propose a mechanism to explain both retrospective and prospective recall activity found in experimental data from hippocampal regions CA3 and CA1. Our model of temporal context dependent episodic memory replicates reverse recall in CA1, as recently recorded and published (Foster & Wilson, 2006), as well as the prospective and retrospective activity recorded in region CA3 during spatial tasks (Johnson & Redish, 2006). We suppose that CA3 encodes episodic memory of both forward and reversed sequences of perforant path spikes representing place input. Using a persistent firing buffer mechanism in layer II of entorhinal cortex, simulated episodic learning involves dentate gyrus, layer III of entorhinal cortex, and hippocampal regions CA3 and CA1. Associations are formed between buffered episodic cues, unique temporal context specific representations in dentate gyrus, and episodic memory in the CA3 recurrent network.

I. Introduction

Electrophysiological data provides ample evidence for the involvement of hippocampal regions CA3 and CA1 in spatial navigation behavior. Foster and Wilson (2006) recently made a detailed study of the sharp-wave ripple activity of head-direction sensitive place cells in CA1 during awake pauses at the ends of a traversed linear track. They found reversed order reactivations of the place cells, confirming early model predictions by Buzsáki (1989). Johnson and Redish (2007) recorded neural activity in CA3 in a cued T-maze task, and during runs in a multiple-T maze task, in which decisions at the final choice point carried a high cost of errors. They emphasize that when decisions must be made, prospective sweeps of place cell activity occur during theta rhythm. The literature does not yet offer an explanation, in the form of a model mechanism, that adequately addresses the experimental evidence of retrospective and prospective activity during runs (theta rhythm) and pauses (ripple activity) in both CA3 and CA1 regions.

At the end of a linear track, reversed order reactivation of place cell spiking might be explained by the inverse relationship between a place cell’s spiking probability and the distance from the center of its place field to a rat’s current location. The probabilistic mechanism assumed by this distance hypothesis would affect the rate and not the relative timing of spikes at place cells whose field centers are at different distances from current location, and would therefore not predict repeated sequences of reactivation in the correct reversed order. Foster and Wilson largely ruled out the spatial distance hypothesis, since its predicted patterns of place cell activity related to current location were not recorded at the start of a session (Foster & Wilson, 2006). Furthermore, experimental results of Johnson and Redish (2006) showed that reverse sequences of activity can also commence at place cells that represent goal locations.

Colgin and Moser (2006) speculate that Schaffer collateral synapses retain a measure of facilitation within a brief time-window after novel behavior, so that recently activated neurons are more likely to spike. It is unclear if the duration of a supposed brief facilitation at input synapses to CA1 could suffice to produce multiple cycles of sequence reactivation, so that spike-timing dependent plasticity (STDP) can establish corresponding associations between the spiking place cells in CA1 via recurrent pathways or recurrent fibers, which are mainly found in CA3. Evidence for the involvement of reactivation in CA3, prior to propagation through Shaffer collaterals, is provided in the data of Johnson and Redish (2006) and Jackson, Johnson, and Redish (2006).

Previously, we simulated temporal context dependent episodic memory in a delayed spatial alternation task, using integrate-and-fire models of network function in layers II and III of entorhinal cortex (ECII and ECIII), in dentate gyrus (DG) and in hippocampal regions CA3 and CA1 (Koene & Hasselmo, 2006, 2007c). Reversed replay of spatial locations remembered in CA3 indicated which direction had been chosen on the last traversal of a choice point in a T-maze with left and right return arms (Hasselmo & Eichenbaum, 2005). The spiking output of CA3 converged in CA1 with spiking output of ECIII, which represented adjacent spatial locations (explored paths), enabling correctly alternating direction choices.

Here, we propose a mechanism to explain both the retrospective and prospective findings in the experimental data. Reversed encoding may be the cause of reverse sequence replay in both CA3 and CA1. We hypothesize that converging forward and reversed buffered input to CA3 during encoding establishes episodic memories that enable strong prospective and retrospective retrieval spiking of place cells in CA3 and CA1. In fact, reversed order associations may be an essential resource that needs to be established automatically in anticipation of future decision-making that relies on temporal context dependent episodic memory (Hasselmo & Eichenbaum, 2005). Our model achieves this through alternating encoding and retrieval intervals in each cycle of the theta rhythm during mobility.

II. The Model

For the purpose of our simulations, we regard the hippocampus as a gate-keeper that flexibly creates new context dependent associations, involving states of perception, goal representations and behavioral rules. The new associations enable reliable ensemble effects based on multiple sources of noisy spiking input.

Fig. 1b depicts the simulated linear track task environment. Traversal of a T-maze with left and right return arms was simulated in the delayed spatial alternation task.

Figure 1.

(a) Associations encoded and retrieved in the model. An episodic cue formed by buffered input in ECII is associated with a temporal context specific representation in DG, which is in turn associated with successive temporal context specific representations. A representation in DG is associated with a sequence memory in CA3 that is a reversed representation of an entorhinal episode of spikes received during encoding. (b) The Catacomb integrate-and-fire neuron model of temporal context dependent episodic memory. Input spikes enter forward buffer (64 pyrmidal neurons with ADP) and reverse buffer (64 pyramidal neurons with ADP) models of entorhinal cortex layer II (ECII). Buffer output enables encoding in recurrent network models of entorhinal cortex layer III (ECIII, 16 pyramidal neurons) and of hippocampal region CA3 (32 head-direction sensitive pyramidal place cells). During encoding, unique spike patterns in dentate gyrus (DG, 16 excitatory neurons) are associated with the ECII output and with episodic encoding activity in CA3. During retrieval, spikes in hippocampal region CA1 (16 pyramidal neurons) are elicited by input from ECIII that is gated by the episodic activity retrieved in CA3, due to input from DG.

In prior work, we provided detailed descriptions of our forward and reversed order short-term buffer models (Koene & Hasselmo, 2007b, 2007a), and of our episodic memory models with alternating encoding and retrieval in each theta cycle (Koene, Gorchetchnikov, Cannon, & Hasselmo, 2003; Cannon, Hasselmo, & Koene, 2003). Here and in the Appendix, we include a short summary of the model structure and functions. In ECII pyramidal neurons, we simulate persistent firing, due to after-depolarization (ADP) in the presence of cholinergic modulation (Klink & Alonso, 1997; Fransén, Alonso, & Hasselmo, 2002). During short-term buffer function, ADP is saturated, so that each spike is followed by a reset and a ramp-up with the same amplitude and time-constant.

The response of leaky integrate-and-fire model neurons in all regions is affected by specific post-synaptic and intrinsic membrane currents that are characterized by individual values of conductance (g_i) and reversal potential (E_rev,i). Each current is modeled by a double exponential function (see Appendix). The common currents include: A membrane leak current, regular input responsible for modulation at theta rhythm at 8 Hz that originates in the medial septum, afferent stimulus input to the buffer (different sources of excitatory and inhibitory input are specified in the Appendix), and intrinsic after-hyperpolarizing responses to action potentials. In addition, the ECII pyramidal neurons have an intrinsic after-depolarizing response to action potentials and in response to each pyramidal spike receive network-wide inhibitory synaptic input from the recurrent fibres of an interneuron network that is responsible for observed modulation at gamma frequency (25-50 Hz) and maintains a separation of buffered pattern reactivation (Lisman & Idiart, 1995). A spike depolarizes the membrane to 0 mV. A characteristic membrane capacitance and leak current determine the time course of an exponential decay of membrane potential to a characteristic resting potential.

Places (and “contexts” in DG) are represented by patterns of simultaneous spikes. Modulation during theta rhythm produces asymmetric intervals with specific functional preferences in the modeled regions. In forward and reverse ECII buffer populations, afferent task-related “place” input (G_i = 32 nS, T_rise = 1 ms, T_fall = 2 ms) is accepted during the hyperpolarized interval, while the ordered, rhythmic reactivation of buffered spike patterns occurs during the depolarized interval (Jacobs, Kahana, Ekstrom, & Fried, 2007). During the reactivation interval of ECII that coincides with hyperpolarization by theta modulation in ECIII and CA3, afferent input from ECII to ECIII (G_i = 0.6 nS, T_rise = 1 ms, T_fall = 2 ms) and from ECII to CA3 (G_i = 0.15 nS, T_rise = 1 ms, T_fall = 4 ms) leads to encoding by STDP at recurrent synapses. The period of depolarized by theta modulation in ECIII and CA3 favors associative retrieval through recurrent fibers (Hölscher, Anwyl, & Rowan, 1997; Wyble, Hyman, Goyal, & Hasselmo, 2001).

The mechanism that triggers pattern replacement in a full forward-ordered buffer does not depend on the number of spikes that represent a place. Instead, the mechanism detects if two conditions are met: (1) Afferent input spikes appear. (2) Spikes occur in that phase of the theta cycle, at which the spikes of the last reactivated spike pattern in a buffer filled to capacity are expected (see Appendix). Reactivation of the oldest buffered pattern is then suppressed by phase-locked inhibition, to make room for the spike pattern elicited by afferent input.

Persistent firing in the “ECII forward” and “ECII reversed” buffers (Fig. 1b) maintains short sequences of spike patterns, in their original (Fig. 2b) and reversed (Fig. 2c) order (Koene & Hasselmo, 2007b, 2007a). Reversed short-term buffering of input is achieved by accepting afferent input at a different phase of the theta cycle. In a buffer with the reversing property, the first reactivation of novel input by the after-depolarizing response of ECII pyramidal neurons occurs at the onset of reactivated spiking in the next buffer cycle (Koene & Hasselmo, 2007b). The buffers operate at theta frequency in the presence of high levels of acetylcholine (Fig. 2a), and are cleared during awake or sleep periods without theta rhythm. Such buffering may be needed during encoding, so that time-compressed repetition of the buffer output enables spike-timing dependent plasticity (STDP) of synapses that encodes sequence associations in hippocampal recurrent networks (Koene et al., 2003). The performance of this buffer model has been tested in realistic noisy conditions (Fig. 2d). STDP can encode the ordinal distance between buffer output spikes in terms of the resulting connection strength. In simulations with noise, the original time difference between buffered input may also be encoded, due to a gradual reduction of buffer output representing earlier items (Fig. 2d).

Figure 2.

A persistent spiking buffer model of layer II of entorhinal cortex. (a) Responses of a model ECII pyramidal neuron that receives current injection over 400 ms at low and high levels of acetylcholine (ACh), and when regulated by theta rhythm. (b) The membrane response of one neuron in each of the item representations A to F, during spike buffering in order for up to four item spike patterns. First-in first-out item replacement is effected if the buffer receives additional input. Spikes are reactivated at order-specific phases of rhythmic buffer cycles, and followed by hyperpolarization and the onset of an after-depolarizing response. (c) The membrane potential of neurons belonging to six different representations A to F buffered in reversed order. First item (A) spiking terminates after the sixth item (F) enters the buffer. The limited interval of depolarization during a theta cycle constraints the capacity of this model buffer to five items. Despite reversing buffered spikes, the buffer replaces items on a first-in first-out basis. (d) Even with strong noise, when desynchronization causes a gradual drop-out of spikes, the order of buffered spikes is preserved (see inserts).

Perforant, mossy fiber and Schaffer collateral paths of the present model are shown in Fig. 1b. During spatial exploration, recurrent associations (G_i is learned, T_rise = 2 ms, T_fall = 4 ms) that signify adjacent place fields are established between pyramidal neurons in ECIII (Fig. 3d), due to repeated ordered spikes elicited by input from ECII (Fig. 3a). Region CA3 also receives perforant path input from ECII, in the form of both the forward (Fig. 3a) and reversed (Fig 3b) buffered spike sequences and encodes those as episodic memory by strengthening recurrent excitatory synapses (G_i is learned, T_rise = 4 ms, T_fall = 10 ms), according to a graded function of spike timing dependent plasticity. Spiking in model CA3 (Fig. 3e) during the initial encoding intervals of theta cycles depends on a combination of increasing depolarization due to rhythmic theta modulation of membrane potential and sufficiently strong perforant path input. Perforant path synapses from ECII to CA3 that transmit sequence input can be strengthened during earlier appearances of the individual spike patterns, which raises the possibility of one-shot learning that can occur with episodes composed of known items (Moser & Moser, 2003). We assume that the very first CA3 spike of a new sequence also depends on input from DG (Fig. 3c), establishing an association between activity in the two regions¹.

Figure 3.

Plots of neuron membrane potentials during encoding in the model of temporal context dependent episodic memory, between t = 2250 ms and t = 2750 ms. (a) Forward order buffering of the first four place representations involves 16 spikes. After t = 2500 ms, the oldest place in the buffer is replaced with a 4 spike representation of the fifth place. (b) The reverse order buffer operates similarly, though with identical model parameters it can maintain five place representations, due to its simpler mechanism for item-replacement when filled to capacity (Koene & Hasselmo, 2007a). (c) Independent spikes in dentate gyrus are associated with the output of entorhinal cortex layer II, thereby generating unique representations for a specific temporal context. (d) Successive place spikes elicited in ECIII fall within the window of STDP, so that associations are formed between adjacent places. (e) Similarly, STDP establishes associations in CA3, involving spikes elicited by forward (top half of the plot) and reversed (bottom half of the plot) sequences of perforant path input.

We presuppose that a unique pattern of spikes in dentate gyrus (DG) accompanies each temporal context, encoding associations between those spiking neurons and simultaneously spiking neurons in ECII and CA3. Context specific patterns of input or random depolarization² may be responsible for the unique spike patterns in DG. The encoding protocol binds each specific sequence of episodic cue spikes in ECII with a unique representation in DG, and binds those to episodic memory encoded in the recurrent CA3 network.

In the hippocampus, we assume that the recurrent CA3 network and the recurrent network formed by layers of DG (Lisman, 1999) are each engaged in heteroassociative encoding (Fig. 1a).

Context dependent retrieval in our model is elicited through either of two paths. A PFC pathway into our model provides spike pattern input that elicits suprathreshold post-synaptic responses at target DG neurons during goal directed retrieval of a specific context representation, and to elicit ripple activity in the hippocampus. Alternatively, perceiving a familiar context can generate a known spike sequence in ECII, which is a contextual cue for the retrieval of a unique spiking representation in DG. During the depolarizing “retrieval” phase of a theta cycle, or during up-regulated transmission of non-theta sharp-waves, spiking activity in DG elicits retrospective or prospective retrieval of associated episodic memory in CA3 (G_i was learned, T_rise = 1 ms, T_fall = 4 ms). During simulated non-theta periods, the transmission of entorhinal activity through perforant path synapses in the model is reduced (Lisman, Talamini, & Raffone, 2005), simulating observed replay without environmental stimulation (Lee & Wilson, 2002) and characteristic sharp-wave spiking at ripple frequency in CA3 (Buzsáki, 1986). For computational efficiency, the demonstrateed non-theta periods in each case account for less than one simulated second.

Retrospective and prospective sequence spikes are elicited in model CA1, where forward and reverse episodic replay in CA3 converges (CA1 from CA3: G_i = 0.2 nS, T_rise = 4 ms, T_fall = 4 ms) with retrieved feature associations in ECIII (CA1 from ECIII: G_i = 0.7 nS, T_rise = 0.5 ms, T_fall = 2 ms). Retrieval in the model hippocampal networks includes a winner-take-all strategy, implemented by competitive recurrent inhibition via interneurons in region CA3 (Sik, Penttonen, Ylinen, & Buzsaki, 1995).

III. Results

A. Reverse replay of behavioral sequences in hippocampal place cells in the linear track

Our simulation of the Foster and Wilson task assumes 8 non-overlapping place field representations of equal diameter (20 cm) along a 160 cm linear track. The virtual rat takes 625 ms (5 theta cycles) to travel through one place field into a non-overlapping adjacent place field, an average speed of 32 cm/s. Within each place field, spikes are elicited in a distinct set of four pyramidal neurons buffering in forward order (Fig. 4a)), as well as four pyramidal neurons buffering in the reversed order (Fig. 4b)). The sets of buffer neurons represent that spatial location. Analogous to Foster and Wilson (2006), who did not include bidirectional place cells in their final analysis, we model only the activity of hippocampal neurons that respond in a unidirectional manner. In the same manner, our model ECIII pyramidal neurons respond unidirectionally (Sargolini et al., 2006).

Figure 4.

Activity of neurons in model layers II and III of entorhinal cortex, during our simulation of the Foster and Wilson (2006) linear-track behavioral task. (a) Buffered persistent spiking of a sequence of track representations in the same order as recently traversed place fields in the linear track. (b) Persistent spiking of a reversed sequence of spikes representing traversed place fields. (c) Spike sequences representing neighboring place fields on the track are elicited by input from ECII during theta rhythm. At reward locations at the end and start of the track, spikes representing the virtual rat’s location are elicited, but no associated features are retrieved.

On simulated runs, the virtual rat starts after one theta cycle (at t = 125 ms), traverses each of the place fields on the track, reaches a food-reward location at the end of the track (at t = 4500 ms) and pauses there (until t = 5750 ms), before it returns along the track to the start location. During the period in the reward location, our hippocampal simulation enters an awake state without theta rhythmic subthreshold oscillations (from t = 5025 ms), when simulated sharp-wave activity dominates output of CA1. During this period, there is no persistent spiking in ECII, due to reduced cholinergic modulation. The most recent context representation in DG is reactivated by input that may be mediated by prefrontal activity, and elicits retrieval in CA3. Figs. 5b and 5c show that episodic associations in CA3 elicit a ripple of spikes in the reversed order of the place field traversal. Both neurons that were initially activated by forward and by reversed sequence input from ECII are affected by the reversal, because the coincidence of their spikes during encoding in CA3 resulted in associations between neurons of both order representations. Some minor mixing of forward and reversed order occurs in the tail of each retrieved sequence, which resembles the variance seen in the spike plots of Foster and Wilson (2006, Fig.3). Both high frequency sharp wave (ripple) activity in CA3 and in ECIII can contribute to model CA1 activity during non-theta periods (Chrobak & Buzsáki, 1996; Remondes & Schuman, 2004), but in this case activity in CA1 is dominated by Schaffer collateral input from CA3, as shown by the reverse replay in Figs. 5e and 5f, which was in turn elicited by input from PFC. (In cases where converging input from ECIII does add to the depolarization, the combined response function may determine specific neurons that reach the spiking threshold earlier.)

Figure 5.

Temporal context dependent activity in simulated regions CA3 and CA1 that replicates a reversal of spike sequences in CA1 reported by Foster and Wilson (2006). The place cell activity in each line corresponds to the place fields depicted in the adjacent vertical diagrams of the linear track (“s” and “e” mark start and end place fields). (a) Membrane potential of CA3 pyramidal neurons. Select membrane potentials during non-theta mode are shown in a “blow-up” within pane (a), showing the absence of theta modulation during this period. Membrane potential is modulated only by the network-wide inhibition in response to spiking pyramidal neurons. The spikes during non-theta periods in start and end reward locations are shown in the expanded plots (b) and (c). (d) Membrane potential of CA1 pyramidal neurons, again with high resolution spike plots (e) and (f) of sharp-wave activity.

Foster and Wilson reported that even after the first traversal of a novel track, reversed sequence spikes were recorded at head-direction specific neurons in region CA1. At that time, STDP in model ECIII has not yet established associations from the reward location back to the track. Fig. 4c shows that reward location spikes in ECIII during the non-theta period after t = 5000 ms do not yet elicit any retrieval that could contribute to CA1 spiking. Consequently, we suppose that the strength of Schaffer collateral input during non-theta periods (Fig. 5a-c) must suffice to elicit spikes in CA1 (Fig. 5d-f) without coincident input from ECIII (Csicsvari, Hirase, A., Mamiya, & Buzsáki, 1999; Hirase, Leinekugel, Csicsvari, A., & Buzsáki, 2001).

The Foster and Wilson spatial task did not require retrieval in order to make goal-directed decisions. No recall is needed (or shown in Fig. 5) during retrieval intervals of rhythmic theta activity. We suppose that cholinergic modulation during theta rhythm causes the requirement of converging CA3 and ECIII input in order to elicit spikes in CA1 (Wyble, Linster, & Hasselmo, 2000). In our replication of results in a T-maze task by Johnson and Redish (2006), we demonstrate the different results produced by our model when the retrieval of temporal context dependent episodic memory must take place during theta rhythmic activity.

B. Neural ensemble activity in CA3 provides retrospective and prospective retrieval at decision points for goal-directed behavior

Previously, we showed that correct decision making in a delayed spatial alternation task may depend on retrieved retrospective temporal context dependent episodic memory (Koene & Hasselmo, 2006, 2007c). Johnson and Redish (2006) obtained electrophysiological data in rats performing two modified T-maze tasks. In one task, following a fixed path through multiple T junctions and making the correct left or right return arm decision at a final T was rewarded. Each day, the rewarded direction was switched. In the other task, specific tone cues were associated with reward in the left or right return arm of a single T maze. As in the linear track, the cued task does not depend on episodic retrieval. Therefore, we simulate performance of a spatial alternation task in a T-maze with left and right return arms, which may depend on temporal context dependent episodic retrieval, and emphasize retrieval properties and predictions for the Johnson and Redish alternating multiple-T maze task. The rewarded return arm alternates on each run of the maze. The output of our present model, in which region CA3 receives perforant path input associated both with forward and reversed sequences of place representations, combines prospective planning and retrospective retrieval, both of which were also present in the data by Johnson and Redish (2006). Spikes in CA1, elicited by the convergence of this retrieval activity in CA3 and retrieval activity in ECIII, must indicate the previous direction taken at the choice point. This allows PFC to direct the virtual rat in the opposite direction.

During the retrieval interval of each theta cycle, reactivation of the current location context representation in DG elicits high frequency retrieval in CA3, as shown in Fig. 6a. During theta rhythm, competitive inhibition, theta modulated membrane potentials and theta modulated strength of synaptic transmission (Hasselmo, Bodelon, & Wyble, 2002) restrict the spread of activity. In sharp-waves during non-theta periods, retrieval is less constrained (Fig. 6b). Recurrent transmission in CA3 is then strong, while plasticity is suppressed in model recurrent synapses. With little difference between activity with or without theta modulation, retrieval in model ECIII (Fig. 6c) propagates ahead through known adjacent place representations. Simulated spikes in CA1 during sharp-waves (Fig. 6d) and during retrieval intervals in theta cycles (Fig. 6e) reflect the influence of converging retrieved temporal context dependent episodes (CA3) and retrieved feature associations (ECIII), while the virtual rat is in the stem of the maze. Due to the alternating encoding and retrieval intervals generated in each theta cycle of our hippocampal models (Koene et al., 2003), even the shorter spread of activity during theta rhythm may provide enough information about the previous direction taken at a choice point to correctly perform a delayed spatial alternation task. Both retrospective and prospective sequences are demonstrated.

Figure 6.

Place fields (black squares in diagrams of the T-maze with return arms), corresponding place cell spikes (upper right and excerpted retrieval in middle panels of each subfigure) and pyramidal neuron membrane potential (lower right panels). A black triangle next to maze diagrams indicates the first place activity of sequence retrieval. (a) CA3 activity during 3.5 cycles of theta rhythm and an excerpt of one cycle of reversed retrieval. Retrieved place activity then makes one forward step, followed by retrieval of a reversed memory of the path traversed. (b) CA3 activity during an awake non-theta period, with extended reversed sequence retrieval consisting of 14 steps. (c) In ECIII, current location activity is followed by forward spreading activity through associated places. (d) Two ripples of activity in CA1, while the virtual rat is located in the middle of the stem of the maze, during a non-theta period. Both times, the first spikes represent places ahead of the rat (prospective), and subsequent spikes represent previous path locations (retrospective). (e) Two theta cycles of activity in CA1, while the virtual rat is located in the middle of the stem of the maze. During each retrieval interval of the theta cycles, path locations are retrieved in reverse from the current location.

We plot the ripple activity at the choice point of the maze, after encoding one full circuit with perforant path input to CA3 that represents both forward and reversed place sequences. Some chunking occurs in CA3, whereby some of the directionality of a representation is lost within a window of spikes equal in size to the buffer capacity. CA3 exhibits predominantly retrospective activity shown in Fig. 7a. By contrast, the converging input to CA1 through Schaffer collateral and perforant path inputs results in predominantly prospective spike sequences (Fig. 7b).

Figure 7.

(a) Spiking activity of neurons in model hippocampal region CA3 during retrieval, as elicited in neurons 13 and 29 by temporal context specific input from dentate gyrus. Spiking activity spreads through recurrent excitatory fibers in region CA3 to strongly associated neurons, according to previously encoded episodic memories. (b) Prospective and retrospective activity elicited in CA1.

IV. Discussion

Our model demonstrates encoding and both prospective and retrospective retrieval of temporal context dependent episodic memory. In the linear track, retrospective retrieval dominates, although CA3 exhibited some prospective retrieval even in that case (Fig. 5). We replicated reversed spike sequences in model region CA1 during sharp-wave activity, as observed by Foster and Wilson (2006). In that case, we propose that observed spiking of head-direction sensitive neurons in CA1 closely resembles the spread of sharp-wave activity in CA3. We also replicated both prospective and retrospective activity in model region CA3 (and CA1) during theta rhythm, providing the temporal context dependent behavior required in a spatial alternation task (Johnson & Redish, 2006). As in their data, prospective and retrospective activity appear rapidly at the choice point on multiple runs, regardless of the novelty of the task.

We encode separate memory traces for episodes in the experienced order and in the reversed order of place representations. For this, we use persistent spiking buffers, whose function is reversed by a simple difference of input timing, which underscores the plausibility of episodic memory formation with both forward and reversed spike sequences. Previously, we used a reversing buffer to demonstrate a possible mechanism for reverse replay (Koene & Hasselmo, 2007c), so that the present model hypothesis is a reasonable generalization, given that we have no reason to assume that only buffered spike sequences in one order should reach CA3.

Modulation within each simulated theta cycle of our model determines intervals optimized to encode or retrieve (Hasselmo et al., 2002). Ongoing episodic encoding of the forward and reversed spike sequences may be necessary even when no immediate reward is anticipated (Colgin & Moser, 2006), in case a situation arises in which temporal context dependent episodic memory is needed (Hasselmo & Eichenbaum, 2005). Thus, prospective and retrospective retrieval constrained by a theta cycle may be able to guide behavior at choice-point decisions in the T-maze task. If current location is the cue in successive retrieval intervals, then the shifting elicited spike sequences (Jensen & Lisman, 1996; Skaggs, McNaughton, Wilson, & Barnes, 1996) resemble phase precession plots of electrophysiology in place cells (O’Keefe & Recce, 1993). The experimental data of Johnson and Redish (2006) and recent recordings by Csicsvari, O’Neill, Allen, and Senior (2007) during open-field exploration lend support to the possibility that reversed replay also occurs in ripple activity within theta cycles during mobility.

A. Experiments and models in the literature

In experiments by Foster and Wilson (2006), the average speed of rats was about 13 cm/s, about three times slower than in our simulations. Realistic encoding may therefore benefit from a greater number of theta cycles during the traversal of each place field, or can involve smaller place fields. We minimized the run time, and limited the duration of intervals without theta rhythm at reward sites to less than a second in order to reduce the computational load. In resulting plots, the different interval sizes affect how frequent sequence activity appears to be during theta and non-theta periods. Simple adjustments of the simulations can produce the same proportions as shown in the plots by Foster and Wilson (2006, Fig.2b).

Colgin and Moser (2006) suggest that synaptic facilitation causes post-experience reversal during sharp-waves, used to encode a reversed trace of episodic memory that is initially stored in forward order. We propose instead, that forward and reversed representations in episodic memory may be encoded immediately. The two proposals may be tested, if a behavioral task can be designed in which recent episodic experience is needed, but no opportunity to pause and generate sharp-waves in the absence of theta rhythm is given.

Molter, Sato, and Yamaguchi (2006, 2007) proposed computational models to address hippocampal spiking in forward and reversed sequence order during sharp-waves. They proposed that reversal immediately following experience depends on remapping of place fields toward reward sites and on irregular input to CA3. We address prospective and retrospective activity during theta rhythm and during non-theta periods, as well as how the activity may be elicited immediately or by later retrieval in CA3, to be used in spatial navigation tasks. Our mechanism of reversal depends only on the timing of input to ECII. Reactivated representations in dentate gyrus must be able to elicit associative retrieval in region CA3 that begins at spikes representing the end of an experienced episode. The broad fan-out from the granule layer of DG may, via mossy fiber synapses, elicit a temporal context specific pattern of spikes at CA3 pyramidal neurons, some of which are depolarized by perforant path sequence input from ECII. Mossy fiber synapses may be strengthened with or without consequent spikes at those CA3 neurons (Urban & Barrionuevo, 1996).

Current location is the start of temporal context dependent retrieval in the simulated spatial alternation task, and a propensity of place cell spiking that includes a rat’s current location has been reported by O’Neill, Senior, and Csicsvari (2006). Still, associated knowledge of reward locations may also cue retrieval, as the data of Johnson and Redish (2006) suggests. Our model associates forward and reversed episodic cues in ECII with episodic memory in CA3, as well as with a temporal context specific unique pattern of activity in DG. A second route to the episodic memory in CA3 is completed by associative strengthening of mossy fiber synapses between neurons in the pattern of activity in DG and the neurons in CA3. In addition to task-specific cues mediated by PFC, spontaneous activity through this mossy fiber pathway, as well as autonomous spiking in CA3, may elicit the reactivation of partial sequences that are propagated to CA1 during periods of slow-wave sleep (Wilson & McNaughton, 1994; Lee & Wilson, 2002). The Johnson and Redish data also shows alternating retrieval of different competing sequences, which may be encouraged by the after-hyperpolarization of recently spiking neurons.

Recent data by Diba and Buzsáki (2007) supports the plausibility of both forward and reverse encoding throughout a spatial task, which can explain the appearance of both prospective and retrospective ripple activity at the end points in their linear track experiment. Our present model addresses underlying mechanisms by which forward and reversed associations may be learned. We do not as yet propose task-specific selection mechanisms that would explain how a majority of the ripple activity becomes prospective prior to the onset of a run, and retrospective when a run is completed (Diba & Buzsáki, 2007).

B. Further work

We modeled hippocampal region CA3 as a single recurrent network, in which synaptic modification encodes associations between the activity of any two neurons. It is possible that distinct episodic memory traces are retained after encoding in response to forward and reversed perforant path input. The separation may be temporal, by encoding during different phases of the theta cycle, or the separation may be anatomical, by delivering forward and reversed perforant path input to different subregions of CA3. In that case, it may be possible to retrieve the episodic traces independently. It is also possible that one hippocampal region (e.g. CA3) deals exclusively with retrospective associations, and another (e.g. heteroassociative storage between DG granule and mossy layers) is used during prospective planning. Our model allows that STDP in the recurrent network of DG mossy and granule layers may establish a temporal association between successive context specific representations. In that case, retrieval of episodic memory associated with one temporal context may be followed by retrieval of the next temporal context dependent episodic memory (indicated by the link between DGt1 and DGt2 in Fig. 1a). We seek further evidence with which to discern the most realistic possibility, as well as to determine whether separate networks encode autoassociative relationships between spikes in a pattern and heteroassociative relationships between spikes in a sequence (Lisman, 1999; Lisman et al., 2005).

Initial tests with simulations in which reverse and forward short-term buffers share the same recurrent interneuron network indicate that it is possible that both buffer functions reside in the same network of pyramidal neurons, distinguished only by the phase of afferent input that may be a plausible random variation. Alternatively, each buffer function may reside in a specialized subregion. Within such subregions, the pyramidal neurons may receive inhibitory input from distinct interneuron populations. If there is such an anatomical distinction, then that could prevent interference between concurrently maintained forward and reverse sequence representations, lending greater independence to the operation of different buffers. Determining the localization of and modulatory input to reverse and forward buffers in ECII will require further comparative studies of the cell morphology and electrophysiology in subregions and bands of ECII (e.g. Tahvildari & Alonso, 2005).

Retrospective episodic retrieval is useful in tasks where information about recent events or prior decisions is needed, such as in alternation tasks (Hasselmo & Eichenbaum, 2005). Prospective episodic retrieval can be used when planning immediate task-related actions (e.g. prospective ripple activity seen in Johnson & Redish, 2007; Diba & Buzsáki, 2007) or where biographical episodic memory is required. Underlying mechanisms cannot predict which will be needed. For this reason we propose that buffering and encoding of forward and reversed sequences takes place concurrently. This suggestion that the system prepares for potential needs is comparable to our reasoning in earlier models, in which we proposed that encoding and retrieval occur throughout a task and alternate in each cycle of theta rhythm (Koene et al., 2003). According to our model of temporal context dependent episodic memory, we interpret retrospective and (dominant) prospective ripple activity in anticipation of motion (Johnson & Redish, 2007; Diba & Buzsáki, 2007) as follows:

1. Forward and reversed sequence encoding take place during motion.

2. After motion, retrieval of encoded sequences can be elicited by spontaneous activity in CA3, by cue sequences in ECII or by activity in DG.

3. In preparation of motion, input from PFC to DG can elicit retrospective retrieval in CA3 from a known goal representation, which converges in CA1³ with spreading activity in ECIII (Koene et al., 2003) to produce mainly prospective goal-directed activity.

Appendix

This appendix summarizes some features of the model. Due to space limitations, some details from previous papers are not reviewed.

The conductance response of each membrane currents is described by a double-exponential function,

$$g_i(t)=G_i{a}_{\mathit{norm}}(\exp (-t/T_{\mathit{fall},i})-\exp (-t/T_{\mathit{rise},i})),$$ (1)

where G_i is the characteristic amplitude of the conductance response for a specific membrane current, and T_fall,i and T_rise,i are its fall and rise time constants. A normalizing factor a_norm is used to insure that the maximum value of g_i(t) is G_i,

$$a_{\mathit{norm}}=1/(\exp (-t_{\mathit{max}}/T_{\mathit{fall},i})-\exp (-t_{\mathit{max}}/T_{\mathit{rise},i})),$$ (2)

with the time offset of the maximum response value,

$$t_{\mathit{max}}=\mathit{ln}\{\frac{T_{\mathit{fall},i}/T_{\mathit{rise},i}}{(1/T_{\mathit{rise},i})-(1/T_{\mathit{fall},i})}\}.$$ (3)

Explicit membrane capacitance determines the time-constant of exponential decay to resting potential. Theta modulation is achieved by rhythmic input through inhibitory synapses at a frequency of 8 Hz, which is assumed to originate in the medial septum. Computed individual time-specific conductance values g_i (t) determine contributions to the change Δv of the membrane potential V during a small time interval Δt,

$$\mathrm{\Delta }v=\frac{{\sum }_i(g_i\Delta t(E_{\mathit{rev},i}-V))}{C+{\sum }_i(g_i\Delta t)},$$ (4)

where C = 1 mF is the membrane capacitance and E_rev,i the reversal potential of a contributing membrane current. The firing threshold is −50 mV. The pyramidal neuron resting potential is −60 mV, the interneuron resting potential −70 mV. Action potentials have a duration of 1 ms, are followed by a 2 ms refractory period and by after-hyperpolarization. The parameter values used in simulations are shown in Table 1. The recurrent connectivity in the buffer network is such that one interneuron model representing interneuron network activity receives input from all pyramidal buffer neurons and sends spike output to all pyramidal buffer neurons.

Table 1.

Model current parameter values. Leak currents are modeled as exponential decay functions to resting potential, and therefore have only one time constant. Leak conductance is related to the leak time constant by G_leak = C/T_leak.

membrane current	Trise (ms)	Tfall (ms)	G (nS)	Erev (mV)
pyramidal buffer neurons:
after-hyperpolarization	10−4	30	23	−90
after-depolarization	125	125	30	−45
asymmetric theta modulation	0.1	20	10	−90
input from “gamma” interneuron	0.1	2.5	100	−70
leak		9	111	−60
“gamma” interneuron:
after-hyperpolarization	10−4	4	100	−90
input from buffer pyramidal neurons	1	2	30	0
leak		10	100	−70

Noise was added through simulated current clamps of individual neurons driven by a first order autoregressive process (a model for the response to noise that is similar to a random walk) with Poisson distribution, a mean value of 0, amplitude 1 pA and regression parameter 0.5. Where effects of strong noise are demonstrated, the amplitude is ±10 pA, and for very strong noise the amplitude is in the range [60, 70] pA.

We use symbolic units to describe the replacement mechanism first presented in Koene and Hasselmo (2007b). A P_ADP unit represents a superposition of all the pyramidal buffer neurons that spike persistently and therefore exhibits spikes whenever any of those buffer neurons spike. Units Pf and Pi represent ensembles of pyramidal neurons without intrinsic spiking (implemented as two model pyramidal neurons), where Pi receives external input and Pf receives feedback from P_ADP. An Ir unit represents an ensemble of interneurons (implemented as one model interneuron) that is driven solely by input from Pf and Pi units.

Pf and Pi membrane potentials are theta modulated at the same phase as P_ADP membrane potentials. At Pf, the efficacy of input from P_ADP is modulated in each theta cycle, to transmit spikes at strength sufficient to elicit Pf spiking only at the phase of fourth pattern reactivation (the buffer capacity). Converging Pf and Pi input depolarizes Ir. If thus depolarized, theta modulation elicits a spike in Ir just prior to the reactivation phase of the first buffered spike pattern. At that phase, reactivation depends critically on depolarization by ADP, and is suppressed by the inhibitory input from Ir. The inhibition lasts until the waning ADP can no longer achieve reactivation of the oldest spike pattern. Subsequent buffered spike patterns shift to earlier theta phases and are followed by the reactivation of the spike pattern that was elicited by afferent input.

Reversed order buffering is achieved if the phase of afferent input is delayed by 25 ms, so that afferent input is not reactivated in the same theta cycle (rhythmic hyperpolarization sets in), but is instead reactivated first in the next theta cycle. The reactivation of older spike patterns is shifted to later phases of the theta cycle. In a full buffer, the shifted reactivation phase of the oldest spike pattern falls into the hyperpolarized theta interval. That hyperpolarization suppressed reactivation, without the need for inhibitory input from an Ir unit.