A Unified Dynamic Model for Learning, Replay, and Sharp-Wave/Ripples

Abstract

Hippocampal activity is fundamental for episodic memory formation and consolidation. During phases of rest and sleep, it exhibits sharp-wave/ripple (SPW/R) complexes, which are short episodes of increased activity with superimposed high-frequency oscillations. Simultaneously, spike sequences reflecting previous behavior, such as traversed trajectories in space, are replayed. Whereas these phenomena are thought to be crucial for the formation and consolidation of episodic memory, their neurophysiological mechanisms are not well understood. Here we present a unified model showing how experience may be stored and thereafter replayed in association with SPW/Rs. We propose that replay and SPW/Rs are tightly interconnected as they mutually generate and support each other. The underlying mechanism is based on the nonlinear dendritic computation attributable to dendritic sodium spikes that have been prominently found in the hippocampal regions CA1 and CA3, where SPW/Rs and replay are also generated. Besides assigning SPW/Rs a crucial role for replay and thus memory processing, the proposed mechanism also explains their characteristic features, such as the oscillation frequency and the overall wave form. The results shed a new light on the dynamical aspects of hippocampal circuit learning.

SIGNIFICANCE STATEMENT During phases of rest and sleep, the hippocampus, the “memory center” of the brain, generates intermittent patterns of strongly increased overall activity with high-frequency oscillations, the so-called sharp-wave/ripples. We investigate their role in learning and memory processing. They occur together with replay of activity sequences reflecting previous behavior. Developing a unifying computational model, we propose that both phenomena are tightly linked, by mutually generating and supporting each other. The underlying mechanism depends on nonlinear amplification of synchronous inputs that has been prominently found in the hippocampus. Besides assigning sharp-wave/ripples a crucial role for replay generation and thus memory processing, the proposed mechanism also explains their characteristic features, such as the oscillation frequency and the overall wave form.

Introduction

The interest in spatial processing and memory in the hippocampus has been driven, to a large extent, by the observation of hippocampal “place cells”: during exploratory behavior, the spiking activity of place cells codes for the position of an animal (O'Keefe and Dostrovsky, 1971; O'Keefe, 1976; Ekstrom et al., 2003; Kjelstrup et al., 2008; Harvey et al., 2009). Exploration is accompanied by theta oscillations, rhythmic modulations in neural population activity and in the local field potential (LFP), with frequencies of 4–10 Hz. When an animal visits several locations, sequences of spiking activity emerge within the theta cycles because of preferred place cell firing relative to the phase of the theta oscillation (O'Keefe and Recce, 1993; Skaggs et al., 1996; Maurer and McNaughton, 2007; Gupta et al., 2012). These sequences reflect the recent past and future place field traversing, compressed in time. The sequences of activity are replayed in a further compressed manner, while the animal rests and sleeps (Wilson and McNaughton, 1994; Nadasdy et al., 1999; Lee and Wilson, 2002; Pastalkova et al., 2008; Davidson et al., 2009). This replay happens during so-called sharp-wave/ripple (SPW/R) events, short phases of strongly enhanced activity (“sharp waves”; durations of ∼50–100 ms) with highly synchronous spiking at ∼120–200 Hz (“ripples”; Buzsáki et al., 1992; Ylinen et al., 1995; Maier et al., 2003).

SPW/Rs and the associated replay are assumed to be crucial for memory consolidation (Buzsáki, 1989). Indeed, the replay changes because of experience (Pavlides and Winson, 1989; Wilson and McNaughton, 1994; Sutherland and McNaughton, 2000); in particular, it reflects activity from exploration phases (Wilson and McNaughton, 1994; Nadasdy et al., 1999; Lee and Wilson, 2002; Pastalkova et al., 2008; Davidson et al., 2009), the inter-ripple interval of 5 ms is in the optimal range for the induction of synaptic modifications (Markram et al., 1997; Bi and Poo, 1998), SPW/Rs occur coordinated with activation across neocortical brain regions (Ji and Wilson, 2007; O'Neill et al., 2010; Logothetis et al., 2012), and it is even possible to change spatial preferences by stimulating reward centers during hippocampal replay (de Lavilléon et al., 2015). Finally, suppression of SPW/Rs impairs consolidation of memories (Girardeau et al., 2009; Ego-Stengel and Wilson, 2010; Jadhav et al., 2012) whereas imposed replay during SPWs strengthens them (Barnes and Wilson, 2014). The “two-stage model” for memory (Marr, 1971; Buzsáki, 1989; Willshaw and Buckingham, 1990) assumes that SPW/Rs and replay lead to the consolidation of memory content by transferring it from preliminary storage in the hippocampus to long-term storage in the neocortex (Buzsáki, 1989). It has also been suggested that SPW/Rs and replay are involved in incorporating new information into existing contexts, associating and processing previous experiences, planning future behavior, and deleting memories from hippocampal storage (O'Keefe and Nadel, 1978; Buzsáki, 2006; Mehta, 2007; Tse et al., 2007; Gupta et al., 2010; Pfeiffer and Foster, 2013). To assess whether and how the different suggested tasks may be fulfilled, it is essential to understand the mechanisms underlying SPW/R patterns and the replay. In the present study, we derive a unifying model showing how experience may be stored and thereafter replayed in conjunction with SPW/Rs in spiking neural networks.

Materials and Methods

Neuron model

We consider networks of leaky integrate-and-fire neurons. The state of neuron i is described by its membrane potential V_i, and the temporal dynamics of the membrane potential obey the following:

where C_i is the membrane capacity, g_i^L is the resting conductance, and V_i^eq is the equilibrium potential. I_i⁰ is a constant input current to neuron i, and I_i^ex(t) and I_iⁱⁿ(t) are currents arising from excitatory and inhibitory inputs, respectively. Whenever the membrane potential V_i(t) exceeds the spiking threshold I_i⁰ at some t = t*, a spike is sent to the postsynaptic neurons j, where it arrives after a delay time τ_ji. The sending neuron is reset to V_i(t*) = V_i^reset, and the neuron is refractory for a time period t_i^ref, i.e., V_i(t) ≡ V_i^reset for t ∈ [t*, t* + t_i^ref].

Synapses and dendrites models

Synapses.

The effects of the synaptic inputs on the postsynaptic neurons are modeled by transient conductance changes. The time course of the synaptic conductance is given by the difference of two exponentials with time constants τ^k,1 and τ^k,2, as follows:

where k ∈ {ex, in} describes the effect of an excitatory and inhibitory input, respectively, that is received at time t₀ = 0. The normalization constant A^k is chosen such that the peak conductance max_{t ≥ t0} {f^k(t)} = 1:

Throughout this study, we denote the strength of a synaptic connection by the value of the peak conductance, i.e., a single input from a connection of strength ε causes a conductance change ε · f^k(t). Denoting the reversal potentials of excitatory and inhibitory currents by E^ex and Eⁱⁿ, the input currents to neuron i arising from synaptic inputs from other neurons of the network are given by the following:

g_i^ex(t) and g_iⁱⁿ(t) are linear superpositions of single responses (compare Eq. 2) as follows:

where ε_ij^ex (and ε_ijⁱⁿ) denotes the excitatory (and inhibitory) coupling strength from neuron j to neuron i, t_j,n^f is the nth spiking time of neuron j, and τ_ij is the conduction delay from neuron j to neuron i.

Nonlinear dendrites.

The above model implicitly assumes linear input summation in the dendrites. We now incorporate nonlinear amplification of synchronous inputs mediated by fast dendritic spikes, which have been found in single-neuron experiments (Ariav et al., 2003; Gasparini et al., 2004; Polsky et al., 2004; Gasparini and Magee, 2006) and have been introduced in recent models of neuronal networks (Memmesheimer, 2010; Jahnke et al., 2012, 2013; Breuer et al., 2014). Whenever the excitatory input to a (nonlinear) dendrite summed over a short time interval (ΔT^s) of less than some milliseconds crosses the dendritic threshold Θ_b, a dendritic spike is initiated and causes a depolarization in the soma of the postsynaptic neuron that exceeds the depolarization expected from linear summation of single inputs. We model the effect of a dendritic spike by a stereotypical current pulse that is injected to the soma after a time interval τ^DS after the dendritic threshold is crossed. The temporal offset τ^DS models the latency between the onset of the (linear) postsynaptic response and the response to the dendritic spike as observed in single-neuron experiments (Ariav et al., 2003). The current pulse is described by the sum of three exponential functions:

where positive prefactors A,B,C and decay time constants τ^ds,1, τ^ds,2, and τ^ds,3 are chosen such that the somatic depolarization fits experimental data (compare Fig. 1). To obtain the experimentally observed saturation of the somatic depolarization for inputs exceeding the dendritic threshold, the nonlinear dendrite becomes refractory for a time period t^ref,ds after generation of such a current pulse and does not transmit spikes within the refractory time period.

Figure 1.

Nonlinear dendritic interactions. A, Dendritic current I_ds as defined in Equation 8 for standard parameters. B, Somatic depolarization V of a postsynaptic neuron in response to presynaptic stimulation at time t = 0 (blue solid line) that is sufficiently strong such that a dendritic spike current is elicited (i.e., the total stimulation is larger than the dendritic threshold Θ_b). The black dashed line is the somatic response in the absence of dendritic nonlinearities (linear dendritic input summation).

Synaptic plasticity.

Recurrent excitatory connections are considered to be plastic, and their weights are adjusted depending on activity. We use a pair-based update rule for synaptic connections, i.e., the change of the weight of a synaptic connection depends on the temporal difference between pairs of presynaptic and postsynaptic spikes [Bi and Poo, 1998, 2001; Tetzlaff et al., 2013; spike-timing-dependent plasticity (STDP)]. Let Δ be the temporal difference between the spike of a postsynaptic and a presynaptic neuron and denote the synaptic weight by ε, then this spike pair induces a weight change as follows:

Here τ^± are the time constants of the update window and F^±(ε) describes the weight dependence of the update rule. As proposed by Morrison et al. (2007), we use a power-law update rule:

where λ specifies the learning rate, ε₀ is a reference weight, μ ∈ [0, 1] is the exponent of the synaptic potentiation rule, and α is the asymmetry parameter, which scales the synaptic depressing with respect to the synaptic potentiation. We choose the power-law update rule (compare Eq. 10) because (1) it can be fitted to experimental data (Bi and Poo, 1998) better than, for example, purely additive or multiplicative update rules (Morrison et al., 2007) and (2) it has been shown that this update rule generates a unimodal weight distribution in the presence of uncorrelated Poissonian inputs (Morrison et al., 2007).

Network setup

We consider networks that are composed of N^ex excitatory and Nⁱⁿ inhibitory neurons. The neurons are randomly connected. Single connections are present with probabilities p^X,Y (where X, Y ∈ {ex, in }) that depend on the type of postsynaptic and presynaptic neurons, e.g., p^in,ex specifies the probability of a directed connection from an excitatory to an inhibitory neuron. The connection strengths are drawn from a Gaussian distribution with mean μ^X,Y and SD σ^X,Y truncated at zero to obtain only positive values. The recurrent excitatory–excitatory connections are considered plastic and adjust their weights according to the update rule (Eqs. 9–11), and all other weights are static.

Like the synaptic weights, the conduction delays are chosen heterogeneously. The total conduction delay between the presynaptic spiking and the onset of the postsynaptic somatic response can be decomposed into two components: (1) the axonal delay τ^ax and (2) the time τ^X between the onset of synaptic transmission and the onset of the somatic response. We assume that τ^X is constant, but it depends on the type of postsynaptic neuron, whereas τ^ax depends on the distance between presynaptic and postsynaptic neurons and the conduction velocity. All neurons are randomly placed according to a uniform distribution on a square patch with edge length S, and τ^ax is computed by dividing the Euclidean distance between the neurons by a conduction velocity v^ax. In such a setup, the mean distance between two neurons is given by the following (Mathai et al., 1999):

and thus the mean axonal delay is as follows:

After drawing the delays as described above, we randomly shuffle them between neuron pairs to reduce finite size effects (compare Fig. 2). Additionally, each neuron receives independent external Poissonian spike trains with rate ν^X,Y and constant input strength c^X,Y. Like the recurrent inputs, the external inputs are conductance-based currents with the same temporal profile as the recurrent ones (compare Eqs. 2–7).

Figure 2.

Delay distributions for standard parameters. The green line shows the distribution of the (distance-dependent) axonal delay. The distribution of total conduction delays from excitatory-to-inhibitory and from inhibitory-to-inhibitory neurons is shown in blue (τ^in,ex = τ^in,in = τ^ax + τⁱⁿ). Likewise the distribution of the total conduction delays to excitatory neurons is shown in purple (τ^ex,ex = τ^ex,in = τ^ax + τ^ex).

Place cell tuning curves

In a given environment, a large fraction of pyramidal cells (place cells) in hippocampal regions CA3 and CA1 display well defined sensitivity to spatial positions (Mizuseki et al., 2012), and therewith form a cognitive map of the environment. They fire at increased rates when the current position of an animal is within their receptive (place) fields (O'Keefe and Dostrovsky, 1971; O'Keefe, 1976; Ekstrom et al., 2003; Kjelstrup et al., 2008; Harvey et al., 2009). The preferred timing of single spikes depends on the position relative to the place-field center and on the phase of the background theta oscillation (O'Keefe and Recce, 1993; Skaggs et al., 1996; Maurer and McNaughton, 2007; Gupta et al., 2012): when the animal enters a place field, single cells spike late in the theta cycle. While traversing the place field, spiking occurs at earlier and earlier times (“phase precession”), and when the animal leaves the place field, the place cell preferably spikes at the beginning of the cycle.

The neuronal mechanisms underlying phase precession and location tuning are debated. It has been proposed that phase precession may arise from interference between two oscillatory inputs with slightly different frequencies (O'Keefe and Recce, 1993; Lengyel et al., 2003). Alternatively, phase precession may arise from interaction of theta-modulated inhibition with external excitation of increasing strength (Magee, 2001; Harris et al., 2002; Mehta et al., 2002; Thurley et al., 2008). As a third option, it has been proposed that phase precession is caused by asymmetric recurrent connections between place cells that have been learned in earlier training sessions on the same track and predict the animal's path (Jensen and Lisman, 1996; Tsodyks et al., 1996). Similarly, phase precession can arise for symmetric connections because of short-term depression (Romani and Tsodyks, 2015).

In this study, we implement a phenomenological approach that is compatible with all three models (compare Fig. 3): the place cells receive rectangular current pulses whose amplitude and timing are determined by the current position x on a linear track such that spiking dynamics with the experimentally observed characteristics (i.e., place sensitivity and phase precession) are generated. By this, we generate the experimentally observed spiking, which determines in STDP-based models the learning of spike sequences during exploration phases independent of a specific underlying mechanism.

Figure 3.

Place tuning and phase precession of single cells. A, When the actual position x is within the place field of a neuron, rectangular current pulses are injected to the neuron. Their amplitudes I^pf (solid, left vertical axis; Eq. 16) and temporal offsets (DT) with respect to the background theta rhythm (dashed line, right vertical axis; Eq. 17) are functions of the relative position (rel. pos.) x̃ in the place field (compare Eq. 15). B, Examples of the current pulses for a place field, P = [10, 40] cm, and the positions x = {15, 25, 35} cm. The positions in the place field are shown in the top subpanel; the resulting input currents are shown in the bottom subpanel. C, The main panel displays the resulting spatiotemporal receptive field. The inset shows the average firing rate (solid; left vertical axis) and average phase shift against the theta rhythm (dashed; right vertical axis) as a function of the current position. The neuron increases the firing rate for positions within the place field and shows phase precession. D, Bottom, Two example membrane voltage traces during the traversal of a place field of a neuron with the characteristics summarized in C. To indicate the spikes, voltage excursions to V^max = 0 mV are added to the membrane voltage whenever a spike is generated. Top, The (virtual) theta rhythm; the dots indicate the spiking times of the considered neurons. The place field has a width of Δw = 30 cm, and it is traversed in jumps of Δx = 2 cm per theta cycle (i.e., v = 16 cm/s). For further details, see Materials and Methods.

We assign a place field centered at x_i^ctr with width Δw_i,

randomly to each place encoding neuron i. If the current position is within the place field of neuron i, i.e., x ∈ P_i, additional rectangular current pulses of duration Δτ are injected. The amplitude as well as the timing of these current pulses depend on the following:

which is the relative position within the place field of that cell.

The current pulses are time locked to a (virtual) oscillating background signal, the theta oscillation. During each oscillation period T, all neurons with a place field that encloses the current position x (i.e., 0 ≤ x̃_i ≤ 1) receive an input of strength:

which is unimodal in space and shifted against the background oscillation by the following:

The pulse amplitude is thereby maximal at the place-field center [I^pf (x̃_i = 0.5) = I^max] and decays quadratically to zero toward the beginning [I^pf (x̃_i = 0) = 0] and the end [I^pf (x̃_i = 1) = 0] of the place field (compare Fig. 3A, solid line). Furthermore, the current pulse is shifted by almost a full oscillation period T against the theta oscillation when the position x is in the beginning of the place field [DT(x̃_i = 0) = 0.9T], and this time shift decreases quadratically to zero with increasing position [DT(x̃_i = 1) = 0] (compare Fig. 3A, dashed line). Thus, assuming that the actual oscillation period starts at t = 0 and the actual position within the place field of neuron i is x̃_i, the time course of the injected current to neuron i is given by the following (compare examples traces in Fig. 3B):

where χ_[A](t) is the indicator function; χ_[A](t) = 1 if t ∈ A, and χ_[A](t) = 0 otherwise.

To quantify the location tuning properties, we measure the spatiotemporal receptive field of a neuron with some place field P: we record the spiking activity for different positions x = {x₁, x₂, …, x_n}. For each recorded spike, we calculate the phase shift with respect to the theta oscillation as follows:

where t^sp denotes the spiking time and ⌊x⌋ is the floor function that maps a real number x to the largest integer not greater than x. The spatiotemporal receptive field is then a two-dimensional histogram for the location x (binning δx) and the phase shift φ (binning δφ) normalized by the total time spent in this bin. As illustrated in Figure 3C, the spiking reflects the characteristic of the rectangular input current: when traversing the place field, the average firing rate increases until it reaches the maximum at the center of the place field and decreases afterward. The single spikes are correlated to the theta rhythm: when entering the place field, the neuron spikes late in the theta rhythm, and the preferred spiking moves to earlier and earlier times with increasing position.

Throughout the study, we used neurons with spiking characteristics as illustrated in Figure 3, C and D. The results, in particular the formation of a feedforward structure during the runs along the linear track, are generally robust against changes in the details of these characteristics. Quantitatively, they depend on the functional form of the phase precession (compare Eq. 17), the width of the distribution of phase shifts for a given position (specified by Δτ), and the functional form of the spiking rate (compare Eq. 16). For example, if the distribution of phase shifts is broader, neurons with overlapping place fields still tend to spike in the order of their place-field centers on the linear track, yet the ordering is not as reliable as for narrower distribution. Accordingly, a feedforward structure will form, but it will take more runs (or a faster learning rate λ) to obtain a similar structure.

Running along a linear track

Consider the movement along a linear track of length L. A subset of N^pf < N^ex excitatory neurons is assumed to have an active place field on the track. We choose the first N^pf neurons with the index i ∈ {1, …, N^pf} as neurons with an active place field. For simplicity, we assume that the place fields P_i have identical widths Δw_i Δw and their centers x_i^ctr are distributed homogeneously along the track of length L (also see previous section). To prevent boundary effects, we extend the track by Δw/2 at the beginning and at the end (i.e., total track length L^tot = L + Δw), such that during the movement all place fields are completely traversed (compare gray shaded area in Fig. 4A) and the place-field center of neuron i ∈ {1, …, N^pf} is given by the following:

Furthermore, we discretize the movement along the linear track. We assume that the position is fixed during each theta cycle with period T and that the position increases in a jump-like manner between consecutive cycles. Starting initially with x₀ = 0, the position during the jth oscillation is given by the following:

and the jump size Δx is drawn uniformly from the following interval:

where v̄ specifies the average running velocity. If a track is traversed multiple times, the position is set to x = 0 at the beginning of each run and the track is traversed as described above.

Figure 4.

Learning feedforward structures. A, Example trajectory of a modeled run along a linear track of a total length of L^tot = 190 cm with average velocity v̄ = 8 cm/s. For simplicity, we assume that the position is fixed for the short period of one theta cycle (T = 125 ms). Between consecutive theta cycles, the position increases randomly in a jump-like manner, here on average by 1 cm (compare zoomed view of the trajectory presented in the inset). Place fields are assigned to N^pf = 1500 of N^ex = 2500 excitatory neurons, and the centers are distributed homogeneously in the interval $\left[\frac{\Delta w}{2},L^{\text{tot}}-\frac{\Delta w}{2}\right]$ (indicated by the gray shaded area). B, C, Spiking activity of the excitatory neuron population for the period of one theta cycle at different positions (indicated by the blue and green markers in A). The colored area indicates the fraction of neurons with place fields containing the current position. D, Distribution of synaptic weights as a function of the distance between the place field centers of the postsynaptic and the presynaptic neuron. The solid line indicates the mean weight, and the colored area is the interval containing 80% of the weights (0.1- 0.9 percentile). Different colors show the distribution after different numbers of runs as indicated by the inset. E, Connectivity matrix for connections between the first 400 neurons after 10 runs (D, red curve).

Toward the end of the study, we consider the movement along different tracks. For each track N^pf, neurons are chosen randomly, and a place-field center is assigned to each of these neurons. The place-field centers are distributed homogeneously along each track as before.

Matching index

To quantify the quality of replay, we compare the ordering of spikes in a replay event with the ordering in the original event. Specifically, we consider the matching index I (Ji and Wilson, 2007): we count the total number of spike pairs where the spikes occur in the same order as in the original sequence (n^true), subtract the number of pairs with reverse ordering (n^false), and normalize by the total number of spike pairs occurring in the replay event, as follows:

Thus, the matching index I is a real number in the interval [−1, 1], which is I = 1 for perfect replay of the original sequence, I = −1 for reverse replay, and |I| ≪ 1 for random replay sequences.

Relative power in the high-frequency band

Replay events are initiated in a targeted manner by synchronous stimulation of a number of cells where the actual spiking times are drawn from a Gaussian distribution with SD σ^stim. To evaluate the impact of synchronicity of the initial stimulation (i.e., of the temporal spread σ^stim) on the characteristics of the replay events, we consider the relative power in the high-frequency band between 150 and 200 Hz (compare Fig. 7B). Therefore, we measure the spiking rate r(t) over a time window of length Δ = 150 ms that contains the induced replay event. We calculate the power spectral density as follows:

where r̂(f) is the Fourier transform of r(t). The relative power in the high-frequency band is then given by the following:

normalized by the total power P^tot : = ${\displaystyle {\int }_0^{\infty }{P}_{\text{d}}\left(f\right)df}$.

Figure 7.

Replay of spike patterns during sharp-wave/ripple-like events induced by synchronous stimulation. A, Average number of consecutive synchronous groups (ripples) versus the number of initially synchronously stimulated neurons g₀ (spike times drawn from a Gaussian distribution with σ^stim = 0.5 ms). Different colors correspond to the different number of runs along the linear track. The inset shows the average fraction p^frac of place cells that spike at least once during the replay events. The data are derived by averaging over n = 10,000 replay events. B, The relative power in the high-frequency band (150–250 Hz) versus the temporal width σ^stim of the initial synchronous pulse. With increasing σ^stim, the relative power decreases. For large σ^stim, a signal can still propagate along the feedforward structure, yet the replay is not clearly structured in consecutive firing synchronous groups (therefore, no ripples occur; compare F). C–F, Examples of network activity (C–E, σ^stim = 0.5 ms; F, σ^stim = 5.0 ms): top, firing rate; bottom, spiking activity of the excitatory (black) and the inhibitory (red) neuron population. The size of the initial stimulation and number of runs along the linear track are indicted by the markers (top and A, B). The gray shaded area in bottom panels indicate the subset of neurons that have an active place field on the track. E1, F1, Close-up view of the spike data shown in E and F. For further explanations, see the main text.

Standard neuron and model parameters

We consider networks with N^ex = 2500 identical excitatory and Nⁱⁿ = 250 identical inhibitory neurons. The single-neuron parameters for excitatory cells (compare Eq. 1) are C_i ≡ C = 400 pF, V_i^reset ≡ V^reset = −65 mV, V_i^Θ ≡ V^Θ = −45 mV, g_i^L ≡ g^L = 25 nS, V_i^eq ≡ V^eq = −65 mV and t_i^ref ≡ t^ref = 3 ms (Staff et al., 2000; Andersen et al., 2007) for all i. For inhibitory neurons, we set C_i ≡ C = 200 pF, V_i^reset ≡ V^reset = −65 mV, V_i^Θ ≡ V^Θ = =55 mV, g_i^L ≡ g^L = 25 nS, V_i^eq ≡ V^eq = −65 mV and t_i^ref ≡ t^ref = 3ms(Buhl et al., 1996; Geiger et al., 1997) for all i.

For excitatory neurons, the time constants (compare Eq. 2) of the excitatory conductances (AMPA) are τ^ex,1 = 2.5 ms and τ^ex,2 = 0.5 ms (Jonas et al., 1993; Liu and Tsien, 1995), and the time constants for inhibitory conductances (GABA_A) are τ^in,1 = 4.0 ms and τ^in,2 = 0.3 ms (Pearce, 1993; Hájos and Mody, 1997; Bartos et al., 2007). For inhibitory neurons, the time constants of the excitatory conductances (AMPA) are τ^ex,1 = 2.0 ms and τ^ex,2 = 0.35 ms (Geiger et al., 1997; Angulo et al., 1999; Galarreta and Hestrin, 2001), and the time constants for inhibitory conductances (GABA_A) are τ^in,1 = 2.5 ms and τ^in,2 = 0.4 ms.

The connection probabilities (see above, Network setup) are p^ex,ex = 0.08, p^ex,in = 0.1, p^in,ex = 0.1, and p^in,in = 0.02 (Deuchars and Thomson, 1996; Andersen et al., 2007; Memmesheimer, 2010). We note that some of the parameters such as p^ex,in were modified from the experimentally determined values to avoid pathological dynamics in our limited-size networks of leaky integrate-and-fire neurons. Recurrent excitatory neurons are plastic, with τ⁺ = 15 ms, τ⁻ = 30 ms, λ = 0.05, ε₀ = 1 nS, μ = 0.4, and α = 0.68 as standard values for the power-law update rule. For these parameters. the equilibrium weight distributions are approximately Gaussian with mean μ^ex,ex = 0.7 nS and SD σ^ex,ex = 0.16 nS. All other weights are drawn from Gaussian distributions with μ^ex,in = 2.5 nS, μ^in,ex = 1.0 nS, μ^in,in = 2.0 nS and σ^ex,in = 0.2 5 nS, σ^in,ex = 0.1 nS, σ^in,in = 0.2 nS.

The dendritic conduction delays are τ^ex = 1 ms and τⁱⁿ = 0.5 ms; the axonal delays are distance dependent. Neurons are randomly distributed on a square patch with edge length S = 350 μm, and the conduction velocity is v^ax = 300 μm/ms.

The parameters of the dendritic spike current (compare Eq. 8) are chosen according to single-neuron experiments (Ariav et al., 2003; Gasparini et al., 2004; Polsky et al., 2004; Gasparini and Magee, 2006), Θ_b = 8.65 nS, A = 55 nA, B = 64 nA, C = 9 nA, τ^ds,1 = 0.2 ms, τ^ds,2 = 0.3 ms, τ^ds,3 = 0.7 ms, τ^DS = 2.7 ms, and t^ref,ds = 5 ms (cf. Memmesheimer, 2010; Jahnke et al., 2012). The standard value for the length of the dendritic integration window is ΔT^s = 2 ms.

To emulate input from external sources (see above, Network setup), the neurons receive Poissonian random inputs with rates ν^ex,ex = 1.5 kHz, ν^ex,in = 0.5 kHz, ν^in,ex = 0.3 kHz, and ν^in,in = 0.1 kHz and input strengths c^ex,ex = 1.8 nS, c^ex,in = 2.875 nS, c^in,ex = 1.875 nS, and c^in,in = 2.5 nS.

Estimating the propagation frequency of sharp-wave/ripple events

In our model, the spiking probability of a neuron attributable to an excitatory input below the dendritic threshold (no dendritic spike is elicited) is substantially smaller than the spiking probability caused by a suprathreshold input (cf. Jahnke et al., 2012, 2013, 2014a). Therefore, replay events (i.e., propagating synchronous pulses) are mainly mediated by nonlinearly dendritically amplified inputs, and thus the propagation frequency is determined by the average time between presynaptic spikes and postsynaptic spikes elicited by dendritic ones.

The relevant neurophysiological quantities to estimate the propagation frequency are the axonal delay τ^ax, the synaptic delay τ^ex, the latency of the dendritic spike τ^DS, and the average time difference t^ds between the onset of the response to the dendritic spike and the spiking of the postsynaptic neurons (cf. Memmesheimer, 2010). For hippocampal neurons, the synaptic delay τ^ex is typically in the range of 0.5–1.5 ms (Miles and Wong, 1986; Debanne et al., 1995; Boudkkazi et al., 2007), and the latency of dendritic spikes τ^DS is in the order of 2.4–2.9 ms (Ariav et al., 2003). The average axonal delay τ^ax depends on the distance between presynaptic and postsynaptic neurons and the propagation velocity v^ax. The range of local connections in the hippocampus have been measured to be in the order of 300–400 μm (Knowles and Schwartzkroin, 1981; Christian and Dudek, 1988; Oram et al., 1999; Orman et al., 2008; Yang et al., 2014). Therefore, we assume that the neurons are distributed on a quadratic patch width edge length S in the order of 300–400 μm. Together with a conduction velocity ν^ax of 200–400 μm (Andersen et al., 2000; Meeks and Mennerick, 2007), the average axonal delay τ^ax is in the range of 0.3–1.3 ms. The time t^ds between the onset of the response to the dendritic spike and the spike of the postsynaptic neuron depends weakly on the parameters of the current mimicking the dendritic spike and the neurons' ground state. For standard parameters, t^ds is typically between 0.5 and 0.9 ms. Combining the above estimations, the average temporal difference between presynaptic and postsynaptic spikes elicited by a dendritic spike is between 3.7 and 6.4 ms, and therefore the propagation frequency is expected to be in the range of 150–250 Hz.

Results

We model spatial exploration accompanied by place cell activity and theta oscillations in the hippocampus, followed by a resting or slow-wave-sleep phase in which the network generates SPW/Rs and replay. We first consider the encoding phase and model learning of synaptic weights that leads to network structures reflecting place cell sequences during exploration of an environment. These structures are part of a large, sparse, spiking neural network. Thereafter, we consider the recall and show that the sequences are replayed in conjunction with emergent SPW/R-like global network activity. The third part assesses the quality of the replay as well as different ways of its initiation. Finally, we consider storage and recall of multiple sequences. We conclude the main part of the study with a comprehensive discussion of the biological plausibility of the model.

Spatial exploration phase

After an equilibration phase where the distribution of recurrent synaptic weights has become stationary, we consider the movement along a linear track of length L with average running velocity v̄ (compare Fig. 4A). Independent of the exact underlying mechanism, the phenomenon of phase precession yields a compressed version of such sequences: during one theta cycle, neurons with overlapping place fields spike in the same order as the place fields of the single neurons have been or will be traversed (examples are shown in Fig. 4B,C). The (average) time difference between spikes of neurons with nearby receptive fields is in the range of tens of milliseconds, i.e., in the range of STDP (cf. Caporale and Dan, 2008), where, dependent on the exact timing of presynaptic and postsynaptic spikes, changes of connection strengths between neurons are induced. For hippocampal pyramidal neurons in culture, it has been demonstrated that causal spiking results in potentiation and anticausal spiking results in depression of synaptic efficiencies (Bi and Poo, 1998).

The combination of compressed representation of sequences attributable to phase precession with the potentiation of synaptic efficiencies caused by causal spiking may be expected to allow the emergence of feedforward structures (Skaggs et al., 1996; Mehta et al., 1997; Buzsáki, 2006; Bush et al., 2010). Indeed, we observe the formation of a stripe-like feedforward structure after multiple runs along the linear track. In Figure 4D, we show the distribution of synaptic weights as a function of the distance between the place-field centers of the postsynaptic and the presynaptic neuron. Its shape and development can be understood as follows. With repeating traversals of the track, the feedforward structure becomes more and more prominent. Strengths of connections between neurons with similar place fields are altered only weakly. During the run, such neurons spike at similar times, and therefore the order of spiking of these neurons (and with it the potentiation or depression of synaptic weights) is approximately random. Consequently, even after multiple runs along the same track, the weight distribution is similar to the equilibrium weight distribution. With increasing distance between the centers of the place fields, the average temporal distance between spiking times increases, and the order of spiking is more reliably preserved. Accordingly, the average weight change also increases. However, the amplitude of weight changes induced by a single spike pairing decreases exponentially with temporal difference between spikes. Therefore, the average weight assumes a maximum (minimum) for moderate distances between place-field centers and converges to the equilibrium weight distribution for large distances: during the exploration phase, a “stripe-like” feedforward structure is formed (compare Fig. 4D,E).

With more runs along the track, the feedforward structure becomes more prominent. Eventually, this causes a dynamic instability. Such an “overlearning” might be prevented by different mechanisms. For example, other activity patterns between consecutive runs along the same track (e.g., asynchronous background activity or neuronal activation attributable to the exploration of another track) generate different substructures and therefore overform, in particular attenuate, the considered feedforward substructure. Moreover, homeostatic effects or more elaborate learning rules might antagonize the unbounded learning and yield a stationary state even in the setting of ongoing consecutive runs on the very same track. As proof of principle, we check that a STDP kernel where causal spiking leads to depression for large time differences between presynaptic and postsynaptic spikes as experimentally found at CA1 synapses (Nishiyama et al., 2000; Wittenberg and Wang, 2006; Caporale and Dan, 2008) yields stable learning dynamics, where even after a large number of runs no dynamical instabilities occur. We modify Equation 9 to the following:

where τ^x > τ⁺ is the time scale for depression induced by causal spiking and A > 1 is a normalization constant (Fig. 5B). The potentiation for short time differences leads to the formation of the feedforward structure as before, and the depression for large time differences hinders the increase of synaptic weights between neurons with distant place-field centers.

Figure 5.

Alternative STDP kernel generating stable learning dynamics for large numbers of runs. A, B, Top, STDP time windows (compare Eqs. 9 and 26); bottom, distributions of weights versus the distances of the place-field centers of postsynaptic and presynaptic neurons (compare Fig. 4D). A, A standard STDP window (compare Eqs. 9–11) results in more prominent feedforward substructures for more runs along the linear track. After a large number of runs, replay events are initiated even during the exploration phase and cause pathological activity, which amplifies the connection strengths even more. B, A different STDP window where causal spiking induces depression for larger Δt yields stable learning dynamics (compare Eq. 26, with A = 8, τ^x = 17 ms and α = 0.18; other parameters as before). As in A, a prominent feedforward structure forms, but the depression for large Δt confines strong connections to neurons with nearby place fields and thus prevents overlearning and pathological activity states.

We note that for the sake of simplicity, we have chosen a standard simple pair-based plasticity rule (compare Eqs. 9–11). Neurophysiological experiments have demonstrated that synaptic plasticity in cortical networks is highly diverse and variable and requires more complex models to be adequately described (Pfister and Gerstner, 2006; Wittenberg and Wang, 2006; Clopath et al., 2010; Graupner and Brunel, 2012). However, the formation of feedforward substructures essentially only requires a potentiation attributable to causal spiking, a feature experimentally found in many cortical areas and shared by many, also more elaborate models. For example, the triplet rule proposed by Pfister and Gerstner (2006) that is fitted to explain the experimental data from hippocampal data sets is essentially a standard pair-based STDP rule extended with additional terms that are proportional to presynaptic and postsynaptic activity. Therefore, we checked that this triplet rule yields fast, reliable formation of feedforward structures as well.

Finally, we note that we have chosen a comparatively narrow distribution of phase shifts to illustrate the proposed mechanism for learning and replay in the clearest way possible. A broader distribution yields qualitatively the same results. The spiking times of neurons with activated place fields are distributed over a longer time interval, and thus the potentiation of synaptic weights is less effective because of a stronger mixture of potentiation and depression and the exponential decay of the STDP time window. However, this can be compensated by, for example, an increased learning rate or a larger number of runs along the track (compare Fig. 6).

Figure 6.

Impact of unreliable spiking phase. A–C, Spatiotemporal receptive fields with different widths (A, Δτ = 20 ms, I^max = 500 pA; B, Δτ = 40 ms, I^max = 340 pA; C, Δτ = 60 ms, I^max = 280 pA). The underlying theta oscillation has a frequency of 8 Hz. D–F, Distribution of synaptic weights as a function of the distance between the place field centers of the postsynaptic and the presynaptic neuron after multiple runs along the linear track with phase precession given by the receptive field above. The running speed is v̄ = 8 cm/s, and the place-field width is Δw = 40 cm. Broader distributed, more unreliable phase shifts do not qualitatively change the feedforward structure formation. It only requires more runs, because the actual spiking times have less reliable order and temporal difference.

Replay and emergence of SPW/R-like events

Is it possible to read out the stored information, i.e., to replay the experienced spike patterns, after the exploration phase? In the following, we show that the formed structures indeed enable such a replay and that the observed replay resembles the typical replay observed in the hippocampus in key aspects: it occurs together with ripple oscillations and a sharp-wave-like overall increase in activity.

In the ground state, the neurons spike asynchronously and irregular. To keep the ground-state activity of networks after a different number of runs comparable (and to account for possible homeostatic effects), we adjust an external constant input current I⁰ to the neurons such that the average spiking rate of the neurons equals ν = 1 Hz. Replay events are initiated in a targeted manner by eliciting a group of g₀ neurons with adjacent place fields to spike synchronously (compare Fig. 7). In the hippocampus, appropriate inputs may originate from neocortical areas, as part of the activity typically preceding SPW/Rs (Sirota et al., 2003; Sullivan et al., 2011; Logothetis et al., 2012). The elicited spiking causes a synchronous input to the postsynaptic neurons after some delay time and, therefore, may induce synchronous spiking of a fraction of these neurons. Because of the feedforward structure, neurons with place fields subsequent to place fields of the initially synchronous subset of neurons receive stronger total input than other neurons of the network and, therefore, have a higher probability to spike. In particular, in these neurons more dendritic spikes are initiated that elicit a subsequent somatic spike with high probability and high temporal precision. As a consequence the spikes of the postsynaptic neurons remain synchronized even if the synaptic delays are moderately heterogeneous. Only spikes received within the dendritic integration window ΔT^s can contribute to the initiation of a dendritic spike, and inputs that arrive outside a small window around the peak of an input pulse (consisting of multiple input spikes) are unlikely to generate dendritic spikes and somatic output. This keeps the propagating activity confined to highly synchronous pulses, which appear as ripple oscillations in unspecific mass signals such as the overall network rate.

As a consequence of the stability of the unexcited state of the network dynamics without a propagating pulse, for small g₀, only a small number g₁ of the postsynaptic neurons spike in response to the initial synchronous pulse. With increasing g₀, the size g₁ of the secondary synchronous pulse also increases, and for sufficiently enhanced feedforward coupling, at some point g₁ exceeds g₀, i.e., more than initially synchronized neurons are excited to spike synchronously. This secondary synchronous pulse induces a third one, for which similar rules hold, the third pulse induces a fourth one and so on. The synchronous activity can propagate across the network reflecting the previously learned feedforward structure (compare Fig. 7C–E). The increased activity along the chain appears as a sharp wave increase in unspecific neural mass signals such as the overall network rate.

The replay in form of consecutive synchronized groups (ripples) requires some degree of synchronicity of the initial stimulation. The initial spiking times are drawn from a Gaussian distribution with SD σ^stim = 0.5 ms. With increasing σ^stim, even if the pulse is still able to propagate along the previously learned feedforward structure, the replay is not clearly structured in consecutive synchronized groups (compare Fig. 7F). Accordingly, with increasing σ^stim, the relative power in the high-frequency band between 150 and 250 Hz decreases (compare Fig. 7B).

In contrast to layered feedforward networks (Diesmann et al., 1999; Vogels and Abbott, 2005; Jahnke et al., 2012), where the size of each synchronous pulse is bounded from above by the layer size, in our networks the sizes of the synchronous pulses may be expected to grow, and potentially lead to epileptic-like pathological activity. Such pathological activity, however, is prevented by the inhibitory feedback loop: increasing activity of excitatory neurons causes increasing activity of the inhibitory neuron population. The resulting inhibitory feedback does not hinder the generation of dendritic spikes, but it decreases the probability that a somatic spike is initiated by hyperpolarization of the postsynaptic neurons (cf. Müller et al., 2012). As a consequence, for very large synchronous pulses, the inhibitory feedback overwhelms the excitation, the pulse does not spread further across the network, and the overall activity decays to the level of spontaneous activity. Pathological activity in the form of global synchrony is prevented.

If the initial synchronous pulse is too small, insufficiently many neurons respond to the initial pulse and the activity decreases quickly. If the initial stimulation is too large, the size of the synchronous pulse grows too fast and the replay event is terminated almost immediately by recurrent inhibition (compare Fig. 7A). In between, there is a broad range of initial stimulation sizes for which synchronous activity propagates along the feedforward structure lasting for a moderate number of synchronous pulses (ripples). There is an optimal stimulation size g*, where the number of ripples becomes maximal (Fig. 7, compare A, black square, E). g* separates the regime where the sizes of the synchronous groups decay from the beginning and the regime where the sizes of the synchronous groups initially grow (cf. Jahnke et al., 2012, 2014a).

A replay event is also elicited by initial stimulations g₀ exceeding g* (compare Fig. 7D for an example). However, with increasing g₀, the sizes of subsequent synchronous groups as well as the inhibitory feedback increase, and thus the event is terminated by recurrent inhibition after a shorter number of ripples. The moderate number of ripples is consistent with the low number of ripples observed in neurophysiological experiments (Buzsáki et al., 1992; Ylinen et al., 1995; Maier et al., 2003).

Importantly, the observed replay resembles the typical replay observed in the hippocampus in key aspects: (1) it occurs together with high-frequency ripple oscillations (Nadasdy et al., 1999; Lee and Wilson, 2002); (2) the averaged network activity increases; (3) the waveform of the averaged network activity resembles those of SPW/Rs (Buzsáki et al., 1992; Ylinen et al., 1995; Maier et al., 2003); (4) it is strongly stochastic, i.e., has a high trial-to-trial variability (Wilson and McNaughton, 1994; Nadasdy et al., 1999; Lee and Wilson, 2002; Pastalkova et al., 2008; Davidson et al., 2009); and (5) it is a short intermittent activity pattern on top of asynchronous low-rate activity.

Finally, we note that replay events are mediated by dendritic spikes, and thus a sufficiently synchronous input is required to induce such an event. Therefore, during the exploration phase where phase precession occurs, no (or only marginal) replay takes place as long as the feedforward structure is not too prominent (overlearning; see discussion in the previous section), because the spiking times are not sufficiently synchronized.

Characteristics of replay events

To be able to compare the characteristics of the replay events with experimental data, we quantitatively analyze them in the following. In particular, we focus on the propagation frequency and the quality of the ordering of single spikes during replay events compared with the ordering of the place-field centers in the track.

Replay events are mainly mediated by dendritic spikes, and the propagation frequency is determined by the average temporal difference t^diff between presynaptic spikes and postsynaptic spikes elicited by dendritic ones. The expected propagation frequency can thus be calculated taking the axonal delay τ^ax, the synaptic delay τ^ex, the latency of the dendritic spike τ^DS, and the average time difference t^ds between the onset of the response to the dendritic spike and the spiking of the postsynaptic neurons into account. Considering quantitative neurophysiological measurements, we estimate the expected temporal difference t^diff to be in the range of 3.7–6.4 ms (see Materials and Methods for more details). The expected propagation and therewith ripple frequency (∼150–250 Hz) is consistent with the frequencies observed during SPW/Rs in the hippocampus (Buzsáki et al., 1992; Ylinen et al., 1995; Maier et al., 2003; Stark et al., 2014; cf. Memmesheimer, 2010). Indeed, for the standard parameters used in the study (chosen to be approximately centered within the biologically plausible parameter range), the propagation frequency is ∼200 Hz as illustrated in Figure 8A.

Figure 8.

Characteristics of replay events. A, Bottom, Spectrogram of the spiking rate dynamics of the excitatory neuron population displayed in Figure 7E. Top, Corresponding network rate. There are clear maxima at a frequency of about 200 Hz during replay events. Furthermore, there are peaks at the higher harmonics and at low frequencies reflecting the overall increase of activity. B, Average matching index I (solid lines) versus the number of initially synchronously spiking neurons g₀ after different numbers of runs along a linear track (same data and color code as in Fig. 7A). Only runs where at least two ripples are initiated are considered for the analysis; therefore, the curves are truncated at the left side. The shaded areas indicate the value range containing 50% of the values (0.25–0.75 quantiles). With the increasing number of ripples (compare Fig. 7A), the quality of the ordering within replay events increases. C, As a control experiment, we shuffled the neuron indices and calculated the matching index: it fluctuates around zero, highlighting that the sequence is randomly ordered.

To quantify the quality of ordering of spikes in a replay event, we calculate the matching index I (see Materials and Methods) with respect to the original ordering of place-field centers for each event. Briefly, the matching index I is a real number in the interval [−1, 1], which is I = 1 for perfect replay of the original sequence, I = −1 for reverse replay, and I ≈ 0 for random replay sequences. In Figure 8B, we show the average matching index for replay events induced after different numbers of runs along the linear track. The matching index increases with the number of ripples, i.e., the number of successive synchronous groups (compare Fig. 7A), and becomes maximal for the same initial stimulations for which the number of ripples is maximal.

However, even for the optimal parameter range, we find I < 1, i.e., the sequence is not perfectly ordered. To understand this observation, we keep in mind that in contrast to the often studied layered feedforward networks (Diesmann et al., 1999; Vogels and Abbott, 2005; Kumar et al., 2010; Jahnke et al., 2012), there are no distinct groups of neurons that constitute the potential members of each synchronous pulse. Instead, the neurons participating in subsequent synchronous pulses are recruited from the set of neurons with place-field centers following the place-field centers of the neurons of the preceding synchronous pulse. Thus, whereas the synchronous pulse propagates along the previously learned feedforward structure, the range of place-field centers of neurons of consecutive synchronous pulses overlap (compare Fig. 7E1). Furthermore, the ordering of spikes within one synchronous pulse depends strongly on the current state of the neurons after reception of the synchronous input, and thus the spiking order within one of these pulses is approximately random. Both effects cause deviations between the order of spikes in the replay event and the order of the place-field centers and, therefore, reduce the matching index I below the maximal value I^max = 1. Nonetheless, the “center of place-field centers” of neurons participating in subsequent synchronous pulses propagate along the feedforward structure (Fig. 7, compare D, E). After a few number of runs along the linear track, the observed matching indices are substantially larger than those computed from a control scenario where the neuron indices are shuffled (Fig. 8, compare B, C).

Robustness of the occurrence of replay events

Besides the number of runs along the linear track, the widths Δw of the place fields, the maximal synaptic modification, and the strength of the excitatory–inhibitory feedback loop also influence the properties and quality of replay events. In the following, we show that the emergence of replay and SPW/R-like activity is robust against changes of the network setup.

With increasing place-field width Δw, the range of positions

where a single (place encoding) neuron i becomes active increases. Assuming a fixed running velocity v̄ and learning rate λ, this increase implies that each neuron is activated for a longer time interval during one run. Thus, the total number of spike pairings between place-encoding neurons also increases, resulting in a faster formation of the feedforward structure. Indeed, with increasing Δw, the average number of ripples per replay event (compare Fig. 9A) as well as the ordering (compare Fig. 9B) increase for a fixed number of runs. This increase is similar to the increase achieved by an increasing number of runs for fixed Δw (compare Figs. 7A, 8B).

Figure 9.

Impact of place-field width and strength of inhibitory feedback loop on replay events. The number of ripples (A, C) and the matching index (B, D) versus the number of g₀ of initially synchronous neurons after a fixed number of runs (averaged over n = 10,000 replay events) are shown. In A and B, different colors indicate different place-field widths Δw, and in C and D, different colors correspond to different scaling γ of the inhibitory feedback loop (compare Eq. 28) as indicated above the panels. Increasing Δw yields a faster learning of feedforward structure, and thus a faster increase of the ordering of replay events. The inhibitory feedback controls the number of ripples per replay event: with decreasing γ, more and more ripples (i.e., longer events) emerge. However, the longer replay events do not lead to more ordered sequences as the synchronous signal tends to spread out over larger parts of the recurrent network. For more detailed discussion, see the main text.

As argued above, the excitatory-to-inhibitory and inhibitory-to-excitatory coupling controls the number of ripples and the length of replay events. To illustrate the influence of this feedback loop, we scale the relevant coupling strengths by a factor γ:

where ε_inex (ε′_inex) denotes the (modified) excitatory-to-inhibitory and ε_exin (ε′_exin) the (modified) inhibitory-to-excitatory coupling strengths. With decreasing γ (less prominent feedback loop), we observe more and more ripples (compare Fig. 9C), and replay events become longer and longer. However, the quality of the ordering within the replay events does not increase (compare Fig. 9D). Synchronous activity propagates along the learned feedforward structure (see previous subsection), but additionally the synchronous activity tends to spread to other neurons with place fields not subsequent to the currently active synchronous group. For sufficiently large γ, only the strongest signals (i.e., the projection of the propagating signal on the neurons with subsequent place fields) are sufficiently strong to overcome the inhibitory feedback, which yields more ordered replay sequences (compare Fig. 9D).

We conclude that the occurrence of induced replay events is robust against variations of initial stimulation size, number of runs along the linear track, place-field width, and strength of the inhibitory feedback loop.

Unspecific stimulation

So far, we considered replay events triggered by a specific (read-out) stimulation, i.e., events are started by synchronous spiking of a subset of neurons with neighboring place-field centers. However, with more and more prominent feedforward structures (achieved, for example, by increasing numbers of runs), the minimal number of initially synchronized neurons sufficient to elicit a replay event decreases. This decrease suggests that replay events might also be triggerable by unspecific stimulation of a random subset of neurons, which already contains subsets of neurons with nearby place-field centers by chance.

We test this conjecture, by stimulating a small subset of g₀ randomly selected neurons out of the total population of N^ex neurons to spike synchronously. If the formed feedforward structure is sufficiently prominent (i.e., after sufficiently many runs along the track and/or for sufficiently large place-field widths Δw), this random stimulation indeed elicits SPW/R-like events with a moderate number of consecutive synchronous groups (ripples).

To evaluate whether these events contain information about the order of the place-field centers, we calculate the matching index I with respect to the original order of the place-field centers as before. Additionally, we test whether the information about the order can already be inferred from the activity of a limited number of neurons (as is the case in experimental setups where only spike data from a limited number of neurons are available). We calculate the matching index from a randomly selected subset of S neurons out of all place-encoding neurons. Indeed, the distribution of matching indices (Fig. 10A) clearly shows that the order within the replay events is nonrandom and reflects the original order of place-field centers along the linear track (examples of spiking activity of the subset S of selected neurons are shown in Fig. 10B–D).

Figure 10.

Replay events elicited by unspecific (random) stimulation. After an initial exploration phase (15 runs, Δw = 50 cm, remaining parameters as before), a random subset of g₀ = 50 excitatory neurons (N^ex = 2500) is stimulated to spike synchronously. We consider all events with at least five ripples, i.e., five consecutive synchronous groups (3273 of 10,000 events). For each of these events, we draw S = 200 neurons out of the N^pf = 1500 place-encoding neurons and calculate the matching index I for the spiking activity of these neurons only (to gain better statistics, we repeated the drawing 100 times for each event and evaluated the computed I independently). A, The resulting distribution of matching indices (orange) is shifted to positive values compared with the control setting where the neuron indices are shuffled before calculation of the matching indices (black): the spikes of the neurons within the events are ordered according to the order of their place-field centers along the linear track. B–D, Examples of spiking dynamics of subsets of S = 200 neurons (main panels) and the population activity (top; black, excitatory neurons; red, inhibitory neurons) for different matching indices I as indicated in the top panels.

Recall of multiple sequences

The hippocampus is assumed to serve as a preliminary storage device for information (Marr, 1971; Buzsáki, 1989; Willshaw and Buckingham, 1990), and simultaneous storage (and recall) of multiple memory contents is essential for its function. In the following, we demonstrate that multiple sequences can be simultaneously stored and successfully recalled in a single network.

We consider the encoding of n^tr different linear tracks. For each track, we assume that a number N^pf of neurons that are randomly chosen from the total set of N^ex excitatory neurons encodes a position on the specific track. As before, the encoded positions (i.e., the place-field centers) are distributed homogeneously along the corresponding track and randomly assigned to the selected place-encoding neurons. In the exploration phase, all n^tr tracks are traversed one after each other, and thus multiple stripe-like feedforward substructures are formed. For the recall phase, we selectively stimulate replay events corresponding to one of the traversed tracks: g₀ neurons with neighboring place-field centers with respect to one of the tracks are stimulated synchronously. We analyze the elicited replay events, by calculating the matching index with respect to the order of place-field centers for each of the trained tracks. The distributions of these matching indices show that the different feedforward structures can be selectively activated (Fig. 11). We conclude that even relatively small networks with simple (globally random, sparse) connectivity enable the storage of multiple sequences and that a successful recall is possible, despite a substantial overlap between different feedforward substructures.

Figure 11.

Recall of multiple sequences. Before recall, the network is trained in an exploration phase on n^tr = 3 different linear tracks (Δw = 40 cm, 30 runs in total; remaining parameters as before), which are traversed one after another. The set of neurons (N^pf = 1500 neurons per track) that encode places on each track are drawn randomly. In the recall phase, g₀ = 40 neurons with neighboring place fields are stimulated synchronously (compare Fig. 7). A, C, Distribution of matching indices (n = 10,000 events) with respect to the order of place-field centers of different tracks (color code). The set of initially stimulated neurons is taken from place-encoding neurons of track 2 (A) or track 3 (C), respectively. B, D, Example rasterplots of all place-encoding neurons of different tracks (same color code as in A and C; I_ID refers to the matching index for the track ID). The neurons are ordered according to the ordering of place-field centers. We note that the neurons encode positions on multiple tracks. By chance, the order of firings can agree or disagree on two tracks for a larger majority of neuron pairs. Such an “overlap” leads to distributions of matching indices for the nonstimulated tracks that are clearly shifted from zero, as in (C).

Learning during replay

Experimental observations suggest that plasticity may be reduced during states characterized by SPW/Rs (Leonard et al., 1987; Bramham and Srebro, 1989). However, it is an important and open question, how (remaining) plasticity shapes the network topology during SPW/R events. It has been hypothesized that such events may consolidate (Buzsáki, 1989; Nadasdy et al., 1999) or erase (Buzsáki, 2006; Mehta, 2007) memory content.

Although the considered simple STDP model favors the formation/reinforcement of feedforward substructures in general, there are other mechanisms, e.g., the irregular background activity and the stochasticity of the replay, that counteract such a potentiation. For example, for the stripe-like feedforward pattern (compare Fig. 4), in contrast to layered feedforward structures, there are no defined subsets of neurons that constitute the potential members of each synchronous pulse. Each synchronous pulse recruits the members of the consecutive synchronous pulse predominantly from the neurons with place fields ahead, but the place-field centers of neurons of consecutive pulses overlap such that the spatial order is not faithfully reflected in the temporal one. Moreover, the ordering of spikes within a pulse depends on the neuronal state when receiving the synchronous input and is therefore rather random. Additionally, the consecutive activation of different stored trajectories combined with the overall increased activity during SPW/R events might distort the single substructures.

To address the question in a first, exemplary way, we assume two kinds of replay initiation as before: specific stimulation of neurons at the beginning of a learned structure (compare Fig. 7) and unspecific stimulation (compare Fig. 10). For specific stimulation, we observe that the stimulated neurons strengthen their connections to neurons all over the network, which leads to overall network bursts, both in the spontaneous activity and in response to stimulations. Unspecific stimulation evokes highly noisy replay as displayed in Figure 10. In both cases, perhaps contrary to intuition, we observe that the network structure and thus the replay decline (see Fig. 12 for an illustration of the case of unspecific stimulation). We expect that different ways of replay initiation and/or different learning rules change these observations. Indeed, the plasticity rules responsible for restructuring may be different from those in exploration phases, because of the high level of inhibition during SPW/Rs (Nishiyama et al., 2000; Aihara et al., 2007; Cutsuridis, 2013), and it may be necessary to account for the prevalence of dendritic spikes (Remy and Spruston, 2007; Losonczy et al., 2008; Legenstein and Maass, 2011; Müller et al., 2012).

Figure 12.

Impact of learning during rest/sleep with SPW/R-like replay events. The feedforward substructure is first learned in an exploration phase (for 5, 10, and 15 runs along the tracks in A–C, respectively; the network is the same as in Fig. 10). The networks continue to learn in the subsequent simulated rest/sleep phase, where SPW/R-like events are initiated by stimulating a random subset of g₀ = 50 excitatory neurons every 400 ms. Different colors show the mean connection strength as a function of the distance between the place-field centers of the postsynaptic and the presynaptic neuron after 0, 500, 1000, and 4000 events. The feedforward structure is attenuated.

Models for sequence learning and sequence generation

On the basis of relatively scarce experimental evidence available at the time and earlier theoretical ideas (Marr, 1971), Buzsáki (1989) suggested in an abstract theoretical model that during exploratory activity accompanied by theta rhythms, memories are stored in the hippocampus by synaptic plasticity in a weak and transient form. In subsequent phases of rest and sleep, the memories are consolidated and/or transferred to neocortical regions by SPW bursts. Later, computational approaches focused on different aspects of this process to study how they may be realized in neural networks, using models of different degrees of abstraction. In the following, we will review such models studying sequence replay and generation, to put our model in context. Predominantly, the previous models focused on the encoding part of sequence generation and did not take into account biologically plausible SPW/R-like activity.

In a series of studies, Levy and coworkers (Levy, 1996; August and Levy, 1999; Sullivan and Levy, 2004) explored characteristics of sequence learning, and later time compressed recall, on different levels of abstraction. Most related to our study, August and Levy (1999) considered a population of spiking, leaky integrate-and-fire neurons with excitatory connections and overall averaged inhibition. In an exploration phase, part of the neuron population is sequentially stimulated, and as a consequence of learning, the unstimulated neurons also become sensitive to a specific stimulation period: the entire network dynamics organize into one activity sequence. In a subsequent recall phase, synaptic plasticity is switched off, the level of inhibition is lowered, and random or targeted external stimulation is applied to the network. This evokes compressed replay of the activity sequence observed during exploration. The model was modified to learn multiple sequences of subsequently active neuron groups with a multiplicative learning rule including an additional synaptic competition term (Samura et al., 2008).

Molter et al. (2007) described place cell populations as coupled phase oscillators, which possess an intrinsic theta oscillation frequency during exploration. The model incorporates theta phase precession by assuming that the intrinsic frequency increases with progress in the place field such that the activity in different place cell populations peaks sequentially within the theta cycle. During exploration, recurrent connections are functionally inactivated but learned. This allows later recall of the sequence when recurrent connectivity is switched on.

Bush et al. (2010) implemented the theta phase precession by a modulation of the external input current that depends on the position relative to the place-field center and the theta phase. The study uses a spiking network of 100 excitatory neurons to show that different additive spike- and spike/rate-dependent learning rules are suitable to learn activity sequences. The network organizes into a feedforward chain where the weights between subsequently activated groups of neurons have weights saturated at the maximal weight, whereas background weights that are not part of the chain have weights close to zero.

Cutsuridis and Hasselmo (2011) used a CA1 microcircuit of four pyramidal cells and four interneurons of different types to study how the experimentally found distinct firing patterns of the involved neuron types may be relevant for encoding and replay. The generation of a short activity sequence is imposed on the CA1 microcircuit model by a CA3 network that is not explicitly modeled.

Recently, Scarpetta and coworkers (Scarpetta and Giacco, 2013; Scarpetta et al., 2013) estimated the ability of networks to store multiple precisely timed, simple periodic spike sequences (every neuron contributes one spike) in networks of leaky integrate-and-fire neurons endowed with additive Hebbian learning. Additional studies used two-state neurons to assess the network capacity for activity sequences (Leibold and Kempter, 2006; Scarpetta et al., 2011) or rate units to model learning of sequential activation of neuron populations (Verduzco-Flores et al., 2012).

Another recent study by Vladimirov et al. (2013) considered the replay of spike sequences in conjunction with SPW/R-like activity for a gap junction-based SPW/R model. The model assumes that the plexus of proximal axon collaterals is connected by axo-axonic gap junctions. In this plexus, spikes multiply in an avalanche-like manner, overall generating a continuous ripple-frequency oscillation. The spikes enter the main axon and the soma only if the soma receives a (subthreshold) dendritic depolarization. By this mechanism, weak dendritic inputs are amplified, and somatic spiking was observed to be propagated over short chains of monosynaptically connected single neurons.

Replay (and preplay) of spike sequences might also arise from continuous attractor dynamics. Here a localized bump of activity in the neural tissue moves around because of asymmetric synaptic connections, short-term plasticity, or adaptation mechanisms (Tsodyks et al., 1996; Tsodyks, 1999; Itskov et al., 2011; Azizi et al., 2013; Romani and Tsodyks, 2015).

Activity propagation along feedforward structures has, furthermore, been investigated as a model for reliable information transmission in noisy networks, essentially independently from hippocampal sequence learning (Abeles, 1982; Diesmann et al., 1999; Kumar et al., 2010). These studies on “synfire chains” have mostly considered feedforward networks with a dense, often all-to-all connectivity between subsequent layers (Aviel et al., 2003; Mehring et al., 2003; Kumar et al., 2008). However, since cortical neural networks are overall sparse (Braitenberg and Schüz, 1998; Holmgren et al., 2003), we may also expect some level of dilution for embedded feedforward chains. Such chains created from existing connections in sparse recurrent networks require strong synaptic efficiencies and specifically modified neuron properties to enable synchrony propagation (Vogels and Abbott, 2005). Recently, we have shown that the nonlinear dendritic interactions that have been suggested to underlie generation of SPW/Rs (Memmesheimer, 2010) promote propagation of activity along biologically plausible, comparably weak, and highly diluted synfire chains (Jahnke et al., 2012, 2013).

SPW/Rs and replay with dendritic spikes in the light of neurobiological knowledge

Above we have derived a unifying model to understand learning of activity during exploratory phases and its recall together with emergent SPW/Rs. In our networks, SPW/Rs occur because of nonlinear dendritic interactions. In the following, we review this and other proposed mechanisms for their generation and justify our model by discussing experimental evidence and theoretical arguments.

Models for SPW/Rs

So far, mainly three candidate mechanisms have been suggested to underlie or support sharp waves. They may depend on short-term plasticity leading to population bursts (Deuchars and Thomson, 1996; Loebel and Tsodyks, 2002; Memmesheimer, 2010), on recurrent excitation enhanced by nonlinear dendrites and inhibition (Memmesheimer, 2010) and/or on recurrent excitation and inhibition only (Taxidis et al., 2012).

For the ripple oscillations, mainly four models exist. The first one assumes that the presence of excitatory input attributable to a sharp-wave input excites the interneuron networks to oscillate. The inhibition from the interneurons entrains the phasic spiking of the pyramidal cells (Buzsáki and Chrobak, 1995; Ylinen et al., 1995) and thereby yields network oscillations in the ripple-frequency range in response to both constant (Brunel and Wang, 2003; Geisler et al., 2005) and sharp-wave-like transient input (Taxidis et al., 2012, 2013).

Second, it has been proposed that the oscillations might depend on the excitatory–inhibitory loop (Geisler et al., 2005; Stark et al., 2014), like in the gamma “PING” mechanism (Börgers and Kopell, 2003; Brunel and Wang, 2003; Geisler et al., 2005; Bartos et al., 2007; Tiesinga and Sejnowski, 2009): This feedback loop can generate oscillations in the ripple-frequency range, and the fast response properties of the involved interneurons (cf. Geiger et al., 1997; Csicsvari et al., 1998; Jonas et al., 2004) allow for the experimentally observed phase differences between the preferred spiking of the pyramidal and the interneuron populations (Geisler et al., 2005; Memmesheimer, 2010). Recurrent inhibition may serve to synchronize spatially distinct oscillations (Stark et al., 2014).

The third model is based on the assumption that axo-axonal gap junctions (Schmitz et al., 2001; Hamzei-Sichani et al., 2007) connect pyramidal cell axons to a network where spikes can propagate and multiply in the presence of an external depolarizing input (Traub et al., 1999; Traub and Bibbig, 2000; Maex and De Schutter, 2007). Rhythmic generation of bursts of axonal spiking are generated, which excite pyramidal cell and interneuron somata to spike after antidromic and orthodromic spike propagation.

The fourth model is based on nonlinear dendrites that enable propagation of synchronous activity (Memmesheimer, 2010; Memmesheimer and Timme, 2012; Jahnke et al., 2013) and thereby generate sharp-wave-like events with high-frequency ripples. The ripple-frequency range is determined by experimentally measured characteristics of nonlinear dendrites (Ariav et al., 2003; Müller et al., 2012) and agrees with the experimentally found one. Slow dendritic spikes in the interneurons may also enable or contribute to ripple-frequency oscillations (Chiovini et al., 2014).

The models mentioned above are not mutually exclusive, and in view of constraints imposed by current neurobiological knowledge on SPW/Rs, all models may be considered biologically plausible.

The choice of the SPW/R model in our study

As basis of our study, we have chosen the model first suggested by Memmesheimer (2010), because of recently increasing experimental evidence and theoretical arguments in its favor as discussed in the following. For completeness, we also shortly mention some of the points already given by Memmesheimer (2010) again.

1. We first consider basic anatomy. Both hippocampal regions CA1 (Deuchars and Thomson, 1996; Orman et al., 2008) and CA3 (Miles and Wong, 1986; Amaral et al., 1990; Traub and Miles, 1991) exhibit excitatory recurrent connections between pyramidal neurons. Individual connections are strong (Deuchars and Thomson, 1996; Le Duigou et al., 2014). Supporting the model, it has very recently been found that the hippocampal region CA1 possesses a more prominent recurrent excitatory connectivity than previously thought (Yang et al., 2014). Recurrent pyramidal–pyramidal connections target basal dendrites in CA1 (Deuchars and Thomson, 1996; Klausberger and Somogyi, 2008) and basal and apical dendrites in CA3 (Li et al., 1994; Le Duigou et al., 2014). As required for the model, basal dendrites in CA1 (Ariav et al., 2003; Müller et al., 2012) and basal and apical dendrites in CA3 (Kim et al., 2012; Makara and Magee, 2013) exhibit fast dendritic spikes. We note that these had not been demonstrated in CA3 when the model was originally proposed; rather, it predicted their existence. The newly demonstrated prominent longitudinal connectivity may not only contribute to SPW/R generation but also enable their propagation along the longitudinal axis (Patel et al., 2013).

2. The events generated in the model networks resemble the experimentally measured ones in their shape, duration, firing, and current input characteristics. In particular, as discussed in Materials and Methods, the model explains the frequency of the SPW/Rs from anatomical knowledge on hippocampal recurrent excitatory connectivity (Knowles and Schwartzkroin, 1981; Christian and Dudek, 1988; Yang et al., 2014), from experimentally measured conduction delays (Andersen et al., 2000; Meeks and Mennerick, 2007), and from single-neuron properties, from the latency of dendritic spikes and their impact on the somatic membrane potential (Ariav et al., 2003; Kim et al., 2012; Müller et al., 2012; Makara and Magee, 2013). The model explains that ripples in CA3 have lower frequency and are less marked than those in CA1, by the larger average and broader distributed axonal conduction delay in the more globally connected CA3 network. Similarly, the higher frequency of in vitro ripples is explained by the loss of longer-range recurrent connections during slicing. Recent experiments show that fast dendritic spikes evoke action potentials in CA3 neurons at a longer latency than in CA1 (Makara and Magee, 2013). In the framework of the model, such an increase will also contribute to a lower ripple frequency. Furthermore, the somatic depolarization in response to fast dendritic spikes is generally smaller in CA3 than in CA1. The accompanying NMDA spikes may thus exert a stronger influence on ripple generation. They are less synchrony dependent, have a longer latency, and lead to somatic action potentials with lower temporal precision; this may add to the relatively lower frequency and less precision of CA3 ripples.

3. As shown in Results, the model jointly explains the sharp wave and the ripples as one collective phenomenon and does not require the assumption of an external sharp-wave input for region CA1, consistent with the experimental observation of SPW/Rs in the functionally disconnected or partially deafferented CA1 in vitro and in vivo (Maier et al., 2003, 2011; Nimmrich et al., 2005; Nakashiba et al., 2009). In the present study, replay events are initiated by synchronous (possibly unspecific) input, yet the sharp wave and the ripples arise from the dynamics of the recurrent network. Furthermore, it has been shown for the case of randomly connected model networks that SPW/R-like events can occur spontaneously (Memmesheimer, 2010).

4. The model depends on recurrent excitation between pyramidal neurons. Indeed, in vitro experiments find during SPW/Rs locally generated ripple-locked excitatory input in pyramidal neurons, and the suppression of locally generated excitation suppresses SPW/Rs (Maier et al., 2003, 2011). Consistently, in vivo the activation of local pyramidal neurons leads to ripples and prolongates ripple oscillations in spontaneous events; the suppression of local pyramidal neurons even during an event suppresses the oscillations (Stark et al., 2014). In contrast, the activation of interneurons does not generate ripple oscillations and rather terminates spontaneous events by suppressing pyramidal neurons. This supports the chosen model and other models involving local pyramidal neuron activity, and it may question models that explain the ripple oscillations on the basis of externally driven interneuron networks.

5. The model assumes that the excitatory inputs mediated by recurrent connections can generate dendritic spikes. Dendritic spikes have been directly detected in hippocampal region CA1 during SPW/Rs, albeit in apical dendrites (Kamondi et al., 1998). Furthermore, spikelets, short deflections of voltage that may reflect dendritic spikes (or action potential transmission through gap junctions), have been detected during exploration and rest (Harvey et al., 2009; Epsztein et al., 2010). In the next section, we provide a rough estimate indicating that we may expect dendritic spikes to be generated by recurrent excitation even in the very sparsely excitatorily interconnected region CA1.

6. During SPW/Rs, basket cells spike at a high rate and exert strong inhibition on pyramidal neurons (Ylinen et al., 1995; Klausberger and Somogyi, 2008; Maier et al., 2011). Such inhibition suppresses the impact of excitatory inputs that are not or only weakly dendritically amplified (Müller et al., 2012). In contrast, strong dendritic spikes that form the basis of the chosen model, and their triggering of action potentials, are robust against recurrent and feedforward inhibition (Kamondi et al., 1998; Müller et al., 2012). The presence of several dendrites can lead to a further increase in the effect of the supralinearity, because more than one dendrite can generate spikes (Breuer et al., 2014). As shown above, in our model the strong inhibition serves to confine the replay to the previously learned structures.

7. Under a local block of inhibition, externally stimulated ripple oscillations give way to higher-frequency pathological epileptic oscillations (Stark et al., 2014). In the model, ripples give way to continued high-frequency firing (Memmesheimer, 2010). Incorporation of adaptation or short-term synaptic plasticity might restore intermittent events. Weakening/elimination of oscillations under partial inhibition block (Stark et al., 2014) might occur because of stronger influence of interactions that do not depend on synchronous inputs. When general inhibition strength is increased, in vivo recordings in region CA1 find a moderate decrease of ripple frequency (Ponomarenko et al., 2004). In contrast, in vitro recordings in CA3 find no influence of either increased or prolonged inhibition (Viereckel et al., 2013). In the study by Memmesheimer (2010), an increase in inhibition was found to generate a moderate decrease of ripple frequency. This may be a consequence of the particular implementation and should be systematically investigated in future work.

8. In vitro experiments found that action potentials generated during SPW/Rs are distinct from others (Bähner et al., 2011). They are abruptly generated from strongly hyperpolarized potentials, probably because of backpropagating action potentials from the axon. It is unlikely that single dendritic spikes directly lead to voltage deflections that are sufficiently strong to explain the findings (Ariav et al., 2003; Losonczy et al., 2008; Bähner et al., 2011; Müller et al., 2012; compare also Fig. 1). A possible indirect pathway may be through “privileged” dendrites (Thome et al., 2014). These are directly connected to the axon and can therefore trigger axonal action potentials that travel back to the soma. The dendrites are found in large fractions of hippocampal pyramidal neurons (e.g., in a majority of CA1 pyramidal neurons) and are able to generate strong dendritic spikes. In vivo experiments found that action potentials during SPW/Rs were not distinct (English et al., 2014). Nevertheless, dendritic spikes are known to be present and increased during SPW/Rs in vivo, they are likely to generate somatic action potentials because of their strength and robustness against inhibition, and they moderately change the onset characteristics of the triggered action potentials (Kamondi et al., 1998; Gasparini et al., 2004; Müller et al., 2012). A possible explanation for the in vivo observation would thus be that (similar fractions of) dendritic spike-triggered action potentials are generated outside of SPW/Rs (Harvey et al., 2009; Epsztein et al., 2010). However, attributable to the higher robustness of dendritic spikes to inhibition, we would generally expect a higher fraction of dendritic spike-triggered action potentials during SPW/Rs. An additional possible explanation is that dendritic spike-triggered and ordinary action potentials might appear similar under in vivo conditions since the increase in a dendritic spike-triggered action potential is initially comparably shallow, suggesting an unchanged somatic threshold (Gasparini et al., 2004), especially when superimposed by other input. The conflicting experimental findings for in vitro and in vivo SPW/Rs might indicate that the patterns are generated by different underlying mechanisms.

9. A functional connection between fast dendritic spikes and SPW/Rs may be suggested by the fact that SPW/Rs are most prominent in the hippocampal region CA1 (Buzsáki and Silva, 2012), the only brain structure where fast dendritic sodium spikes have been found to prevail (Ariav et al., 2003; Nevian et al., 2007; Müller et al., 2012; Major et al., 2013; Makara and Magee, 2013).

10. Theoretical studies have found that nonlinear dendrites can promote the propagation and the gating of information (represented by pulses of synchronous spikes) in noisy environments (Jahnke et al., 2012, 2014b). Furthermore, they increase the memory capacity and computational power of single neurons and neural networks (Mel, 1992, 1999; Rhodes, 2008; Cazé et al., 2013; Breuer et al., 2014). Therefore, it seems plausible that a neural system under evolutionary pressure such as the brain utilizes the present nonlinear dendrites in information processing. We may then expect that they are activated during SPW/Rs, when the level of overall input and synchrony they depend on is highest.

11. Activity during SPW/Rs does not always contain replay of recent experience, and replay can exceed SPW/Rs (Nadasdy et al., 1999; Lee and Wilson, 2002). We may expect the hippocampal regions have stored many memories and sequences from the past, and Memmesheimer (2010) suggested that completely random subnetworks may also generate SPW/R events, which are then not accompanied by replay of any experienced activity. Also from our model, we thus expect that SPW/Rs in the hippocampus often do not contain spike sequences that can be assigned to recent experience. In our model, replay can begin before prominent ripples are present; this happens, in particular, when the external input initiating it is unspecific or lacks synchrony. If we imagine the feedforward structure generating replay to be incorporated in a larger network that strongly contributes to the SPW/R, we expect that the overall random activity can fade away, while activity is still propagating along the more prominent feedforward structure before it also extinguishes, thus generating replay beyond the SPW/R event. Sufficiently strong propagation might also generate another, subsequent SPW/R (Davidson et al., 2009). Finally, we expect that localized or partial replay events may not always generate a full-blown, detectable SPW/R.

Recurrent connectivity may be expected to induce dendritic spiking

It is important to note that we may expect even the very sparse recurrent connectivity in CA1 to be sufficient to generate dendritic spiking during SPW/Rs. In the following, we show this by analyzing the strength of input that a pyramidal neuron receives during SPW/Rs from recurrent connectivity. We find that it may be comparable to the input a CA1 neuron receives from Schaffer collaterals. Since the latter has been experimentally directly shown to generate dendritic spikes (Kamondi et al., 1998), we conclude that the recurrent CA1 input may achieve this equally well.

Recurrent connectivity in both hippocampal regions CA3 and CA1 is sparse, but individual connections are strong. The estimates for the connection probability are ∼2% for CA3 neurons <200 μm apart (MacVicar and Dudek, 1980; Miles and Wong, 1986; Traub and Miles, 1991) and 1% for CA1 neurons with a distance of 200 μm (Deuchars and Thomson, 1996). The estimates are conservative because of their derivation from dual recordings in slices that lose part of connectivity during preparation. Anatomical data indicate a rather global recurrent connectivity for CA3 (Andersen et al., 2007; Cutsuridis et al., 2010). CA1 has localized coupling mediated by a prominent transversal and longitudinal local axonal plexus (Knowles and Schwartzkroin, 1981; Orman et al., 2008; Yang et al., 2014). Comparison with other brain areas may suggest an increase in connectivity with greater proximity (Holmgren et al., 2003). We first consider CA1 and assume that the connectivity is global and unstructured and that a possible coupling is present with a probability of 1%. The fraction of CA1 pyramidal neurons sending a spike within a window of 5 ms around the peak of a SPW is ∼5% (Csicsvari et al., 2000; we neglect SPW/R propagation and spatial incoherence), and the total number of pyramidal neurons within CA1 is ∼3 × 10⁵ (Andersen et al., 2007). So, approximately

recurrent inputs per neuron are generated around the peak of a SPW.

The estimated number of inputs can now be compared with a similarly estimated number of inputs from CA3 neurons. The connectivity from CA3 to CA1 pyramidal neurons is also sparse; it was estimated to 6% in regions where the considered CA3 neuron projects to (Sayer et al., 1990). Like for CA1 recurrent connections, we neglect the structure of this connectivity, keeping in mind that a CA3 neuron projects to a several-times-larger area in CA1 than a CA1 neuron (Brivanlou et al., 2004; Andersen et al., 2007; Cutsuridis et al., 2010; Yang et al., 2014). We also neglect SPW/R propagation and spatial incoherence. In a window of 5 ms around the peak of a SPW, 1% of the CA3 pyramidal neurons send a spike (Csicsvari et al., 2000), and the total number of pyramidal neurons within CA3 is ∼2 × 10⁵ (Andersen et al., 2007). This estimation yields a number of

inputs to each CA1 pyramidal neuron.

The input strength of CA3 to CA1 connections is several times smaller than the strength of CA1 recurrent inputs [mean somatic EPSP 0.1 mV for CA3→CA1 connections (Sayer et al., 1990) vs 0.7 mV for CA1→CA1 connections (Deuchars and Thomson, 1996)], which may compensate the more widespread connectivity. The complexity of the basal and apical dendrites (such as the number of branches) is similar (Andersen et al., 2007).

We thus conclude that the total excitatory input received by a basal dendritic branch within SPW/Rs attributable to recurrent CA1→CA1 connections may be comparable to the input an apical dendritic branch receives through CA3→CA1 Schaffer collaterals. Inputs from CA3 are known to generate dendritic spikes during SPW/Rs in the apical dendrites of CA1 neurons (Kamondi et al., 1998). Thus, the comparison indicates that we may expect inputs from recurrent connections to generate dendritic spikes in CA1 basal dendrites. Indeed a few synchronous recurrent inputs are sufficient to generate a dendritic spike [approximately six for the mean coupling strength measured by Deuchars and Thomson (1996)].

For CA3 recurrent connections, a similar argument with 2% recurrent connectivity yields 40 inputs per neuron around the peak of the SPW. The strength of individual inputs is 0.6–1.3 mV (Traub and Miles, 1991). Compared with the result for CA3 to CA1 connections that may have a similar spread (Andersen et al., 2007; Cutsuridis et al., 2010), stronger synaptic strength might compensate for a smaller number of inputs.

A more detailed estimate may account for the actual ranges of connectivity, possible local increases (Holmgren et al., 2003), and the neuron densities, to compute the numbers of inputs received in CA1 neurons, from CA1 and CA3. We will not undertake it here because of uncertainties in the parameters.

Proposed experiments

Our model suggests several ways to experimentally test its validity.

1. It predicts dendritic spikes during SPW/Rs and replay in the basal dendrites of the hippocampal region CA1 pyramidal neurons and in the basal and/or apical dendrites of hippocampal region CA3 pyramidal neurons. Dendritic spikes have already been found in the apical dendrites of CA1 neurons (Kamondi et al., 1998). Experiments, especially in basal dendrites, are technically demanding because of the dendrites' generally small diameter.

2. Our model predicts that dendritic spikes play a crucial role in the generation of somatic action potentials during SPW/Rs and replay. It would be important to reveal whether and how action potentials triggered by spikes in the relevant dendrites are distinguishable from other action potentials by their waveform in neurons within an active network. Their prevalence and thus their importance in background activity, SPW/Rs, and replay can then be analyzed (cf. Gasparini and Magee, 2006; Epsztein et al., 2010; Bähner et al., 2011; English et al., 2014; see above, The choice of the SPW/R model in our study).

3. If spiking in a neuron can be abolished, a combined experimental and modeling approach may identify the impact of dendritic spiking. For this, the output spikes in the native neuron and the input currents (or voltage traces) in the regime without spikes need to be measured. A model neuron receiving the input currents (or a spike generation model applied to the voltage traces) will reveal the expected somatic action potentials in the absence of dendritic spikes. Their analysis and comparison with spike trains of the native neuron will allow us to test whether the neuron still participates with similar precision and reliability in SPW/Rs and replay.

4. If fast dendritic spikes could be selectively reduced or abolished (e.g., because they depend on a combination of ion channels with different (sub)types, states, and properties; Gasparini and Magee, 2002; Ariav et al., 2003; Lorincz and Nusser, 2010), their effect on the network level may be directly investigated. Our model implies that a sufficiently strong network-wide reduction of dendritic spikes will impair SPW/Rs and replay.

Discussion

So far, most studies on hippocampal dynamics have investigated either learning and recall or mechanisms of emergent network phenomena. In the present study, we have proposed a unifying model for learning, replay, sharp-wave generation, and ripple generation. In our model, all four experimentally observed capabilities of hippocampal networks are intimately interrelated. They are enhanced or enabled by nonlinear dendritic interactions mediated by fast dendritic spikes. The replay can take place on underlying networks that are only weakly structured and sparse, therewith the nonlinear dendrites indirectly simplify learning.

We introduced learning during an exploration phase by incorporating a standard type of network plasticity; changes of synaptic strengths depend on the timing of presynaptic and postsynaptic somatic spikes. We have incorporated a learning rule that possesses a power-law dependence on the synaptic weight before modification, in agreement with experimental findings (Bi and Poo, 1998; Morrison et al., 2007). It leads to a biologically plausible, stable, unimodal synaptic weight distribution. Our work shows that despite the tendency of the synapses to converge to the same equilibrium value, and the persistence of non-negligible background weights, the network structures established during exploration are strong enough to generate replay. We emphasize that our networks do not organize into a single feedforward structure to enable replay and that they allow learning of multiple sequences.

Besides synaptic weight modification also, other forms of network plasticity have been experimentally found (Mozzachiodi and Byrne, 2010). Our network model suggest that especially the activity-dependent change of coupling between nonlinear dendritic branches and the soma (branch strength potentiation) may play a prominent role during learning of activity patterns in the hippocampus: when dendritic spike initiation occurs together with somatic action potentials, the strength of the dendritic spikes and their impact on the soma increase (Losonczy et al., 2008; Müller et al., 2012). This leads to strong dendritic spikes that support the reproduction of input–output relationships from exploration phases, and may thus be expected to enhance replay of activity during SPW/Rs based on dendritic spikes.

If supported by nonlinear dendrites, activity propagates along comparatively weakly enhanced, sparse, biologically plausible network structures. This generates a recall of the original sequence, which is noisy in the sense that the spike order is only roughly preserved and not every neuron of the sequence participates in every recall, in agreement with experimental findings (Wilson and McNaughton, 1994; Nadasdy et al., 1999; Lee and Wilson, 2002; Pastalkova et al., 2008; Davidson et al., 2009).

Our model suggests that replay in hippocampal region CA1 may proceed based on CA1 internal recurrent connectivity and does not need to be directly imposed by replay in CA3. This is consistent with anatomical findings (Deuchars and Thomson, 1996; Yang et al., 2014) and with experimental observations of replay in CA1 that is deafferented from CA3 (Nakashiba et al., 2009; Maier et al., 2011; see Results for a comprehensive discussion). It may support the different functionality of cells in CA3 and CA1 (Mizuseki et al., 2012). Interactions between mostly independent replay in different regions may be functionally relevant.

A common question is why replay is absent during exploration phases, whereas similar network structures generate replay during recall phases. This is usually explained by a strengthening of effective recurrent excitation by neuromodulators during recall (Hasselmo, 2006). In contrast to previous models, we do not have to assume such a strengthening. In our model, replay requires synchronous spiking, caused by the sensitivity of nonlinear dendrites to it. To evoke the common targeted recall starting from a salient location, a group of neurons with a subgroup encoding the location is activated to spike synchronously. For spontaneous, random recall, the overall spiking activity may be increased such that a high level of synchronous spiking is also present, which at times evokes spontaneous recall. Since sufficient synchronous spiking is not present during exploration, replay is not generated.

The learned network structures are stripe-like, meaning they do not possess segmented, distinct groups of neurons like synfire chains (cf. Abeles, 1982, 1991). Importantly, we nevertheless observe that the propagation of activity proceeds in synchronous pulses, and thus ripples are generated superimposed to an overall sharp wave increase of activity, as experimentally found. The occurrence of the pulses can be understood as follows: replay is initiated by a synchronous stimulation of a few neurons partially sensitive to places near the same position in a trained sequence. This increases synchronous input and thus dendritic spiking in postsynaptic excitatory neurons, especially in such postsynaptic neurons that receive stronger synaptic input from many of the stimulated neurons. Because of the previous learning phase, these are neurons that signal places in the near future relative to the original position within the trained sequence. The dendritic spikes promote somatic spikes or directly generate them, with high temporal precision after the experimentally measured delay time of about 5 ms (Ariav et al., 2003; Müller et al., 2012). Together with conventional inputs, this evokes a better synchronized, larger pulse of response spikes, in which neurons signaling the near future of the learned sequence are overrepresented. The pulse evokes a third one, with neurons farer in the future of the sequence being particularly prevalent, and so on. Our simulations show that the pulses do not broaden despite the lack of underlying group structure in the network; they stay narrow and precise, attributable to the high precision of dendritic spike-triggered somatic output spikes. In turn, the temporal structuring of propagating activity and the consequent input synchronization allow dendritic amplification of many inputs and thus promote the underlying replay.

Our model explains the experimentally found sharp wave in the hippocampus by an initial increase in the size of the synchronous pulses and a subsequent decrease by inhibitory feedback. The amplitude of the increase is determined by the broadness of the learned network structure, by the amount of recurrent excitation, and by the limiting action of accumulating overall recurrent inhibition in the network. The event terminates when the overall inhibition overcomes excitation.

We have recently shown that synchronous activity does not need to stay confined to a feedforward structure, as is essentially the case in the present study, but may also spread to the remaining neurons (Jahnke et al., 2014a). This supports replay by providing additional synchronous input to the neurons and the dendrites, thus making them more ready to generate spikes in response to the input that generates the replay.

Together, our model suggests that ripple oscillations are generated by propagating activity and in turn enhance it. This idea of ripples supporting replay does not necessarily require that either of them depends on nonlinear dendrites. We have recently shown that externally imposed high-frequency oscillations support propagation of synchrony along synfire chains in the presence of nonlinear dendrites (Jahnke et al., 2014b). Also, in the absence of nonlinear dendrites, high-frequency oscillations can have a beneficial effect, which is, however, equivalent to that of the time-averaged mean input.

Recent experiments found sequential spiking activity that reflects a track already during rest and sleep before experiencing it (Dragoi and Tonegawa, 2011; Azizi et al., 2013). Such sequences might depend on network structures that have previously been learned, e.g., according to learning as implemented in our model, or emerged because of self-organization processes (Cheng, 2013). In both cases, our results on replay and SPW/Rs remain valid, and additional learning during experiencing the track may strengthen the structures to increase the probability of their replay in subsequent sleep. Also in the framework of continuous attractor models of hippocampal networks (Tsodyks, 1999; Azizi et al., 2013), our model may contribute to understand this increase as well as the role of SPW/R dynamics.

Our study clears the path toward investigating the role of replay and SPW/Rs in learning and memory. An important direction of future research is to further investigate how replay and SPW/Rs act back to restructure hippocampal networks. In particular, it has been hypothesized that SPW/Rs may consolidate (Buzsáki, 1989; Nadasdy et al., 1999) or erase (Buzsáki, 2006; Mehta, 2007) memory content in the hippocampus (also compare our simulation results) and that they may lead to associative processes (Buzsáki, 2006; Andersen et al., 2007).

Another important question is how replay and SPW/Rs shape neocortical networks. In particular, future research will clarify whether and how highly noisy, scarce replay in conjunction with SPW/R activity and neocortical sleep spindles may imprint and consolidate memory content, as assumed by the two-stage memory hypothesis.