Identification of Functional Synaptic Plasticity from Spiking Activities Using Nonlinear Dynamical Modeling

Abstract

This paper presents a systems identification approach for studying the long-term synaptic plasticity using natural spiking activities. This approach consists of three modeling steps. First, a multi-input, single-output (MISO), nonlinear dynamical spiking neuron model is formulated to estimate and represent the synaptic strength in means of functional connectivity between input and output neurons. Second, this MISO model is extended to a nonstationary form to track the time-varying properties of the synaptic strength. Finally, a Volterra modeling method is used to extract the synaptic learning rule, e.g., spike-timing-dependent plasticity, for the explanation of the input-output nonstationarity as the consequence of the past input-output spiking patterns. This framework is developed to study the underlying mechanisms of learning and memory formation in behaving animals, and may serve as the computational basis for building the next-generation adaptive cortical prostheses.

1. Introduction

Long-term synaptic plasticity has long been postulated as the underlying mechanism of learning and memory formation (Hebb, 1949). This hypothesis is supported by diverse lines of evidence involving various experimental approaches and animal species (Alkon et al., 1991; Barnes, 1979; Berger, 1984; Lynch and Baudry, 1984; Martin et al., 1997; McHugh et al., 1996; McNaughton et al., 1986; Morris, 1989; Nakazawa et al., 2002; Tonegawa et al., 1996). Specific forms of activity-dependent synaptic plasticity such as long-term potentiation/depression (LTP/LTD) and spike-timing-dependent plasticity (STDP) have been revealed and intensively investigated (Bi and Poo, 1998; Bliss and Lomo, 1973; Ito, 1989; Levy and Steward, 1983; Markram et al., 1997; Caporale and Dan, 2008; Feldman, 2012). However, the exact nature and functional properties of neural plasticity during learning and memory formation are less known, especially in the case of the mammalian brain in behaving animals. In fact, for many researchers, whether learning and memory rely on LTP-like synaptic plasticity still remains controversial (Bramham, 2010; Stevens, 1998). The aim of this paper is to formulate a computational framework for the identification of long-term synaptic plasticity and the associated learning rule from natural ensemble spiking activities recorded in animals performing learning and memory tasks. It has the potential to provide direct evidence and new insights to this fundamental question of neuroscience.

Synaptic plasticity in behaving animals is difficult to study for a variety of reasons. First, brain regions underlying learning and memory, like the hippocampus, are massively parallel, point- process MIMO systems. In any given brain region, information is represented in the ensemble firing of populations of neurons (Deadwyler and Hampson, 1995; Freeman, 1999; Georgopoulos et al., 1986; Pouget et al., 2003; Salinas and Abbott, 1994). The input-output signals are stimulus- or behaviorally-driven (spontaneous) spike trains, as opposed to the artificially delivered electrical stimuli and the evoked responses typically used in the in vitro studies of synaptic plasticity (Anderson et al., 1971; Berger et al., 1988a, b). Extraction of the neuron-to-neuron connection strength, e.g., synaptic weight or EPSP, and further changes of such a strength from ensemble spiking activities, which is a necessary step for studying neural plasticity, poses serious modeling and computational challenges (Berger et al., 2010; Brown et al., 2004; Song and Berger, 2009). Second, the spike train-to-spike train transformation exhibits strong nonlinear dynamical properties due to the numerous underlying nonlinear biological mechanisms and processes (Berger et al., 1994; Johnston and Wu, 1995). These nonlinear dynamical properties make the quantification of neural plasticity, which is equivalent to the identification of the nonstationarity of the nonlinear dynamics (Song and Berger, 2009), even more difficult. Third, to study neural plasticity caused by learning and memory formation, the experimental paradigm should be designed in such a way that both the behavioral performance and the changes of neural representation are shaped by the animals’ learning experience instead of incrementally forced by the training paradigm. Fourth, reliable and stable unitary recordings of multiple neuron ensembles are required during the whole learning process (Buzsaki, 2004). The scope of this study is to build mathematical methodologies to address the first two computational challenges.

The proposed modeling strategy consists of three steps (Fig. 1). First, we formulate a nonlinear dynamical spiking neuron model to extract and represent the functional synaptic strength between the input and output neurons using spiking data. Model structure, parameter estimate method, and interpretations of the model variables are described. In this step, the model is stationary, i.e., it is not a function of time (Fig. 1, top). Second, we extend the model to a nonstationary nonlinear dynamical model to track the changes of the synaptic strength over time. In this step, the model becomes a function of time (Fig. 1, middle). Third, we utilize a Volterra modeling approach to identify the synaptic learning rule that can explain the changes of the synaptic strength as the consequence of the input-output spiking activities (Fig. 1, bottom). We elucidate the relationship between the Volterra kernel and the STDP learning rule, and further test the proposed modeling methods with simulations.

Figure 1.

Schematic diagram of the three modeling steps. X: input sequences; Y: output sequences; S: spiking neuron model; L: learning rule for S. In Step 1, S is not a function of time. In Steps 2 and 3, S varies with time. During learning, S evolves as the consequence of input and output activities following a learning rule. Colored boxes indicate the functions need to be identified in each modeling step.

2. Materials and Methods

2.1. Nonlinear dynamical spiking neuron model

First, a model structure is required for the extraction of the synaptic strength from the spiking activities of the input-output neurons. In such a model, the synaptic strength, as well as other parameters, should be estimated from the input-output spiking activities. This is a non-trivial task, since both inputs and output are point-process signals containing only the timings of the spikes and the synaptic strength thus need to be inferred instead of directly measured. We use the nonlinear dynamical spiking neuron model we previously developed for such a task (Song and Berger, 2009; Song et al., 2007, 2009a, b; Song et al., 2013).

In this approach, a spiking neuron is represented as a multi-input, single-output (MISO) model with a physiologically plausible structure (Fig. 2).

$$w=u(K,x)+a(H,y)+\epsilon \left(\sigma \right),$$ (1)

$$y=\{\begin{array}{cc}0\hfill & \text{when}w<\theta \hfill \\ 1\hfill & \text{when}w\ge \theta \hfill \end{array}\phantom{\}}$$ (2)

Figure 2.

Multi-input, single-output (MISO) nonlinear dynamical model of spiking neuron. The MISO model takes a physiologically plausible structure. All model variables are simultaneously estimated from the input-output spiking activities.

Variable x represents input spike trains; y represents the output spike train; w represents the pre-threshold membrane potential of the output neurons, that is expressed as the summation of the post-synaptic potential u caused by input spike trains, the output spike-triggered after-potential a, and a Gaussian white noise ε with standard deviation σ. A threshold, θ, determines the generation of the output spike and the associated feedback after-potential (a).

The feedforward transformation from x to u and the feedback transformation from y to a take the form of a second-order Volterra model K and a first-order Volterra model H, respectively.

$$u\left(t\right)=k_0+\sum _{n=1}^N\sum _{\tau =1}^{M_k}{k}_1^{\left(n\right)}\left(\tau \right)x_n(t-\tau )+\sum _{n=1}^N\sum _{{\tau }_1=0}^{M_k}\sum _{{\tau }_2=0}^{M_k}{k}_{2s}^{\left(n\right)}({\tau }_1,{\tau }_2)x_n(t-{\tau }_1)x_n(t-{\tau }_2)+\sum _{n_1=1}^N\sum _{n_2=1}^{n_1-1}\sum _{{\tau }_1=0}^{M_k}\sum _{{\tau }_2=0}^{M_k}{k}_{2x}^{(n_1,n_2)}({\tau }_1,{\tau }_2)x_{n_1}(t-{\tau }_2)x_{n_2}(t-{\tau }_2)+\dots$$ (3)

$$a\left(t\right)=\sum _{\tau =1}^{M_k}h\left(\tau \right)y(t-\tau )$$ (4)

The zeroth-order kernel, k₀ , is the value of u when there is no input. It determines the spontaneous firing rate of the output neuron. First-order kernels $k_1^{\left(n\right)}$ describe the linear relation between the n^th input x_n and u, as functions of the time intervals (τ) between the present time and the past time. Second-order self kernels $k_{2s}^{\left(n\right)}$ describe the nonlinear interactions between pairs of spikes within the n^th input x_n as they affect u. Second-order cross kernels $k_{2x}^{(n_1,n_2)}$ describe the nonlinear interactions between pairs of spikes from different inputs (x_n1 and x_n2 ) as they affect u. N is the number of inputs. Higher-order kernels are not included in this study for simplicity. h is a linear feedback kernel that transforms the output spike to the after-potential a. M_k and M_h are memory lengths of feedforward and feedback processes, respectively.

To reduce number of parameters and avoid overfitting, basis functions are utilized in model estimations (Song et al., 2013). Basis functions b can take the forms of Laguerre basis (Marmarelis, 1993; Marmarelis, 2004; Song et al., 2009c; Song et al., 2009d) or B-spine basis (de Boor, 1972). With input and output spike trains x and y convolved with b:

$$v_j^{\left(n\right)}\left(t\right)=\sum _{\tau =0}^{M_k}{b}_j\left(\tau \right)x_n(t-\tau ),$$ (5)

$$v_j^{\left(h\right)}\left(t\right)=\sum _{\tau =1}^{M_k}{b}_j\left(\tau \right)y(t-\tau )$$ (6)

Equations 3 and 4 are rewritten into:

$$u\left(t\right)=c_0+\sum _{n=1}^N\sum _{j=1}^L{c}_1^{\left(n\right)}\left(j\right)v_j^{\left(n\right)}\left(t\right)+\sum _{n=1}^N\sum _{j_1=1}^L\sum _{j_2=1}^{j_1}{c}_{2s}^{\left(n\right)}(j_1,j_2)v_{j_1}^{\left(n\right)}\left(t\right)v_{j_2}^{\left(n\right)}\left(t\right)+\sum _{n_1=1}^N\sum _{n_2=1}^{n_1-1}\sum _{j_1=1}^L\sum _{j_2=1}^L{c}_{2x}^{(n_1,n_2)}(j_1,j_2)v_{j_1}^{\left(n\right)}\left(t\right)v_{j_2}^{\left(n\right)}\left(t\right)+\dots$$ (7)

$$a\left(t\right)=\sum _{j=1}^L{c}_h\left(j\right)v_j^{\left(h\right)}\left(t\right)$$

$c_1^{\left(n\right)}$, $c_{2s}^{\left(n\right)}$, $c_{2x}^{(n_1,n_2)}$, and c_h are the model coefficients of $k_1^{\left(n\right)}$, $k_{2s}^{\left(n\right)}$, $k_{2x}^{(n_1,n_2)}$, and h , respectively. c₀ is equal to k₀. Given the kernels are smooth and continuous functions, the number of basis functions L can be much smaller than the memory length (M_k and M_h).

In Equation 7 and 8, v and vv can be calculated from the known x, y and b. The Volterra series essentially expresses the nonlinear relationship between u and x into a linear relationship between u and [v, vv]. The joint effect of the threshold θ and the Gaussian noise ε is equivalent to a probit link function that maps the value of u + a into the probability of y is equal to 1. The whole model thus can be expressed as a generalized linear model with the nonlinearity structured in the Volterra series. Therefore, this MISO model can be termed as a generalized Volterra model (GVM) (Song et al., 2009a, b). As a special case, the first-order GVM is equivalent to the commonly used generalized linear models (Paninski et al., 2004; Okatan et al., 2005; Pillow et al., 2005; Truccolo et al., 2005; Eldawlatly et al., 2009; Chen et al., 2011; Zhao et al., 2012).

Due to the point-process nature of the input and output signals, all model variables are dimensionless. In addition, the values of k, h, u, a, w, ε, σ, and θ can be scaled; θ and k₀ can be translated, without influencing the probability of generating output spikes. So in practice, k₀, k₁, k_2s, k_2x, and h are first estimated with a unitary σ and a zero-valued θ using the Matlab® glmfit function, and then normalized with the absolute value of k₀. In the final format, σ is equal to1/|k₀|; θ is equal to zero; the baseline value of w (i.e., k₀) is -1 (Song et al., 2007, 2009a). The Laguerre parameter controlling the asymptotic decaying rate of the basis functions, and the total number of basis functions L, are optimized with respect to the log-likelihood function (Song et al., 2013).

The Volterra kernels quantitatively describe the input-output nonlinear dynamics of the neuron. A more intuitive representation is the single-pulse and paired-pulse response functions (r₁ and r₂) derived from the kernels.

$$r_1^{\left(n\right)}\left(\tau \right)=k_1^{\left(n\right)}\left(\tau \right)+k_{2s}^{\left(n\right)}(\tau ,\tau )$$ (9)

$$r_{2s}^{\left(n\right)}({\tau }_1,{\tau }_2)=2k_{2s}^{\left(n\right)}({\tau }_1,{\tau }_2)$$ (10)

$$r_{2x}^{(n_1,n_2)}({\tau }_1,{\tau }_2)=k_{2x}^{(n_1,n_2)}({\tau }_1,{\tau }_2)$$ (11)

$r_1^{\left(n\right)}$ is essentially the postsynaptic potential (PSP) elicited by a single spike from the n^th input neuron; $r_2^{\left(n\right)}$ describes the joint nonlinear effect of pairs of spikes from the n^th input neuron in addition to the summation of their first order responses, i.e., $r_1^{\left(n\right)}\left({\tau }_1\right)+r_1^{\left(n\right)}\left({\tau }_2\right)$. $r_{2x}^{(n_1,n_2)}({\tau }_1,{\tau }_2)$ represents the joint nonlinear effect of pairs of spikes with one spike from neuron n₁ and one spike from neuron n₂ (Fig. 3)

Figure 3.

Physiological interpretation of the MISO spiking neuron model variables. k₀ (r₀) is resting membrane potential. r₁ is the postsynaptic potential. r₂ is the paired-pulse facilitation/depression function. h is the output spike-triggered after-potential. u is the synaptic potential. σ is the standard deviation of the Gaussian noise.

In this formulation, the ensemble neuronal input-output properties is represented by the GVM coefficients (S = [k, h]). All coefficients are simultaneously estimated from the input and output spike trains (x and y). Specifically, single-pulse response function r₁ can be interpreted as the PSP, which is typically used as the measure of synaptic strength in experimental studies on long-term synaptic plasticity.

2.2. Nonstationary nonlinear dynamical spiking neuron model

The nonlinear dynamical spiking neuron model has provided a quantitative way to infer synaptic strength from spiking input-output activities. In order to study the long-term synaptic plasticity, we extend the GVM to be time-varying for the tracking of system nonstationarity. In this case, the GVM coefficients are estimated recursively from the input-output spikes and the changes of the nonlinear dynamics are tracked by the temporal evolution of the kernel functions (Fig.1, middle).

We develop the nonstationary model by combining the GVM with the point-process adaptive filtering method (Chan et al., 2008; Eden et al., 2004). In this approach, GVM coefficients (c) are taken as state variables while the input-output spikes are taken as observable variables. Using adaptive filtering methods, state variables are recursively updated as the observable variables unfold in time. The underlying change of system input-output properties then is represented by the time-varying GVM (S(t) = [k(t), h(t)]) reconstructed with the time-varying coefficients c(t).

Specifically, the probability of observing an output spike at time t, i.e., P(t), is predicted by the GVM at time t-1 based on the inputs up to t and output before t. Secondly, the difference between P(t) and the new observation of output y(t) is used to update the GVM coefficients. Using the stochastic state point process filtering algorithm, coefficient vector C(t) and its covariance matrix W(t) are both updated iteratively at each time step t as in:

$$P\left(t\right)=0.5-0.5\cdot \mathrm{eff}\left(\frac{\theta -u\left(t\right)-a\left(t\right)}{\sqrt{2}\sigma }\right)$$ (12)

$$c\left(t\right)=c(t-1)+W\left(t\right)\left[{\left(\frac{\partial \log P\left(t\right)}{\partial C\left(t\right)}\right)}^T(y\left(t\right)-P\left(t\right))\right]$$ (13)

$$W{\left(t\right)}^{-1}={[W(t-1)+Q]}^{-1}+\left[{\left(\frac{\partial \log P\left(t\right)}{\partial c\left(t\right)}\right)}^{T}P\left(t\right)\left(\frac{\partial \log P\left(t\right)}{\partial c\left(t\right)}\right)-(y\left(t\right)-P\left(t\right))\frac{{\partial }^2\log P\left(t\right)}{\partial c\left(t\right)\partial c{\left(t\right)}^{\prime }}\right]$$ (14)

Q is the coefficient noise covariance matrix. erf is the Gauss error function. In this method, W acts as the adaptive “learning rate” that allows reliable and rapid tracking of the model coefficients.

2.3. Identification of the synaptic learning rule

The nonstationary GVM above describes how the ensemble neuronal input-output properties evolve overtime. In the third step, we aim to explain how the nonstationarity happens by identifying the underlying synaptic learning rule (Fig. 1, bottom).

Given the estimated nonstationary model S(t), the change at time t can be calculated as ΔS(t) = S(t) - S(t-1), i.e., Δk(t) = k(t) - k(t-1), Δh(t) = h(t) - h(t-1), and ΔS(t) = [Δk(t), Δh(t)]. Given that ΔS(t) is caused by the preceding input and output activities (z = [x, y]), the sought ensemble synaptic learning rule L is essentially the causal relationship between the spatio-temporal pattern z and ΔS(t) (Fig. 4). ΔS(t) can be expressed as a Volterra functional power series of z in a finite memory window (M_L) as:

$$\Sigma S\left(t\right)=L_0+\sum _{n=1}^{N+1}\sum _{\tau =0}^{M_L}{L}_1^{\left(n\right)}\left(\tau \right)z_n(t-\tau )+\sum _{n_1=1}^{N+1}\sum _{n_2=1}^{n_1}\sum _{{\tau }_1=0}^{M_L}\sum _{{\tau }_2=0}^{M_L}{L}_2^{(n_1,n_2)}({\tau }_1,{\tau }_2)z_{n_1}(t-{\tau }_1)z_{n_2}(t-{\tau }_2)+\sum _{n_1=1}^{N+1}\sum _{n_2=1}^{n_1}\sum _{n_3=1}^{n_2}\sum _{{\tau }_1=0}^{M_L}\sum _{{\tau }_2=0}^{M_L}\sum _{{\tau }_3=0}^{M_L}{L}_3^{(n_1,n_2,n_3)}({\tau }_1,{\tau }_2,{\tau }_3)z_{n_1}(t-{\tau }_1)z_{n_2}(t-{\tau }_2)z_{n_3}(t-{\tau }_3)+\cdots$$ (15)

Figure 4.

Schematic diagram showing the learning rule (L) from the ensemble spatio-temporal pattern of input/output spike trains (z: [x, y]) to the changes of the MISO GVM (Δk₁). For simplicity, only two inputs and one first-order kernel (k₁ between input x₁ and the output y) are shown.

In this formulation, L₀ is the zeroth-order learning rule describing the input/output activity-independent drift of the system; L₁ is the first-order learning rule describing the linear relation between the changes of the GVM and the input or output activities; L₂ is the second-order learning rule describing how the pair-wise nonlinear interactions between input/output spikes change the GVM; L₃ is the third-order learning rule describing how the triplet-wise nonlinear interactions between input/output spikes change the MIMO model (Note that there are redundancies in L₂ and L₃ when n₁, n₂ and n₃ contain same inputs. Equation 15 is used nonetheless for its simplicity). Higher-order learning rules are not included for simplicity.

Since ΔS(t) has been estimated, x and y are known, Equation 15 is essentially a linear model and the learning rule L thus can be estimated with the standard least-squares method. In practice, basis functions and penalized likelihood estimation methods can be utilized to reduce the total number of coefficients and select the significant terms in Equation 15. Compared with the existing learning rules such as (a) the classical Bienenstock-Cooper-Munro (BCM) model of synaptic modification and (b) the STDP, this formulation provides a general form of ensemble learning rule defining how the changes of GVM are determined by the spatio-temporal patterns of the input-output spikes. Linear and nonlinear interactions between multiple input/output spikes with different inter-spike intervals are explicitly included in this formulation (Fig. 4).

3. Results

3.1. Nonlinear dynamical model of the hippocampal CA3-CA1

We have applied the nonlinear dynamical modeling to the hippocampal CA3 to CA1 in rodents and nonhuman primates (Berger et al., 2011; Berger et al., 2012; Hampson et al., 2012b; Hampson et al., 2012c; Hampson et al., 2013; Song et al., 2007, 2009a), and the prefrontal cortex layer 2/3 to layer 5 in nonhuman primates (Hampson et al., 2012a; Song et al., 2012). In these applications, the input-output spike trains are recorded from well-trained animals, where the input and output signals as well as the input-output transformations are stabilized, and thus sufficiently described by stationary GVMs. Figure 5 shows a GVM of a hippocampal CA1 neuron estimated from a rodent performing a delayed-nonmatch-to-sample (DNMS) task. Results show that this neuron receives inputs from 6 out of 24 CA3 neurons. The single-pulse and paired-pulse response functions (r₁ and r₂) are the PSPs and paired-pulse facilitation/depression functions inferred from the CA3-CA1 spiking activities. The zeroth-order kernel is -1. The threshold is 0. The standard deviation of the Gaussian noise is estimated to be 0.418. This MISO model can accurately predict the output spike train based on the input spike trains (Fig.5, bottom-middle) as verified with the out-of-sample Kolmogorov-Smirnov test based on the time-rescaling theorem (Fig. 5, bottom-right). In this stationary model, all model variables are time-invariant.

Figure 5.

A stationary MISO GVM of a hippocampal CA1 neuron. r₁ are the single-pulse response functions. r₂ are the paired-pulse response functions for the same input neuron. k_2x are cross kernels for pairs of neurons. h is the feedback kernel. Output spike train is predicted with the model and the input spike trains. The model is validated with a out-of-sample, Kolmogorov- Smirnov (KS) test based on the time-rescaling theorem. In the KS plot, dashed lines are the 95% confidence bounds.

3.2. Simulation studies on nonstationary nonlinear dynamical model

We have tested intensively the nonstationary nonlinear dynamical modeling algorithm with synthetic input-output spike train data obtained through simulations. In all simulations, the inputs are Poisson random spike trains with mean firing rates range from 2 Hz to 6 Hz.

3.2.1. Step changes

First, we simulate a second-order, 2-input, single-output spiking neuron model with step changes. The total simulation length is 8000 s. At 4000 s, the first-order kernel and the second-order self kernel of the first input (i.e., $k_1^{\left(1\right)}$ and $k_{2s}^{\left(1\right)}$) decrease the amplitudes by half with the same waveforms; the first-order kernel and the second-order self kernel of the second input (i.e., $k_1^{\left(2\right)}$ and $k_{2s}^{\left(2\right)}$) double the amplitudes with the same waveforms. Other kernels (i.e., k₀, $k_{2x}^{(1,2)}$, and h) remain constant during the whole simulation. Using the simulated input-output spiking data, we apply the nonstationary nonlinear dynamical modeling algorithm to track the changes of the kernels. Results show that all kernels as well as the changes of the kernels can be recovered accurately from the simulated input-output spike trains (Fig. 6). In this nonstationary model, model variables become time-varying.

Figure 6.

Tracking a second-order, 2-input, single-output system with step changes. First-order kernels (k₁) and second-order self-kernels (k_2s) have step changes at 4000 s. Zeroth-order kernel (k₀), second-order cross-kernels (k_2x) and feedback kernel (h) remain constant. Memory length is 500 ms for k₁, k_2s, and k_2x, 1000 ms for h. The amplitude of kernel is color coded. Only diagonal values of second-order kernels are plotted for simplicity. A: actual kernels; E: estimated kernels.

3.2.2. LTP and LTD-like changes

We further test the capability of the nonstationary modeling algorithm to track biologically plausible forms of synaptic plasticity. A first-order, 2-input system with concurrent LTP- and LTD-like changes at different inputs is simulated (Fig. 7). The total simulation length is 9000 s. At 3000 s, the kernels of the first input and the second input have LTP-like change (i.e., a near instantaneous increase followed by an exponential decay to a potentiated level double the initial amplitude) and LTD-like change (i.e., an exponential decay to half of the initial amplitude), respectively. Results show that the estimated kernels (red) track faithfully the actual kernels (black) during the whole simulation, given the LTP-like change being much more challenging than the step change.

Figure 7.

Tracking a first-order, 2-input system with LTP- and LTD-like changes. Black line: peak amplitudes of the actual kernels; red dots: peak amplitudes of the estimated kernels. Interval between successive estimation points plotted is 20 s.

3.2.3. Input-independent changes

In our formulation of the nonstationary model, the zeroth-order time-varying kernel k₀(t) (i.e., c₀(t)) in Equation 13) describes the input-independent nonstationarity, i.e., changes of the output firing probability caused by latent factors other than the observed input spike trains. To test whether the algorithm can gracefully handle this type of nonstationarity, we modify the step-change simulation in Section 3.2.1 by varying the input-independent, baseline firing probability following a linear chirp (Fig. 8, top) and a random noise sequence (Fig. 8, bottom). Results show that the estimated baselines (red) rapidly converge to the actual baselines (black) in both simulations without interfering the estimation of higher-order kernels, which represent the input-dependent nonstationarities.

Figure 8.

Tracking the input-independent baseline k₀ (c₀). Top panel: k₀ is a linear chirp from 0 Hz at 0 s to 0.0125 Hz at 2500 s. Bottom panel: k₀ is a random noise sequence with mean at -3 and low pass filtered at 0.05Hz.

3.2.4. Larger number of inputs

To validate the capability of the algorithm to tack simultaneous gradual changes with larger number of inputs, we design a first-order, 8-input system with sigmoidal changes. Among the eight inputs, three inputs increase amplitudes following a sigmoidal curve; three inputs decrease amplitudes following a sigmoidal curve; two inputs remain constant. The output mean firing rates before and after the sigmoidal changes are 3.677 Hz and 5.112 Hz, respectively. The total simulation length is 9000 s. Results show that all eight estimated kernels converge to the actual kernels during the simulation (Fig. 9).

Figure 9.

Tracking a first-order, 8-input system with sigmoidal changes. Left column: actual and estimated kernels across simulation time. The memory length is 200 ms. Right column: peak amplitudes of actual kernels (black) and estimated kernel (red for positive kernels and blue for negative kernels) across simulation time with a 1 s resolution.

3.3. Relationship between Volterra kernels and STDP

Equation 15 provides a general representation of the ensemble learning rule underlying the changes of the neuronal input-output function. It utilizes a Volterra power series to describe how the GVM variables change as the consequence of the interactions between the input-output spikes. Before testing the identification method with simulation, it is instructive to elucidate the relation between the Volterra kernels and the most accepted synaptic learning rule, i.e., STDP (Fig. 10).

Figure 10.

Relationship between the STDP function, the induction function (I), and the second- order cross kernel (k_2x) of input (x) and output (y). Left column: STDP function. Center middle: the integral of the STDP function in k_2x. Right column: the induction function I and its representation in k_2x. The integral of I describes the STDP dynamics. The STDP function determines the steady-state change ( Δk̄) of the synaptic weight k. Center top: the k_2x representation of the STDP function and the induction function. This k_2x is calculated as the element-wise product of the k_2x of STDP function and the k_2x of I. Note that I is not plotted in scale for better visualization. White arrows indicate the directions of the STDP and induction functions in the cross kernel.

According to the STDP rule, the change of synaptic weight between a presynaptic neuron and a postsynaptic neuron is determined by the timings of the presynaptic and postsynaptic spiking activities (τ_x and τ_y). If a presynaptic (i.e., input) spike precedes a postsynaptic spike within a short time window (i.e., τ_x > τ_y), the synaptic weight will be enhanced; if the opposite happens (i.e., τ_x < τ_y), the synaptic weight will be reduced. The relationship between the amount of synaptic weight change and the spike timings, i.e., the STDP function, can be described with two exponential curves (Fig. 10, left). Since in this STDP expression the synaptic weight change depends only on the pair-wise interaction between a presynaptic spike and a postsynaptic spike, the Volterra expression of the learning rule contains only the second-order cross term between x and y. In the case of a single-input system, Equation 15 is reduced to:

$$\Delta S\left(t\right)=\sum _{{\tau }_x=0}^{M_L}\sum _{{\tau }_y=0}^{M_L}{L}_2^{(x,y)}({\tau }_x,{\tau }_y)x(t-{\tau }_x)y(t-{\tau }_y)$$ (16)

In $L_2^{(x,y)}$, The STDP function describes only the steady-state level of synaptic weight change. Another factor contributes to $L_2^{(x,y)}$ is the induction dynamics of the STDP, i.e., the synaptic weight does not change instantaneously following a delta function; instead, it relatively slowly builds up and then stabilizes following a smooth induction function I (Fig. 10, right). The STDP function and the induction function I jointly determine the shape of $L_2^{(x,y)}$ (Fig. 10, middle-top). In other words, this form of cross kernel between x and y is the Volterra representation of the STDP and induction functions.

3.4. Simulations studies on learning rule identification

We further test the learning rule identification method with simulated input-output spiking data. A first-order, single-input, single-output spiking neuron model is built with the GVM structure (Fig. 11). In this model, the first-order feedforward kernel k₁ has a typical EPSP shape determined by two exponentials with the time constants at 30 ms and 150 ms, controlling the onset and decay rates, respectively. The peak amplitude is 0.3. The feedback kernel has a negative exponential shape to model the refractory period of spike generation. The time constant and peak amplitude are 10 ms and -1. Noise standard deviation σ is 0.4. During the simulation, input spike train x is fed into the model to generate output spike train y. The synaptic strength between the input and output neurons, i.e., k₁, is changed following the standard STDP rule and the induction function. In this simulation, the shape of k₁ remains the same; only the amplitude is changed. For convenience, we use k₁ to represent both the peak amplitude and the whole kernel function. The left (LTD) half of the STDP function is a single exponential with time constant and peak amplitude at 33.7 ms and -0.018. The right (LTP) half of the STDP function is a single exponential with time constant and peak amplitude at 16.8 ms and 0.032. Both sides share the same induction function determined by two exponentials with time constants at 900 ms and 4500 ms, for onset and decay rates, respectively. Without loss of generality, the integral of the induction function is set at 1. To prevent the intrinsic instability of the simulated system, we adjust the input patterns based on the level of k₁. When k₁ crosses below a threshold, the input x changes from a 5 Hz Poisson random train to a 50 Hz burst to cause more potentiation and increase of k₁; when k₁ is above a threshold, x is shifted to spike within 10 ms after output spikes to cause more depression and decrease of k₁. This simple method ensures k₁ to fluctuate in a stable manner. The total length of simulation is 200 s.

Figure 11.

Simulation of a first-order, single-input, single-output spiking neuron model with STDP learning rule. During the simulation, presynaptic spikes x and postsynaptic spikes y change the feedforward kernel k₁ following the STDP and the induction functions. STDP function determines the steady-state change ; The induction function I defines the STDP Δk̄₁ dynamics.

Using the nonstationary GVM method, we first track the changes of the synaptic strength, i.e., k₁(t), and compare them with the actual values. Results show that k₁(t) can be accurately recovered from the simulated input-output spike trains (Fig. 12, top).

Figure 12.

Identification of STDP and induction functions from spiking input-output data. A: peak amplitudes of the first-order feedforward kernel k₁. B: STDP functions. C: induction functions. In all plots, back: actual; red: estimated.

Given k₁(t), we further estimate the learning rule, i.e., the second-order cross kernel $L_2^{(x,y)}$. In order to reduce the number of open parameters, we expand the kernel with three sets of Laguerre basis functions as:

$$\Delta k_1\left(t\right)=\sum _{j_A=1}^{L_A}\sum _{j_{\psi }=1}^{L_{\psi }}{c}_{xy}(j_A,j_{\psi })v_{xy}^{j_A}\left(t\right)v_{xy}^{j_{\psi }}\left(t\right)+\sum _{j_A=1}^{L_A}\sum _{j_{\psi }=1}^{L_{\psi }}{c}_{xy}(j_A,j_{\psi })v_{yx}^{j_A}\left(t\right)v_{yx}^{j_{\psi }}\left(t\right),$$ (17)

where

$$v_{xy}^{J_A}\left(t\right)=\sum _{{\tau }_x=1}^{M_{\psi }}{b}_{xy}^{j_A}({\tau }_x-{\tau }_y)x(t-{\tau }_x),$$ (18)

$$v_{xy}^{J_{\psi }}\left(t\right)=\sum _{{\tau }_y=0}^{{\tau }_x-1}{b}_{xy}^{j_{\psi }}\left({\tau }_y\right)y(t-{\tau }_y),$$ (19)

$$v_{yx}^{J_A}\left(t\right)=\sum _{{\tau }_y=1}^{M_{\psi }}{b}_{yx}^{j_A}({\tau }_y-{\tau }_x)y(t-{\tau }_y),$$ (20)

$$v_{yx}^{J_{\psi }}\left(t\right)=\sum _{{\tau }_x=0}^{{\tau }_y-1}{b}_{yx}^{j_{\psi }}\left({\tau }_y\right)x(t-{\tau }_x).$$ (21)

In the equations above, c are the sought learning rule coefficients. They are split into c_xy and c_yx to represent the two halves of the cross kernel for x preceding y and y preceding x, respectively. Subscript A represents the STDP amplitude. Subscript ψ represents the STDP induction. Since all v can be calculated with the predefined basis functions, and the known x and y, Equation 17 is essentially linear and c can be estimated with a least-squares method. With the estimated coefficients, $L_2^{(x,y)}$ can be reconstructed as:

$$L_2^{(x,y)}({\tau }_x,{\tau }_y)=\sum _{j_A=1}^{L_A}\sum _{j_{\psi }=1}^{L_{\psi }}{c}_{xy}(j_A,j_{\psi })b_{xy}^{j_A}({\tau }_x-{\tau }_y)b_{xy}^{j_{\psi }}\left({\tau }_y\right)+\sum _{j_A=1}^{L_A}\sum _{j_{\psi }=1}^{L_{\psi }}{c}_{yx}(j_A,j_{\psi })b_{yx}^{j_A}({\tau }_y-{\tau }_x)b_{yx}^{j_{\psi }}\left({\tau }_x\right)$$ (22)

$L_2^{(x,y)}$ constitutes a general representation of the synaptic learning due to the fact that it does not require the independence of the STDP function and the induction function. In this study, however, since these two functions are assumed to be independent, $L_2^{(x,y)}$ is equal to the outer product of the two functions. The left (LTD) side of the STDP function can be obtained by integrating the vertical vectors in the upper half of $L_2^{(x,y)}$ along its diagonal line; similarly, the right (LTP) side of the STDP function can be obtained by integrating the horizontal vectors in the lower half of $L_2^{(x,y)}$ along its diagonal line. The induction function for LTD can be obtained by integrating the diagonal vectors in the upper half of $L_2^{(x,y)}$ along the vertical axis; similarly, the induction function for LTP can be obtained by integrating the diagonal vectors in the lower half of $L_2^{(x,y)}$ along the horizontal axis. The induction function is then normalized to have a unitary integral. The STDP functions are scaled accordingly.

Figure 12 shows the final identification results using the same 200 s input-output spiking data. It is evident that both the STDP function and the induction function are faithfully recovered with this method (Fig. 12B, C).

4. Discussion

This paper describes a systems identification approach for studying the STDP-like synaptic learning rule using spiking activities in learning animals. In this formulation, synaptic strength is represented as input-output (linear or nonlinear) dynamics between neurons; long-term synaptic plasticity is defined as a form of nonstationarity of such input-output dynamics; synaptic learning rule is essentially the function governing the formation of the long-term synaptic plasticity based on the input-output spiking patterns. As a special case, STDP is equivalent to a second-order learning rule describing the pair-wise interactions between single input spikes and single output spikes. Using simulated input-output data, we have shown that this modeling problem can be solved with a series of systems identification methods; it is possible to infer STDP-like synaptic learning rule purely from the input-output spiking activities.

The MISO model and the multi-input, multi-output (MIMO) model, which consists of a series of MISO models, have been used intensively as a tool to identify the neuronal functional connectivity (Song and Berger, 2009; Song et al., 2013), e.g., hippocampal CA3-CA1, and build cortical prostheses for restoring cognitive functions, e.g., hippocampal memory prosthesis (Berger et al., 2011; Berger et al., 2012; Deadwyler et al., 2013; Hampson et al., 2012b; Hampson et al., 2012c; Hampson et al., 2013). Previous results have shown that the MIMO model can predict accurately the output (e.g., hippocampal CA1) spike trains based on the ongoing input (e.g., hippocampal CA3) spike trains. Electrical stimulation with the spatio-temporal patterns of the predicted output spike trains can restore or even enhance the memory function during a spatial delayed nonmatch-to-sample task (DNMS) in rodents. Since all these previous models are stationary, their success must be due to the fact that these applications only involve learned behaviors in well-trained animals, where the behavioral performance and the spatio-temporal pattern representations of behaviors (or memory events) are all stabilized, and as a consequence, the MIMO input-output transformations can be sufficiently described by stationary MIMO models. The resulting hippocampal prostheses essentially replicate the input-output properties of the hippocampal CA3-CA1 after learning and memory formation. To build hippocampal memory prostheses capable of learning and memory formation in a self-organizing manner, on the other hand, it is required to identify and further mimic the nonstationary behaviors and the underlying learning rules of the MIMO input-output properties. The identification methods described in this paper provide a computational framework for studying such nonstationary properties and the identified functional learning rule may potentially be used as the computational basis for developing the next-generation, adaptive hippocampal memory prostheses.

Our learning rule identification algorithm consists of two steps. The first step relies on accurate estimation of the kernel changes. The second step is based on the assumption that the synaptic learning rule is STDP-like, i.e., the kernel changes depend on the interactions between presynaptic and post-synaptic spikes. We have shown that the nonstationary modeling algorithm can (a) handle various forms of kernel changes with larger number (e.g., eight) of inputs, and (b) differentiate between input-dependent changes and input-independent changes. In addition, given the facts that (a) existing STDP-like learning rules involve only a single input and a single output as in our simulation study on the learning rule identification method (Section 3.4), and (b) STDP-like learning rules do not take arbitrary forms (Caporale and Dan, 2008; Feldman, 2012), i.e., they are all continuous functions with stereotypical shapes that can be easily handled by a Laguerre-Volterra model (e.g., the standard STDP function involves only a single term in the Volterra formulation and two Laguerre basis functions), these two steps of the algorithm can naturally work together to achieve reliable estimation of the STDP-like learning rules, as shown in our simulation results.

Naturally, our next step will be applying these methods to experimental data, i.e., input-output spike trains recorded from animals performing a learning task. For example, we can record from the rat hippocampal CA3 and CA1, a well-defined pair of input-output regions, during the training of the memory-dependent DNMS task. Different from our previous studies on the hippocampal CA3-CA1, the new experiments may focus on the learning (training) phase of the behavioral task where the animal's performance improves over time. Preliminary results have shown that the spatio-temporal pattern representations of memory events (i.e., functional cell types) do evolve over time during the training of DNMS task (Goonawardena et al., 2010). The CA3 and CA1 spike trains simultaneously recorded during the training sessions will likely be the most relevant experimental data for the proposed computational studies.

In this study, we have tested the learning rule identification algorithm with a simple form of STDP. With the experimental data, we may use the full Volterra expression (Equation 15), as opposed to the reduced second-order cross kernel expression of STDP in this study, to identify the ensemble synaptic learning rule. Previous studies have shown the existence of single spike rule (Kempter et al., 1999; van Rossum et al., 2000), triplet rule, and quadruplet rule of STDP (Pfister and Gerstner, 2006; Wang et al., 2005). These learning rules, together with their induction functions, are equivalent to the first-order, third-order, and fourth-order learning rule kernels of the Volterra expression, respectively. A formal approach of identifying the learning rule from experimental data would be including all possible terms (e.g., zeroth-order, first-order, second-order, third-order, and even fourth-order terms) into the model and then using statistical model estimation methods (e.g., regularized estimation) to characterize and select the significant terms (Song et al., 2013). The resulting ensemble learning rule should provide a more thorough description of the long-term synaptic plasticity.

It should be noted that a spiking neuron with the standard paired spike STDP rule is intrinsically unstable since the STDP rule is unbounded, i.e., there is no mechanism to saturate the synaptic weight or prevent it from being negative. In this study, we solve this problem with a rather empirical method: the input patterns, rather than the neuronal input-output properties, are adjusted to keep the synaptic weight fluctuating within a certain range. The main reason behind this strategy is to use the standard STDP function in the simulation and show how it is related to the Volterra kernels. One way to implement a stable model is to add upper and lower bounds to the synaptic weight, so that the synaptic weight is nonnegative and the input will not drive the synaptic weight to very large values. The caveat of this method is that the synaptic weight tends to stay at the maximal or minimal values after a certain length of running time and stops showing plasticity. Another way is to make the changes of synaptic weight depend on not only the input-output spike timing as in the standard STDP learning rule, but also the current value of the synaptic weight (van Rossum et al., 2000): a higher value of synaptic weight will bias the system to more depression and a lower value will cause more potentiation. This method will guarantee the synaptic weight to fluctuate, although, at the same time, an ever changing synaptic weight may make it difficult to form long-term memories. The strategy used by real neurons to maintain stability still remains unclear and poses an interesting question to the exact nature of synaptic learning rule. Performing and extending the computational studies outlined in this paper on experimental data recorded during learning and memory formation seems to be a sensible approach to answer this question.

Highlights.

1. Formulation of a nonstationary nonlinear dynamical modeling approach for the identification of long-term synaptic plasticity using natural spiking activities

2. Elucidation of the relationship between Volterra kernels and spike-timing-dependent plasticity functions

3. Test of nonstationary modeling and learning rule identification methods with simulated spiking input-output data