Low-dimensional spike rate models derived from networks of adaptive integrate-and-fire neurons: Comparison and implementation

Abstract

The spiking activity of single neurons can be well described by a nonlinear integrate-and-fire model that includes somatic adaptation. When exposed to fluctuating inputs sparsely coupled populations of these model neurons exhibit stochastic collective dynamics that can be effectively characterized using the Fokker-Planck equation. This approach, however, leads to a model with an infinite-dimensional state space and non-standard boundary conditions. Here we derive from that description four simple models for the spike rate dynamics in terms of low-dimensional ordinary differential equations using two different reduction techniques: one uses the spectral decomposition of the Fokker-Planck operator, the other is based on a cascade of two linear filters and a nonlinearity, which are determined from the Fokker-Planck equation and semi-analytically approximated. We evaluate the reduced models for a wide range of biologically plausible input statistics and find that both approximation approaches lead to spike rate models that accurately reproduce the spiking behavior of the underlying adaptive integrate-and-fire population. Particularly the cascade-based models are overall most accurate and robust, especially in the sensitive region of rapidly changing input. For the mean-driven regime, when input fluctuations are not too strong and fast, however, the best performing model is based on the spectral decomposition. The low-dimensional models also well reproduce stable oscillatory spike rate dynamics that are generated either by recurrent synaptic excitation and neuronal adaptation or through delayed inhibitory synaptic feedback. The computational demands of the reduced models are very low but the implementation complexity differs between the different model variants. Therefore we have made available implementations that allow to numerically integrate the low-dimensional spike rate models as well as the Fokker-Planck partial differential equation in efficient ways for arbitrary model parametrizations as open source software. The derived spike rate descriptions retain a direct link to the properties of single neurons, allow for convenient mathematical analyses of network states, and are well suited for application in neural mass/mean-field based brain network models.

Author summary

Characterizing the dynamics of biophysically modeled, large neuronal networks usually involves extensive numerical simulations. As an alternative to this expensive procedure we propose efficient models that describe the network activity in terms of a few ordinary differential equations. These systems are simple to solve and allow for convenient investigations of asynchronous, oscillatory or chaotic network states because linear stability analyses and powerful related methods are readily applicable. We build upon two research lines on which substantial efforts have been exerted in the last two decades: (i) the development of single neuron models of reduced complexity that can accurately reproduce a large repertoire of observed neuronal behavior, and (ii) different approaches to approximate the Fokker-Planck equation that represents the collective dynamics of large neuronal networks. We combine these advances and extend recent approximation methods of the latter kind to obtain spike rate models that surprisingly well reproduce the macroscopic dynamics of the underlying neuronal network. At the same time the microscopic properties are retained through the single neuron model parameters. To enable a fast adoption we have released an efficient Python implementation as open source software under a free license.

Introduction

There is prominent evidence that information in the brain, about a particular stimulus for example, is contained in the collective neuronal spiking activity averaged over populations of neurons with similar properties (population spike rate code) [1, 2]. Although these populations can comprise a large number of neurons [3], they often exhibit low-dimensional collective spiking dynamics [4] that can be measured using neural mass signals such as the local field potential or electroencephalography.

The behavior of cortical networks at that level is often studied computationally by employing simulations of multiple (realistically large or subsampled) populations of synaptically coupled individual spiking model neurons. A popular choice of single cell description for this purpose are two-variable integrate-and-fire models [5, 6] which describe the evolution of the fast (somatic) membrane voltage and an adaptation variable that represents a slowly-decaying potassium current. These models are computationally efficient and can be successfully calibrated using electrophysiological recordings of real cortical neurons and standard stimulation protocols [5, 7–10] to accurately reproduce their subthreshold and spiking activity. The choice of such (simple) neuron models, however, does not imply reasonable (short enough) simulation durations for a recurrent network, especially when large numbers of neurons and synaptic connections between them are considered.

A fast and mathematically tractable alternative to simulations of large networks are population activity models in terms of low-dimensional ordinary differential equations (i.e., which consist of only a few variables) that typically describe the evolution of the spike rate. These reduced models can be rapidly solved and allow for convenient analyses of the dynamical network states using well-known methods that are simple to implement. A popular example are the Wilson-Cowan equations [11], which were also extended to account for (slow) neuronal adaptation [12] and short-term synaptic depression [13]. Models of this type have been successfully applied to qualitatively characterize the possible dynamical states of coupled neuronal populations using phase space analyses [11–13], yet a direct link to more biophysically described networks of (calibrated) spiking neurons in terms of model parameters is missing.

Recently, derived population activity models have been proposed that bridge the gap between single neuron properties and mesoscopic network dynamics. These models are described by integral equations [14, 15] or partial differential equations [16, 17]

Here we derive simple models in terms of low-dimensional ordinary differential equations (ODEs) for the spike rate dynamics of sparsely coupled adaptive nonlinear integrate-and-fire neurons that are exposed to noisy synaptic input. The derivations are based on a Fokker-Planck equation that describes the neuronal population activity in the mean-field limit of large networks. We develop reduced models using recent methodological advances on two different approaches: the first is based on a spectral decomposition of the Fokker-Planck operator under two different slowness assumptions [18–20]. In the second approach we consider a cascade of linear temporal filters and a nonlinear function which are determined from the Fokker-Planck equation and semi-analytically approximated, building upon [21]. Both approaches are extended for an adaptation current, a nonlinear spike generating current and recurrent coupling with distributed synaptic delays.

We evaluate the developed low-dimensional spike rate models quantitatively in terms of reproduction accuracy in a systematic manner over a wide range of biologically plausible parameter values. In addition, we provide numerical implementations for the different reduction methods as well as the Fokker-Planck equation under a free license as open source project.

For the derived models in this contribution we use the adaptive exponential integrate-and-fire (aEIF) model [5] to describe individual neurons, which is similar to the model proposed by Izhikevich [6] but includes biophysically meaningful parameters and a refined description of spike initiation. However, the presented derivations are equally applicable when using the Izhikevich model instead (requiring only a small number of simple substitutions in the code).

Through their parameters the derived models retain a direct, quantitative link to the underlying spiking model neurons, and they are described in a well-established, convenient form (ODEs) that can be rapidly solved and analyzed. Therefore, these models are well suited (i) for mathematical analyses of dynamical states at the population level, e.g., linear stability analyses of attractors, and (ii) for application in multi-population brain network models. Apart from a specific network setting, the derived models are also appropriate as a spike rate description of individual neurons under noisy input conditions.

The structure of this article contains mildly redundant model specifications allowing the readers who are not interested in the methodological foundation to directly read the self-contained Sect. Results.

Results

Model reduction

The quantity of our interest is the population-averaged number of spikes emitted by a large homogeneous network of N sparsely coupled aEIF model neurons per small time interval, i.e., the spike rate r_N(t). The state of neuron i at time t is described by the membrane voltage V_i(t) and adaptation current w_i(t), which evolve piecewise continuously in response to overall synaptic current I_syn,i = I_ext,i(t) + I_rec,i(t). This input current consists of fluctuating network-external drive I_ext,i = C[μ_ext(t) + σ_ext(t)ξ_ext,i(t)] with membrane capacitance C, time-varying moments μ_ext, ${\sigma }_{\text{ext}}^2$ and unit Gaussian white noise process ξ_ext,i as well as recurrent input I_rec,i. The latter causes delayed postsynaptic potentials (i.e., deflections of V_i) of small amplitude J triggered by the spikes of K presynaptic neurons (see Sect. Methods for details).

Here we present two approaches of how the spike rate dynamics of the large, stochastic delay-differential equation system for the 2N states (V_i, w_i) can be described by simple models in terms of low-dimensional ODEs. Both approaches (i) take into account adaptation current dynamics that are sufficiently slow, allowing to replace the individual adaptation current w_i by its population-average 〈w〉, governed by

$$\frac{d⟨w⟩}{dt}=\frac{a({⟨V⟩}_{\infty }-E_w)-⟨w⟩}{{\tau }_w}+br\left(t\right),$$ (1)

where a, E_w, b, τ_w are the adaptation current model parameters (subthreshold conductance, reversal potential, spike-triggered increment, time constant, respectively), 〈V〉_∞ is the steady-state membrane voltage averaged across the population (which can vary over time, see below), and r is the spike rate of the respective low-dimensional model. Furthermore, both approaches (ii) are based on the observation that the collective dynamics of a large, sparsely coupled (and noise driven) network of integrate-and-fire type neurons can be well described by a Fokker-Planck equation. In this intermediate Fokker-Planck (FP) model the overall synaptic input is approximated by a mean part with additive white Gaussian fluctuations, I_syn,i/C ≈ μ_syn(t, r_d) + σ_syn(t, r_d)ξ_i(t), that are uncorrelated between neurons. The moments of the overall synaptic input,

$${\mu }_{\text{syn}}={\mu }_{\text{ext}}\left(t\right)+JK{r}_d\left(t\right),{\sigma }_{\text{syn}}^2={\sigma }_{\text{ext}}^2\left(t\right)+J^{2}K{r}_d\left(t\right),$$ (2)

depend on time via the moments of the external input and, due to recurrent coupling, on the delayed spike rate r_d. The latter is governed by

$$\frac{d{r}_d}{dt}=\frac{r-r_d}{{\tau }_d},$$ (3)

which corresponds to individual propagation delays drawn from an exponentially distributed random variable with mean τ_d. The FP model involves solving a partial differential equation (PDE) to obtain the time-varying membrane voltage distribution p(V, t) and the spike rate r(t).

The first reduction approach is based on the spectral decomposition of the Fokker-Planck operator $\mathcal{L}$ and leads to the following two low-dimensional models: the “basic” model variant (spec₁) is given by a complex-valued differential equation describing the spike rate evolution in its real part,

$$\frac{d\tilde{r}}{dt}={\lambda }_1(\tilde{r}-r_{\infty }),r\left(t\right)=\text{Re}\left\{\tilde{r}\right\},$$ (4)

where λ₁(μ_tot, σ_tot) is the dominant eigenvalue of $\mathcal{L}$ and r_∞(μ_tot, σ_tot) is the steady-state spike rate. Its parameters λ₁, r_∞, and 〈V〉_∞ (cf. Eq (1)) depend on the total input moments given by μ_tot(t) = μ_syn − 〈w〉/C and ${\sigma }_{\text{tot}}^2\left(t\right)={\sigma }_{\text{syn}}^2$ which closes the model (Eqs (1)–(4)). The other, “advanced” spectral model variant (spec₂) is given by a real-valued second order differential equation for the spike rate,

$${\beta }_2\ddot{r}+{\beta }_1\dot{r}+{\beta }_{0}r=r_{\infty }-r-{\beta }_c,$$ (5)

where the dots denote time derivatives. Its parameters β₂, β₁, β₀, β_c, r_∞ and 〈V〉_∞ depend on the total input moments (μ_tot, ${\sigma }_{\text{tot}}^2$) as follows: the latter two parameters explicitly as in the basic model above, the former four indirectly via the first two dominant eigenvalues λ₁, λ₂ and via additional quantities obtained from the (stationary and the first two nonstationary) eigenfunctions of $\mathcal{L}$ and its adjoint ${\mathcal{L}}^{*}$. Furthermore, the parameter β_c depends explicitly on the population-averaged adaptation current 〈w〉, the delayed spike rate r_d, and on the first and second order time derivatives of the external moments μ_ext and ${\sigma }_{\text{ext}}^2$.

The second approach is based on a Linear-Nonlinear (LN) cascade, in which the population spike rate is generated by applying to the time-varying mean and standard deviation of the overall synaptic input, μ_syn and σ_syn, separately a linear temporal filter, followed by a common nonlinear function. These three components–two linear filters and a nonlinearity–are extracted from the Fokker-Planck equation. Approximating the linear filters using exponentials and damped oscillating functions yields two model variants: In the basic “exponential” (LN_exp) model the filtered mean μ_f and standard deviation σ_f of the overall synaptic input are given by

$$\frac{d{\mu }_{\mathrm{f}}}{dt}=\frac{{\mu }_{\text{syn}}-{\mu }_{\mathrm{f}}}{{\tau }_{\mu }},\frac{d{\sigma }_{\mathrm{f}}}{dt}=\frac{{\sigma }_{\text{syn}}-{\sigma }_{\mathrm{f}}}{{\tau }_{\sigma }},$$ (6)

where the time constants τ_μ(μ_eff, σ_eff), τ_σ(μ_eff, σ_eff) depend on the effective (filtered) input mean μ_eff(t) = μ_f − 〈w〉/C and standard deviation σ_eff(t) = σ_f. The “damped oscillator” (LN_dos) model variant, on the other hand, describes the filtered input moments by

$${\ddot{\mu }}_{\mathrm{f}}+\frac{2}{\tau }{\dot{\mu }}_{\mathrm{f}}+(\frac{2}{{\tau }^2}+{\omega }^2){\mu }_{\mathrm{f}}=\frac{1+{\tau }^2{\omega }^2}{\tau }(\frac{{\mu }_{\text{syn}}}{\tau }+{\dot{\mu }}_{\text{syn}}),$$ (7)

$$\frac{d{\sigma }_{\mathrm{f}}}{dt}=\frac{{\sigma }_{\text{syn}}-{\sigma }_{\mathrm{f}}}{{\tau }_{\sigma }},$$ (8)

where the time constants τ(μ_tot, σ_tot), τ_σ(μ_tot, σ_tot) and the angular frequency ω(μ_tot, σ_tot) depend on the total input moments defined above. In both LN model variants the spike rate is obtained by the nonlinear transformation of the effective input moments through the steady-state spike rate,

$$r\left(t\right)=r_{\infty }({\mu }_{\text{eff}},{\sigma }_{\text{eff}}),$$ (9)

and the steady-state mean membrane voltage 〈V〉_∞ (cf. Eq (1)) is also evaluated at (μ_eff, σ_eff).

These four models (spec₁, spec₂, LN_exp, LN_dos) from both reduction approaches involve a number of parameters that depend on the strengths of synaptic input and adaptation current only via the total or effective input moments. We refer to these parameters as quantities below to distinguish them from fixed (independent) parameters. The computational complexity when numerically solving the models forward in time (for different parametrizations) can be greatly reduced by precomputing those quantities for a range of values for the total/effective input moments and using look-up tables during time integration. Changing any parameter value of the external input, the recurrent coupling or the adaptation current does not require renewed precomputations, enabling rapid explorations of parameter space and efficient (linear) stability analyses of network states.

The full specification of the “ground truth” system (network of aEIF neurons), the derivations of the intermediate description (FP model) and the low-dimensional spike rate models complemented by concrete numerical implementations are provided in Sect. Methods (that is complemented by the supporting material S1 Text). In Fig 1 we visualize the outputs of the different models using an example excitatory aEIF network exposed to external input with varying mean μ_ext(t) and standard deviation σ_ext(t).

Fig 1. Example of aEIF network response and output of derived models for varying input.

From top to bottom: Mean input μ_ext (black) together with input standard deviation σ_ext (gray, visualized for one neuron by sampling the respective white noise process ξ_ext,i). 2^nd row: Membrane voltage V of one neuron (gray, with spike times highlighted by black dots) and membrane voltage statistics from the excitatory coupled aEIF population of 50,000 neurons (red) and from the FP model (blue dashed): mean ± standard deviation over time, as well as voltage histograms (gray) and probability densities p(V, t) (blue dashed) at three indicated time points. 3^rd row: Adaptation current w of one neuron (gray) and mean adaptation currents of all models ± standard deviation for the aEIF network (shaded area). Note that differences in the mean adaptation currents of the different models are hardly recognizable. 4^th row: Spike times of a subset of 25 neurons randomly chosen from the network. Below: Spike rate r of the LN cascade based models (LN_exp, LN_dos) and the spectral models (spec₁, spec₂) in comparison to the FP model and the aEIF network (r_N). The values of the coupling parameters were J = 0.05 mV, K = 100, τ_d = 3 ms.

Performance for variations of the mean input

Here, and in the subsequent two sections, we assess the accuracy of the four low-dimensional models to reproduce the spike rate dynamics of the underlying aEIF population. The intermediate FP model is included for reference. The derived models generate population activity in response to overall synaptic input moments μ_syn and ${\sigma }_{\text{syn}}^2$. These depend on time via the external moments μ_ext(t) and ${\sigma }_{\text{ext}}^2\left(t\right)$, and the delayed spike rate r_d(t). Therefore, it is instrumental to first consider an uncoupled population and suitable variations of external input moments that effectively mimic a range of biologically plausible presynaptic spike rate dynamics. This allows us to systematically compare the reproduction performance of the different models over a manageable parameter space (without K, J, τ_d), yet it provides useful information on the accuracy for recurrent networks.

For many network settings the dominant effect of synaptic coupling is on the mean input (cf. Eq (2)). Therefore, we consider first in detail time-varying mean but constant variance of the input. Specifically, to account for a wide range of oscillation frequencies for presynaptic spike rates, μ_ext is described by an Ornstein-Uhlenbeck (OU) process

$${\dot{\mu }}_{\text{ext}}=\frac{\overline{\mu }-{\mu }_{\text{ext}}}{{\tau }_{\text{ou}}^{\mu }}+\sqrt{\frac{2}{{\tau }_{\text{ou}}^{\mu }}}{\vartheta }_{\mu }\xi \left(t\right),$$ (10)

where ${\tau }_{\text{ou}}^{\mu }$ denotes the correlation time, $\overline{\mu }$ and ϑ_μ are the mean and standard deviation of the stationary normal distribution, i.e., ${lim}_{t\to \infty }{\mu }_{\text{ext}}\left(t\right)\sim \mathcal{N}(\overline{\mu },{\vartheta }_{\mu }^2)$, and ξ is a unit Gaussian white noise process. Sample time series generated from the OU process are filtered using a Gaussian kernel with a small standard deviation σ_t to obtain sufficiently differentiable time series ${\tilde{\mu }}_{\text{ext}}$ (due to the requirements of the spec₂ model and the LN_dos model). The filtered realization ${\tilde{\mu }}_{\text{ext}}\left(t\right)$ is then used for all models to allow for a quantitative comparison of the different spike rate responses to the same input. The value of σ_t we use in this study effectively removes very large oscillation frequencies which are rarely observed, while lower frequencies [22] are passed.

The parameter space we explore covers large and small correlation times ${\tau }_{\text{ou}}^{\mu }$, strong and weak input mean $\overline{\mu }$ and standard deviation σ_ext, and for each of these combinations we consider an interval from small to large variation magnitudes ϑ_μ. The values of ${\tau }_{\text{ou}}^{\mu }$ and ϑ_μ determine how rapid and intense μ_ext(t) fluctuates.

We apply two performance measures, as in [21]. One is given by Pearson’s correlation coefficient,

$$\rho (r_N,r)≔\frac{{\sum }_{k=1}^M(r_N\left(t_k\right)-{\overline{r}}_N)(r\left(t_k\right)-\overline{r})}{\sqrt{{\sum }_{k=1}^M{(r_N\left(t_k\right)-{\overline{r}}_N)}^2{\sum }_{k=1}^M{(r\left(t_k\right)-\overline{r})}^2}},$$ (11)

between the (discretely given) spike rates of the aEIF population and each derived model with time averages ${\overline{r}}_N=1/M{\sum }_{k=1}^M{r}_N\left(t_k\right)$ and $\overline{r}=1/M{\sum }_{k=1}^{M}r\left(t_k\right)$ over a time interval of length t_M − t₁. For comparison, we also include the correlation coefficient between the aEIF population spike rate and the time-varying mean input, ρ(r_N, μ_ext). In addition, to assess absolute differences we calculate the root mean square (RMS) distance,

$${\text{d}}_{\text{RMS}}\left(r_N,r\right)≔\sqrt{\frac{1}{M}{\displaystyle \sum _{k=1}^M{\left(r_N\left(t_k\right)-r\left(t_k\right)\right)}^2}},$$ (12)

where M denotes the number of elements of the respective time series (r_N, r).

We find that three of the four low-dimensional spike rate models (spec₂, LN_exp, LN_dos) very well reproduce the spike rate r_N of the aEIF neurons: for the LN_exp model ρ > 0.95 and for the spec₂ and LN_dos models ρ ≳ 0.8 (each) over the explored parameter space, see Fig 2. Only the basic spectral model (spec₁) is substantially less accurate. Among the best models, the simplest (LN_exp) overall outperforms spec₂ and LN_dos, in particular for fast and strong mean input variations. However, in the strongly mean-driven regime the best performing model is spec₂.

Fig 2. Reproduction accuracy of the reduced models for variations of the mean input.

Pearson correlation coefficient (ρ) and root mean square distance (d_RMS) between the spike rate time series r_N(t) of the aEIF population and r(t) of each derived model (FP, spec₁, spec₂, LN_exp, LN_dos) for different strengths of baseline mean input $\overline{\mu }$, input standard deviation σ_ext, mean input variation ϑ_μ and for a large value of time constant ${\tau }_{\text{ou}}^{\mu }$ (A, moderately fast variations) as well as a smaller value (B, rapid variations). The input-output correlation (between μ_ext and r_N) is included as a reference (black dashed lines), and mean ± standard deviation of the population spike rate r_N are indicated (gray dashed lines, shaded areas). For each parametrization, activity time series of 60 s duration were generated (from 50,000 aEIF neurons and each derived model), from which the first second was omitted (each) to exclude transients, since the initial conditions of the models were not matched. Representative time series examples are shown on the right side with parameter values indicated (C), where empty and filled symbols correspond to large and small correlation time ${\tau }_{\text{ou}}^{\mu }$, respectively, relating the examples to the panels A and B. The adaptation current traces were excluded in all but the first example to allow for a larger number of parameter points.

We observe that the performance of any of the spike rate models decreases (with model-specific slope) with (i) increasing variation strength ϑ_μ larger than a certain (small) value, and with (ii) smaller ${\tau }_{\text{ou}}^{\mu }$, i.e., faster changes of μ_ext. For small values of ϑ_μ fluctuations of r_N, which are caused by the finite aEIF population size N and do not depend on the fluctuations of μ_ext, deteriorate the performance measured by ρ (see also [21], p.13 right). This explains why ρ does not increase as ϑ_μ decreases (towards zero) for any of the models. Naturally, the FP model is the by far most accurate spike rate description in terms of both measures, correlation coefficient ρ and RMS distance. This is not surprising because the four low-dimensional models are derived from that (infinite-dimensional) representation. Thus, the performance of the FP system defines an upper bound on the correlation coefficient ρ and a lower bound on the RMS distance for the low-dimensional models.

In detail: for moderately fast changing mean input (large ${\tau }_{\text{ou}}^{\mu }$) the three models spec₂, LN_exp and LN_dos exhibit excellent reproduction performance with ρ > 0.95, and spec₁ shows correlation coefficients of at least ρ = 0.9 (Fig 2A), which is substantially better than ρ(r_N, μ_ext). The small differences between the three top models can be better assessed from the RMS distance measure. For large input variance ${\sigma }_{\text{ext}}^2$ the two LN models perform best (cf. Fig 2A, top, and for an example, 2C). For weak input variance and large mean (small σ_ext, large $\overline{\mu }$) the spec₂ model outperforms the LN models, unless the variation magnitude ϑ_μ is very large. For small mean $\overline{\mu }$, where transient activity is interleaved with periods of quiescence, the LN_exp model performs best, except for weak variations ϑ_μ, where LN_dos is slightly better (see Fig 2A, bottom).

Stronger differences in performance emerge when considering faster changes of the mean input μ_ext(t) (i.e., for small ${\tau }_{\text{ou}}^{\mu }$), see Fig 2B, and for examples, Fig 2C. The spec₁ model again performs worst with ρ values even below the input/output correlation baseline ρ(r_N, μ_ext) for large mean input $\overline{\mu }$ (cf. Fig 2B, left). The spec₁ spike rate typically decays too slowly (cf. Fig 2C). The three better performing models differ as follows: for large input variance and mean (large σ_ext and $\overline{\mu }$), where the spike rate response to the input is rather fast (cf. increased ρ(r_N, μ_ext)), the performance of all three models in terms of ρ is very high, but the RMS distance measure indicates that LN_exp is the most accurate model (cf. Fig 2B, top). For weak mean input LN_exp is once again the top model while LN_dos and, especially noticeable, spec₂ show a performance decline (see example in Fig 2C). For weak input variance (Fig 2A, bottom), where significant (oscillatory) excursions of the spike rates in response to changes in the mean input can be observed (see also Fig 1), we obtain the following benchmark contrast: for large mean drive $\overline{\mu }$ the spec₂ model performs best, except for large variation amplitudes ϑ_μ, at which LN_exp is more accurate. Smaller mean input on the other hand corresponds to the most sensitive regime where periods of quiescence alternate with rapidly increasing and decaying spike rates. The LN_exp model shows the most robust and accurate spike rate reproduction in this setting, while LN_dos and spec₂ each exhibit decreased correlation and larger RMS distances–spec₂ even for moderate input variation intensities ϑ_μ. The slowness approximation underlying the spec₂ model likely induces an error due to the fast external input changes in comparison with the rather slow intrinsic time scale by the dominant eigenvalue, ${\tau }_{\text{ou}}^{\mu }=5\text{ms}$ vs. 1/|Re{λ₁}| ≈ 15 ms (cf. visualization of the spectrum in Sect. Spectral models). Note that for these weak inputs the distribution of the spike rate is rather asymmetric (cf. Fig 2B). Interestingly the LN_dos model performs worse than LN_exp for large mean input variations (i.e., large ϑ_μ) in general, and only slightly better for small input variance and mean input variations that are not too large and fast.

We would like to note that decreasing the Gaussian filter width σ_t to smaller values, e.g., fractions of a millisecond, can lead to a strong performance decline for the spec₂ model because of its explicit dependence on first and second order time derivatives of the mean input.

Furthermore, we show how the adaptation parameters affect the reproduction performance of the different models in Fig 3. The adaptation time constant τ_w and spike-triggered adaptation increment b are varied simultaneously (keeping their product constant) such that the average spike rate and adaptation current, and thus the spiking regime, remain comparable for all parametrizations. As expected, the accuracy of the derived models decreases for faster adaptation current dynamics, due to the adiabatic approximation that relies on sufficiently slow adaptation (cf. Sect. Methods). Interestingly however, the performance of all reduced models (except spec₁) declines only slightly as the adaptation time constant decreases to the value of the membrane time constant (which means the assumption of separated time scales underlying the adiabatic approximation is clearly violated). This kind of robustness is particularly pronounced for input with large baseline mean $\overline{\mu }$ and small noise amplitude σ_ext, cf. Fig 3B.

Fig 3. Effect of adaptation current timescale on reproduction accuracy.

Performance measures and population spike rate statistics (cf. Fig 2A and 2B) as a function of the adaptation time constant τ_w, that takes values between 20 ms (equal to the membrane time constant) and 200 ms (used throughout the rest of the study). The spike-triggered adaptation increment b was co-varied (antiproportional to τ_w) such that the product τ_w b = 8 pAs is fixed for all shown parametrizations. The input mean μ_ext(t) fluctuates with timescale ${\tau }_{\text{ou}}^{\mu }=50\text{ms}$ and strength ϑ_μ = 0.54 mV/ms (same value as for examples in Fig 2C) around a smaller (A) and a larger (B) baseline mean $\overline{\mu }$, while the input deviation σ_ext is constant. Note that the rightmost parametrization of A corresponds to Fig 2C (top example) and is contained in Fig 2A (bottom right) while that of B is shown in Fig 2A (bottom left).

Performance for variations of the input variance

For perfectly balanced excitatory and inhibitory synaptic coupling the contribution of presynaptic activity to the mean input μ_syn is zero by definition, but the input variance ${\sigma }_{\text{syn}}^2$ is always positively (linearly) affected by a presynaptic spike rate–even for a negative synaptic efficacy J (cf. Eq (2)). To assess the performance of the derived models in this scenario, but within the reference setting of an uncoupled population, we consider constant external mean drive μ_ext and let the variance ${\sigma }_{\text{ext}}^2\left(t\right)$ evolve according to a filtered OU process (such as that used for the mean input μ_ext in the previous section) with parameters $\overline{{\sigma }^2}$ and ϑ_σ² of the stationary normal distribution $\mathcal{N}(\overline{{\sigma }^2},{\vartheta }_{{\sigma }^2}^2)$, correlation time ${\tau }_{\text{ou}}^{{\sigma }^2}$ and Gaussian filter standard deviation σ_t as before.

The results of two input parametrizations are shown in Fig 4. For large input mean μ_ext and rapidly varying variance ${\sigma }_{\text{ext}}^2\left(t\right)$ the spike rate response of the aEIF population is very well reproduced by the FP model and, to a large extent, by the spec₂ model (cf. Fig 4A). This may be attributed to the fact that the latter model depends on the first two time derivatives of the input variance ${\sigma }_{\text{ext}}^2$. The LN models cannot well reproduce the rapid spike rate excursions in this setting, and the spec₁ model performs worst, exhibiting time-lagged spike rate dynamics compared to r_N(t) which leads to a very small value of correlation coefficient ρ (below the input/output correlation baseline $\rho (r_N,{\sigma }_{\text{ext}}^2)$). For smaller mean input μ_ext and moderately fast varying variance ${\sigma }_{\text{ext}}^2\left(t\right)$ (larger correlation time ${\tau }_{\text{ou}}^{{\sigma }^2}$) the fluctuating aEIF population spike rate is again nicely reproduced by the FP model while the rate response of the spec₂ model exhibits over-sensitive behavior to changes in the input variance, as indicated by the large RMS distance (see Fig 4B). This effect is even stronger for faster variations, i.e., smaller ${\tau }_{\text{ou}}^{{\sigma }^2}$ (cf. supplementary visualization S1 Fig). The LN models perform better in this setting, and the spec₁ model (again) performs worst in terms of correlation coefficient ρ due to its time-lagged spike rate response.

Fig 4. Performance for variations of the input variance.

Time series of population spike rate and mean adaptation current from the different models in response to varying ${\sigma }_{\text{ext}}^2$ for large mean input and rapid variations, μ_ext = 4 mV/ms, ${\tau }_{\text{ou}}^{{\sigma }^2}=5\text{ms}$ (A) and for small mean input and moderately fast variations, μ_ext = 1.5 mV/ms, ${\tau }_{\text{ou}}^{{\sigma }^2}=50\text{ms}$ (B). The values for the remaining input parameters were ${\overline{\sigma }}_{\text{ext}}^2=9{\text{mV}}^2/\text{ms}$, ϑ_σ² = 2 mV²/ms. For the aEIF population 〈w〉± standard deviation are visualized (red shaded areas). Note that the mean adaptation time series of all models as well as the spike rates of the cascade based models are on top of each other. The indicated Pearson correlation coefficients (with dashed input-output correlation) and root mean square distances were calculated from simulated spike rate time series of 60 s duration from which the first second was excluded, as the initial conditions of the models were not matched. In A the correlation ρ (but not the distance d_RMS) between the spike rates of the model spec₁ and the aEIF population) is strongly decreased due to a small time lag between the two time series which is difficult to see in the figure.

It should be noted that the lowest possible value of the input standard deviation, i.e., σ_ext (plus a nonnegative number in case of recurrent input) cannot be chosen completely freely but must be large enough ($\gtrsim 0.5\text{mV}/\sqrt{\text{ms}}$) for our parametrization. This is due to theoretical reasons (Fokker-Planck formalism) and practical reasons (numerics for Fokker-Planck solution and for calculation of the derived quantities, such as r_∞).

Oscillations in a recurrent network

To demonstrate the applicability of the low-dimensional models for network analyses we consider a recurrently coupled population of aEIF neurons that produces self-sustained network oscillations by the interplay of strong excitatory feedback and spike-triggered adaptation or, alternatively, by delayed recurrent synaptic inhibition [16, 23]. The former oscillation type is quite sensitive to changes in input, adaptation and especially coupling parameters for the current-based type of synaptic coupling considered here and due to lack of (synaptic) inhibition and refractoriness. For example, a small increase in coupling strength can lead to a dramatic (unphysiologic) increase in oscillation amplitude because of strong recurrent excitation. Hence we consider a difficult setting here to evaluate the reduced spike rate models–in particular, when the network operates close to a bifurcation.

In Fig 5A we present two example parametrizations from a region (in parameter space) that is characterized by stable oscillations. This means the network exhibits oscillatory spike rate dynamics for constant external input moments μ_ext and ${\sigma }_{\text{ext}}^2$. The derived models reproduce the limit cycle behavior of the aEIF network surprisingly well, except for small frequency and amplitude deviations (FP, spec₂, LN_dos, LN_exp) and larger frequency mismatch (spec₁), see Fig 5A, top. For weaker input moments and increased spike-triggered adaptation strength the network is closer to a Hopf bifurcation [16, 23]. It is, therefore, not surprising that the differences in oscillation period and amplitude are more prominent (cf. Fig 5A, bottom). The bifurcation point of the LN_exp model is slightly shifted, shown by the slowly damped oscillatory convergence to a fixed point. This suggests that the bifurcation parameter value of each of the derived models is not far from the true critical parameter value of the aEIF network but can quantitatively differ (slightly) in a model-dependent way.

Fig 5. Network-generated oscillations.

Oscillatory population spike rate and mean adaptation current of 50,000 excitatory coupled aEIF neurons and each of the derived models (for constant external input moments) generated by the interplay of recurrent excitation/adaptation current (A) and by delayed recurrent inhibition (B). In addition, the limit cycle of the LN_exp model is shown in terms of the (quantity) steady-state spike rate r_∞ as a function of effective input moments μ_eff, ${\sigma }_{\text{eff}}^2$ (A, top) and for the spec₂ model in dependence of the total input moments (μ_tot, ${\sigma }_{\text{tot}}^2$ (A, bottom). The phase of the cycle is visualized by grayscale color code (increasing phase from black to white). The values for the input, adaptation and coupling parameters were μ_ext = 1.5 mV/ms, ${\sigma }_{\text{ext}}=2\text{mV}/\sqrt{\text{ms}}$, a = 3 nS, b = 30 pA (A, top), μ_ext = 1.275 mV/ms, ${\sigma }_{\text{ext}}=1.5\text{mV}/\sqrt{\text{ms}}$, a = 3 nS, b = 60 pA (A, bottom), K = 1000, J = 0.03 mV, τ_d = 3 ms (A, both). In B adaptation was removed (a = b = 0) and delays were identical d_ij = d; input and coupling parameter values were μ_ext = 1.5 mV/ms, ${\sigma }_{\text{ext}}=1.5\text{mV}/\sqrt{\text{ms}}$, K = 1000, J = −0.0357 mV, d = 10 ms (top) and μ_ext = 3 mV/ms, ${\sigma }_{\text{ext}}=2\text{mV}/\sqrt{\text{ms}}$, K = 1000, J = −0.087 mV, d = 5 ms (bottom).

The second type of oscillation is generated by delayed synaptic inhibition [22] and does not depend on the (neuronal) inhibition that is provided by an adaptation current. To demonstrate this independence the adaptation current was disabled (by setting the parameters a = b = 0) for the two respective examples that are shown in Fig 5B. Similarly as for the previous oscillation type, the low-dimensional models (except spec₁) reproduce the spike rate limit cycle of the aEIF network surprisingly well, in particular for weak external input (see Fig 5B, top). For larger external input and stronger inhibition with shorter delay the network operates close to a Hopf bifurcation, leading to larger differences in oscillation amplitude and frequency in a model-dependent way (Fig 5B, bottom). Note that the intermediate (Fokker-Planck) model very well reproduces the inhibition-based type of oscillation which demonstrates the applicability of the underlying mean-field approximation. We would also like to note that enabling the adaptation current dynamics (only) leads to decreased average spike rates but does not affect the reproduction accuracy.

We would like to emphasize that the previous comprehensive evaluations for an uncoupled population provide a deeper insight on the reproduction performance–also for a recurrent network–than the four examples shown here, as explained in the Sect. Performance for variations of the mean input. For example, the (improved) reproduction performance for increased input variance in the uncoupled setting (cf. Fig 2) informs about the reproduction performance for networks of excitatory and inhibitory neurons that are roughly balanced, i.e., where the overall input mean is rather small compared to the input standard deviation.

Implementation and computational complexity

We have developed efficient implementations of the derived models using the Python programming language and by employing the library Numba for low-level machine acceleration [24]. These include: (i) the numerical integration of the Fokker-Planck model using an accurate finite volume scheme with implicit time discretization (cf. Sect. Methods), (ii) the parallelized precalculation of the quantities required by the low-dimensional spike rate models and (iii) the time integration of the latter models, as well as example scripts demonstrating (i)–(iii). The code is available as open source software under a free license at GitHub: https://github.com/neuromethods/fokker-planck-based-spike-rate-models

With regards to computational cost, summarizing the results of several aEIF network parametrizations, the duration to generate population activity time series for the low-dimensional spike rate models is usually several orders of magnitude smaller compared to numerical simulation of the original aEIF network and a few orders of magnitude smaller in comparison to the numerical solution of the FP model. For example, considering a population of 50,000 coupled neurons with 2% connection probability, a single simulation run of 5 s and the same integration time step across the models, the computation times amounted to 1.1–3.6 s for the low-dimensional models (with order–fast to slow–LN_exp, spec₁, LN_dos, spec₂), about 100 s for the FP model and roughly 1500 s for the aEIF network simulation on a dual-core laptop computer. The time difference to the network simulation substantially increases with the numbers of neurons and connections, and with spiking activity within the network due to the extensive propagation of synaptic events. Note that the speedup becomes even more pronounced with increasing number of populations, where the runtimes of the FP model and the aEIF network simulation scale linearly and the low-dimensional models show a sublinear runtime increase due to vectorization of the state variables representing the different populations.

The derived low-dimensional (ODE) spike rate models are very efficient to integrate given that the required input-dependent parameters are available as precalulated look-up quantities. For the grids used in this contribution, the precomputation time was 40 min. for the cascade (LN_exp, LN_dos) models and 120 min. for the spectral (spec₁, spec₂) models, both on a hexa-core desktop computer. The longer calculation time for the spectral models was due to the finer internal grid for the mean input (see S1 Text).

Note that while the time integration of the spec₂ model is on the same order as for the other low-dimensional models its implementation complexity is larger because of the many quantities it depends on, cf. Eqs (63)–(66).

Discussion

In this contribution we have developed four low-dimensional models that approximate the spike rate dynamics of coupled aEIF neurons and retain all parameters of the underlying model neurons. These simple spike rate models were derived in two different ways from a Fokker-Planck PDE that describes the evolving membrane voltage distribution in the mean-field limit of large networks, and is complemented by an ODE for the population-averaged slow adaptation current. Two of the reduced spike rate models (spec₁ and spec₂) were obtained by a truncated spectral decomposition of the Fokker-Planck operator assuming vanishingly slow (for spec₁) or moderately slow (for spec₂) changes of the input moments. The other two reduced models (LN_exp and LN_dos) are described by a cascade of linear filters (one for the input mean and another for its standard deviation) and a nonlinearity which were derived from the Fokker-Planck equation, and subsequently the filters were semi-analytically approximated. Our approaches build upon [18–20] as well as [21], and extend those methods for adaptive nonlinear integrate-and-fire neurons that are sparsely coupled with distributed delays (cf. Sect. Methods).

We have compared the different spike rate representations for a range of biologically plausible input statistics and found that three of the reduced models (spec₂, LN_exp and LN_dos) accurately reproduce the spiking activity of the underlying aEIF population while one model (spec₁) shows the least accuracy. Among the best models, the simplest (LN_exp) was the most robust and (somewhat surprisingly) overall outperformed spec₂ and LN_dos–especially in the sensitive regime of rapidly changing sub- and suprathreshold mean drive and in general for rapid and strong input variations. The LN_exp model did not exhibit exaggerated deflections in that regime as compared to the other two models. This result is likely due to the importance of the quantitatively correct decay time of the filter for the mean input in the LN_exp model, while the violations of the slowness assumptions for the spec₂ and LN_dos models seem more harmful in this regime. In the strongly mean-driven regime, however, the best performing model was spec₂ for variations both in the mean drive (as long as those variations are not too strong and fast) and for variations of the input variance.

We have also demonstrated that the low-dimensional models well reproduce the dynamics of recurrently coupled aEIF populations in terms of asynchronous states (see Fig 1) and spike rate oscillations (cf. Fig 5), where mild deviations at critical (bifurcation) parameter values are expected due to the approximative nature of the model reduction.

The computational demands of the low-dimensional models are very modest in comparison to the aEIF network and also to the integration of the Fokker-Planck PDE, for which we have developed a novel finite volume discretization scheme. We would like to emphasize that any change of a parameter value for input, coupling or adaptation current does not require renewed precomputations. To facilitate the application of the presented models we have made available implementations that precompute all required quantities and numerically integrate the derived low-dimensional spike rate models as well as the Fokker-Planck equation, together with example (Python) scripts, as open source software.

Since the derived models are formulated in terms of simple ODEs, they allow to conveniently perform linear stability analyses, e.g., based on the eigenvalues of the Jacobian matrix of the respective vector field. In this way network states can be rapidly characterized by quantifying the bifurcation structure of the population dynamics–including regions of the parameter space where multiple fixed points and/or limit cycle attractors co-exist. For a characterization of stable network states by numerical continuation and an assessment of their controllability through neuromodulators using the LN_exp model see [23] ch. 4.2 and [25]. Furthermore, the low-dimensional models are well suited to be employed in large neuronal networks of multiple populations for efficient simulations of population-averaged activity time series. Overall, the LN_exp model seems a good candidate for that purpose considering its accuracy and robustness, as well as its computational and implementational simplicity.

Extensions

Heterogeneity

We considered a homogeneous population of neurons in the sense that the parameter values across model neurons are identical except those for synaptic input. Thereby we assume that neurons with similar dynamical properties can be grouped into populations [3]. Heterogeneity is incorporated by distributed synaptic delays, by sparse random coupling, and by fluctuating external inputs for each neuron. The (reduced) population models further allow for heterogeneous synaptic strengths that are sampled from a Gaussian distribution and can be included in a straightforward way [16, 26] (see also Sect. Methods). Distributed values for other parameters (of the isolated model neurons within the same population) are currently not supported.

Multiple populations

The presented mean-field network model can be easily adjusted for multiple populations. In this case we obtain a low-dimensional ODE for each population and the overall synaptic moments for population k become

$${\mu }_{\text{syn},k}={\mu }_{\text{ext},k}\left(t\right)+\sum _l{J}_{kl}{K}_{kl}{r}_{d,kl}\left(t\right),{\sigma }_{\text{syn},k}^2={\sigma }_{\text{ext},k}^2\left(t\right)+\sum _l{J}_{kl}^2{K}_{kl}{r}_{d,kl}\left(t\right),$$ (13)

where J_kl is the synaptic strength for the K_kl neurons from population l targeting neurons from population k and r_d,kl is the delayed spike rate of population l affecting population k (cf. Eq (2)). For each pair of coupled populations we may consider identical or distributed delays (using distributions from the exponential family) as well as identical or distributed synaptic strengths (sampled from a Gaussian distribution).

Synaptic coupling

Here we described synaptic interaction by delayed (delta) current pulses with delays sampled from an exponential distribution. This description leads to a fluctuating overall synaptic input current with white noise characteristics. Interestingly, for the mean-field dynamics this setting is very similar to considering exponentially decaying synaptic currents with a decay constant that matches that of the delay distribution, although the overall synaptic input current is a colored noise process in that case, see [27] and, for an intuitive explanation [28].

A conductance-based model of synaptic coupling can also be considered in principle [16, 29], which results in a multiplicative noise process for the overall synaptic input. This, however, would in general impede the beneficial concept of precalculated “look-up” quantities that are unaffected by the input and coupling parameters.

It should be noted that most current- or conductance-based models of synaptic coupling (including the one considered here) can produce unphysiologically large amounts of synaptic current in case of high presynaptic activity, unless the coupling parameters are carefully tuned. This problem can be solved, for example, by considering a (more realistic) model of synaptic coupling based on [30], from which activity-dependent coupling terms can be derived for the mean-field and reduced population models [23] ch. 4.2. Using that description ensures robust simulation of population activity time series without having to fine-tune the coupling parameter values, which is particularly useful for multi-population network models. In this contribution though we used for simplicity a basic synaptic coupling model that has frequently been applied in the mean-field literature.

Input noise process

The Gaussian stochastic process driving the individual neurons could also be substituted by colored noise, which would lead to a Fokker-Planck model with increased dimensionality [31]. However, this would require more complex and computationally expensive numerical schemes not only to solve that model but also for the different dimension reduction approaches.

Slow adaptation

To derive low-dimensional models of population activity we approximated the adaptation current by its population average, justified by its slow dynamics compared to the other time scales of the system. This approximation is equivalent to a first order moment closure method [17]. In case of a faster adaptation time scale the approximation may be improved by considering second and higher order moments [17, 32].

Population size

The mean-field models presented here can well reproduce the dynamics of population-averaged state variables (that is, spike rate, mean membrane voltage, and mean adaptation current) for large populations (N → ∞ in the derivation). Fluctuations of those average variables due to the finite size of neuronal populations, however, are not captured. Hence, it would be interesting to extend the mean-field models so as to reproduce these (so-called) finite size effects, for example, by incorporating an appropriate stochastic process [18] or using concepts from [15].

Cascade approach

For uncoupled EIF populations (without an adaptation current) and constant input standard deviation it has been shown that the LN cascade approximation performs well for physiological ranges of amplitude and time scale for mean input variations [21]. Our results for the cascade models are consistent with [21], but the performance is substantially improved for the sensitive low (baseline) input regime (LN_exp and LN_dos, also in absence of adaptation), and damped oscillatory behavior (including over- and undershoots) is accounted for by the LN_dos model.

To achieve these improvements we semi-analytically fit the linear filters derived from the Fokker-Planck equation using exponential and damped oscillator functions considering a range of input frequencies. The approximation can be further improved by using more complex functions, such as a damped oscillator with two time scales. That, however, can lead to less robustness (i.e., undesired model behavior) for rapid and strong changes of the input moments (cf. Sect. Methods).

LN cascade models are frequently applied in neuroscience to describe population activity, and the model components are often determined from electrophysiological recordings using established techniques. The methodology presented here contributes to establishing quantitative links between networks of spiking neurons, a mesoscopic description of population activity and recordings at the population level.

Spectral approach

Here we provide a new numerical solver for the eigenvalue problem of the Fokker-Planck operator and its adjoint. This allows to compute the full spectrum together with associated eigenfunctions and is applicable to nonlinear integrate-and-fire models, extending [18, 19, 33].

Using that solver the spec₂ model, which is based on two eigenvalues, can be further improved by interpolating its coefficients, Eqs (63)–(66), around the double eigenvalues at the spectrum’s real-to-complex transition. This interpolation would effectively smooth the quantities—e.g., preventing the jumps and kinks that are present in the visualization of Sect. Spectral models—and is expected to increase the spike rate reproduction accuracy (particularly for weak mean input) beyond what was reported in this contribution.

The spec₂ model can also be extended to yield a third order ODE with everywhere smooth coefficients by considering an additional eigenvalue (cf. Sect. Remarks on the spectrum).

Moreover, the spec₂ model, and more generally the whole spectral decomposition approach, can be extended to account for a refractory period in the presence of time-varying total input moments, e.g., by building upon previous attempts [18, 34, 35].

Furthermore, it could be beneficial to explicitly quantify the approximation error due to the slowness assumption that underlies the spec₂ model by integration of the (original) spectral representation of the Fokker-Planck model.

Both reduced spectral models allow for a refined description of the mean adaptation current dynamics, cf. Eq (1), by replacing the mean membrane voltage 〈V〉 with its steady-state value 〈V〉_∞, using that the membrane voltage distribution is available through the eigenfunctions of the Fokker-Planck operator.

The numerical eigenvalue solver can be extended in a straightforward way to yield quantities that are required by the original spectral representation of the Fokker-Planck model and by the corresponding stochastic equation for finite population size [18].

Alternative derived models

In addition to the work we build upon [18–21] (cf. Sect. Methods) there are a few other approaches to derive spike rate models from populations of spiking neurons. Some methods also result in an ODE system, taking into account (slow) neuronal adaptation [17, 26, 36–38] or disregarding it [39]. The settings differ from the work presented here in that (i) the intrinsic neuronal dynamics are adiabatically neglected [17, 26, 36, 37], (ii) only uncoupled populations [38] or all-to-all connected networks [17, 36, 39] are assumed in contrast to sparse connectivity, and (iii) (fixed) heterogeneous instead of fluctuating input is considered [39]. Notably, these previous methods yield rather qualitative agreements with the underlying spiking neuron population activity except for [39] where an excellent quantitative reproduction for (non-adaptive) quadratic integrate-and-fire oscillators with quenched input randomness is reported.

Other approaches yield mesoscopic representations of population activity in terms of model classes that are substantially less efficient to simulate and more complicated to analyze than low-dimensional ODEs [14–17, 40–42]. The spike rate dynamics in these models has been described (i) by a rather complex ODE system that depends on a stochastic jump process derived for integrate-and-fire neurons without adaptation [40], (ii) by PDEs for recurrently connected aEIF [16] or Izhikevich [17] neurons, (iii) by an integro-PDE with displacement for non-adaptive neurons [42] or (iv) by integral equations that represent the (mean) activity of coupled phenomenological spiking neurons without [41] and with adaptation [14, 15].

Furthermore, the stationary condition of a noise-driven population of adaptive EIF neurons [32, 43, 44] and the first order spike rate response to weak input modulations [43, 44] have been analyzed using the Fokker-Planck equation. Ref. [32] also considered a refined approximation of the (purely spike-triggered) adaptation current including higher order moments.

It may be interesting for future studies to explore ways to extend the presented methods and relax some of the underlying assumptions, in particular, considering (i) the diffusion approximation (via shot noise input, e.g., [45, 46]), (ii) the Poisson assumption (e.g., using the concept from [47] in combination with results from [48]) and (iii) (noise) correlations (see, e.g., [49]).

Methods

Here we present all models in detail—the aEIF network (ground truth), the mean-field FP system (intermediate model) and the low-dimensional models: spec₁, spec₂, LN_exp, LN_dos—including step-by-step derivations and essential information on the respective numerical solution methods. An implementation of these models using Python is made available at GitHub: https://github.com/neuromethods/fokker-planck-based-spike-rate-models

Network model

We consider a large (homogeneous) population of N synaptically coupled aEIF model neurons [5]. Specifically, for each neuron (i = 1, …, N), the dynamics of the membrane voltage V_i is described by

$$C\frac{d{V}_i}{dt}=I_{\mathrm{L}}\left(V_i\right)+I_{\text{exp}}\left(V_i\right)-w_i+I_{\text{syn},i}\left(t\right),$$ (14)

where the capacitive current through the membrane with capacitance C equals the sum of three ionic currents and the synaptic current I_syn,i. The ionic currents consist of a linear leak current I_L(V_i) = −g_L(V_i − E_L) with conductance g_L and reversal potential E_L, a nonlinear term I_exp(V_i) = g_L Δ_T exp((V_i − V_T)/Δ_T) that approximates the rapidly increasing Na⁺ current at spike initiation with threshold slope factor Δ_T and effective threshold voltage V_T, and the adaptation current w_i which reflects a slowly deactivating K⁺ current. The adaptation current evolves according to

$${\tau }_w\frac{d{w}_i}{dt}=a(V_i-E_w)-w_i,$$ (15)

with adaptation time constant τ_w. Its strength depends on the subthreshold membrane voltage via conductance a. E_w denotes its reversal potential. When V_i increases beyond V_T, it diverges to infinity in finite time due to the exponentially increasing current I_exp(V_i), which defines a spike. In practice, however, the spike is said to occur when V_i reaches a given value V_s—the spike voltage. The downswing of the spike is not explicitly modeled; instead, when V_i ≥ V_s, the membrane voltage V_i is instantaneously reset to a lower value V_r. At the same time, the adaptation current w_i is incremented by a value of parameter b, which implements suprathreshold (spike-dependent) activation of the adaptation current.

Immediately after the reset, V_i and w_i are clamped (i.e., remain constant) for a short refractory period T_ref, and subsequently governed again by Eqs (14) and (15). At the end of the Methods section we describe how (optionally) a spike shape can be included in the aEIF model, together with the associated small changes for the models derived from it.

To complete the network model the synaptic current in Eq (14) needs to be specified: for each cell it is given by the sum of recurrent and external input, I_syn,i = I_rec,i(t) + I_ext,i(t). Recurrent synaptic input is received from K other neurons of the network, that are connected in a sparse (K ≪ N) and uniformly random way, and is modeled by

$$I_{\text{rec},i}=C{\displaystyle \sum _j{J}_{ij}}{\displaystyle \sum _{t_j}\delta }(t-t_j-d_{ij}),$$ (16)

where δ denotes the Dirac delta function. Every spike by one of the K presynaptic neurons with indices j and spike times t_j causes a postsynaptic membrane voltage jump of size J_ij. The coupling strength is positive (negative) for excitation (inhibition) and of small magnitude. Here it is chosen to be constant, i.e., J_ij = J. Each of these membrane voltage deflections occur after a time delay d_ij that takes into account (axonal and dendritic) spike propagation times and is sampled (independently) from a probability distribution p_d. In this work we use exponentially distributed delays, i.e., p_d(τ) = exp(−τ/τ_d)/τ_d (for τ ≥ 0) with mean delay τ_d.

The second type of synaptic input is a fluctuating current generated from network-external neurons,

$$I_{\text{ext},i}=C[{\mu }_{\text{ext}}\left(t\right)+{\sigma }_{\text{ext}}\left(t\right){\xi }_{\text{ext},i}\left(t\right)],$$ (17)

with time-varying moments μ_ext and ${\sigma }_{\text{ext}}^2$, and unit Gaussian white noise process ξ_ext,i. The latter is uncorrelated with that of other neurons j ≠ i, i.e., 〈ξ_ext,i(t)ξ_ext,j(t + τ)〉 = δ(τ)δ_ij, where 〈·〉 denotes expectation (w.r.t. the joint ensemble of noise realizations at times t and t + τ) and δ_ij is the Kronecker delta. This external current, for example, accurately approximates the input generated from a large number of independent Poisson neurons that produce instantaneous postsynaptic potentials of small magnitude, cf. [48].

The spike rate r_N of the network is defined as the population-averaged number of emitted spikes per time interval [t, t + ΔT],

$$r_N(t)=\frac{1}{N}{\displaystyle \sum _{i=1}^N\frac{1}{\Delta T}}{\displaystyle {\int }_t^{t+\Delta T}{\displaystyle \sum _{t_i}\delta }}(s-t_i)ds,$$ (18)

where the interval size ΔT is practically chosen small enough to capture the dynamical structure and large enough to yield a comparably smooth time evolution for a finite network, i.e., N < ∞.

We chose values for the neuron model parameters to describe cortical pyramidal cells, which exhibit “regular spiking” behavior and spike frequency adaptation [7, 50, 51]. For the complete parameter specification see Table 1.

Table 1. Parameter values used throughout the study.

Name	Symbol	Value
Network model
Number of neurons	N	50,000
Membrane capacitance	C	200 pF
Leak conductance	gL	10 nS
Leak reversal potential	EL	−65 mV
Threshold slope factor	ΔT	1.5 mV
Threshold voltage	VT	−50 mV
Spike voltage	Vs	−40 mV
Reset voltage	Vr	−70 mV
Subthreshold adaptation conductance1	a	4 nS
Spike-triggered adaptation increment1	b	40 pA
Adaptation reversal potential	Ew	−80 mV
Adaptation time constant	τw	200 ms
Refractory period2	Tref	0 ms
Gaussian filter width for external input	σt	1 ms
Discretization time step	Δt	0.05 ms
Spike rate estimation bin width	ΔT	1 ms
Fokker-Planck model
Membrane voltage lower bound	Vlb	−200 mV
Finite-volume membrane voltage spacing	ΔV	0.028 mV
Discretization time step	Δt	0.05 ms
Low-dimensional models
Discretization time step	Δt	0.01 ms
Membrane voltage spacing3	ΔV	0.01 mV
Spacing of mean input3	Δμ	0.025 mV/ms
Spacing of input standard deviation3	Δσ	$0.1\text{mV}/\sqrt{\text{ms}}$

¹If not specified otherwise.

²A nonzero refractory period is not supported by the spec₂ model.

³Parameters for precalculation of model quantities (before simulation).

The values of coupling parameters (K, J, τ_d) are specified in the captions of Figs 1 and 5, the values of parameters for the external input, μ_ext or ($\overline{\mu }$, ${\tau }_{\text{ou}}^{\mu }$, ϑ_μ), and σ_ext or ($\overline{{\sigma }^2}$, ${\tau }_{\text{ou}}^{{\sigma }^2}$, ϑ_σ²), are provided in each figure (caption).

All network simulations were performed using the Python software BRIAN2 [52, 53] with C++ code generation enabled for efficiency. The aEIF model Eqs (14) and (15) were discretized using the Euler-Maruyama method with equidistant time step Δt and initialized with w_i(0) = 0 and V_i(0) that is (independently) sampled from a Gaussian initial distribution p₀(V) with mean V_r − δV and standard deviation δV/2 where δV = V_T − V_r. Note that the models derived in the following Sects. do not depend on this particular initial density shape but allow for an arbitrary (density) function p₀.

Fokker-Planck system

Adiabatic approximation

The time scales of (slow) K⁺ channel kinetics which are effectively described by the adaptation current w_i, cf. Eq (15), are typically much larger than the faster membrane voltage dynamics modeled by Eq (14), i.e., τ_w ≫ C/g_L [54–57]. This observation justifies to replace the individual adaptation current w_i in Eq (14) by its average across the population, ${\langle w\rangle }_N=1/N{\sum }_{i=1}^N{w}_i\left(t\right)$, in order to reduce computational demands and enable further analysis. The mean adaptation current is then governed by [16, 26, 48, 58]

$$\frac{d{⟨w⟩}_N}{dt}=\frac{a({⟨V⟩}_N-E_w)-{⟨w⟩}_N}{{\tau }_w}+b{r}_N\left(t\right),$$ (19)

where 〈V〉_N denotes the time-varying population average of the membrane voltage of non-refractory neurons.

The dynamics of the population-averaged adaptation current reflecting the non-refractory proportion of neurons are well captured by Eq (19) as long as T_ref is small compared to τ_w. In this (physiologically plausible) case 〈w〉_N from Eq (19) can be considered equal to the average adaptation current over the refractory proportion of neurons [16, 48].

Mean field limit

For large networks (N → ∞) the recurrent input can be approximated by a mean part with additive fluctuations, $I_{\text{rec}}{}_{,\text{i}}/C\approx JK{r}_d\left(t\right)+J\sqrt{K{r}_d\left(t\right)}{\xi }_{\text{rec},i}\left(t\right)$ with delayed spike rate

$$r_d=r*p_d,$$ (20)

i.e., the spike rate convolved with the delay distribution, and unit white Gaussian noise process ξ_rec,i that is uncorrelated to that of any other neuron [16, 18, 26, 34].

The step is valid under the assumptions of (i) sufficiently many incoming synaptic connections (K ≫ 1) with small enough weights |J_ij| in comparison with V_T − V_r and sufficient presynaptic activity (diffusion approximation) (ii) that neuronal spike trains can be approximated by independent Poisson processes (Poisson assumption) and (iii) that the correlations between the fluctuations of synaptic inputs for different neurons vanish (mean-field limit). The latter assumption is fulfilled by sparse and uniformly random synaptic connectivity, but also when synaptic strengths J_ij and delays d_ij are independently distributed (in case of less sparse or random connections) [18].

This approximation of the recurrent input allows to replace the overall synaptic current in Eq (14) by I_syn,i = C[μ_syn(t, r_d) + σ_syn(t, r_d)ξ_i(t)] with overall synaptic moments

$${\mu }_{\text{syn}}={\mu }_{\text{ext}}\left(t\right)+JK{r}_d\left(t\right),{\sigma }_{\text{syn}}^2={\sigma }_{\text{ext}}^2\left(t\right)+J^{2}K{r}_d\left(t\right),$$ (21)

and (overall) unit Gaussian white noise ξ_i that is uncorrelated to that of any other neuron. Here we have used that external I_ext,i and recurrent synaptic current I_rec,i are independent from each other.

The resulting mean-field dynamics of the membrane voltage is given by

$$\frac{d{V}_i}{dt}=\frac{I_{\mathrm{L}}\left(V_i\right)+I_{\text{exp}}\left(V_i\right)-⟨w⟩}{C}+{\mu }_{\text{syn}}(t,r_d)+{\sigma }_{\text{syn}}(t,r_d){\xi }_i\left(t\right),$$ (22)

and corresponds to a McKean-Vlasov type of equation with distributed delays [59] and discontinuity due to the reset mechanism [60] that complements the dynamics of V_i as before. The population-averaged adaptation current 〈w〉 = lim_{N → ∞}〈w〉_N is governed by

$$\frac{d⟨w⟩}{dt}=\frac{a(⟨V⟩-E_w\phantom{{}^2})-⟨w⟩}{{\tau }_w}+br\left(t\right),$$ (23)

with mean membrane voltage (of non-refractory neurons), 〈V〉 = lim_N→∞〈V〉_N, and spike rate r = lim_{N→∞,Δt→0} r_N(t).

Remarks: Instead of exponentially distributed synaptic delays we may also consider other continuous densities p_d, identical delays, p_d(τ) = δ(τ − d) with d > 0, or no delays at all, p_d(τ) = δ(τ). Instead of identical synaptic strengths one may also consider strengths J_ij that are drawn independently from a normal distribution with mean J_m and variance J_v instead, in which case the overall synaptic moments become μ_syn = μ_ext(t) + J_mKr_d(t) and ${\sigma }_{\text{syn}}^2={\sigma }_{\text{ext}}^2\left(t\right)+(J_m^2+J_v)K{r}_d\left(t\right)$, cf. [16, 26].

Continuity equation

In the membrane voltage evolution, Eq (22), individual neurons are exchangeable as they are described by the same stochastic equations and are coupled to each other exclusively through the (delayed) spike rate via the overall synaptic moments μ_syn and ${\sigma }_{\text{syn}}^2$. Therefore, the adiabatic and mean-field approximations allow us to represent the collective dynamics of a large network by a (1+1-dimensional) Fokker-Planck equation [16, 18, 26, 34],

$$\frac{\partial }{\partial t}p(V,t)+\frac{\partial }{\partial V}{q}_p(V,t)=0\text{for}V\in (-\infty ,V_{\mathrm{s}}],t>0,$$ (24)

which describes the evolution of the probability density p(V, t) to find a neuron in state V at time t (in continuity form). The probability flux is given by

$$q_p(V,t)=(\frac{I_{\mathrm{L}}\left(V\right)+I_{\text{exp}}\left(V\right)}{C}+{\mu }_{\text{tot}}\left(t\right))p(V,t)-\frac{{\sigma }_{\text{tot}}^2\left(t\right)}{2}\frac{\partial }{\partial V}p(V,t),$$ (25)

with total input mean and standard deviation,

$${\mu }_{\text{tot}}\left(t\right)={\mu }_{\text{syn}}({\mu }_{\text{ext}}\left(t\right),r_d\left(t\right))-⟨w⟩\left(t\right)/C$$ (26)

$${\sigma }_{\text{tot}}\left(t\right)={\sigma }_{\text{syn}}({\sigma }_{\text{ext}}\left(t\right),r_d\left(t\right)).$$ (27)

Note that the mean adaptation current (simply) subtracts from the synaptic mean in the drift term, cf. Eq (22).

The mean adaptation current evolves according to Eq (23) with time-dependent mean membrane voltage (of the non-refractory neurons)

$$⟨V⟩=\frac{{\int }_{-\infty }^{V_{\mathrm{s}}}vp(v,t)dv}{{\int }_{-\infty }^{V_{\mathrm{s}}}p(v,t)dv}.$$ (28)

The spike rate r is obtained by the probability flux through V_s,

$$r\left(t\right)=q_p(V_s,t).$$ (29)

To account for the reset condition of the aEIF neuron dynamics and ensuring that probability mass is conserved, Eq (24) is complemented by the reinjection condition,

$$q_p(V_{\mathrm{r}}^{+},t)-q_p(V_{\mathrm{r}}^{-},t)=q_p(V_{\mathrm{s}},t-T_{\text{ref}}),$$ (30)

where $q_p\left(V_{\mathrm{r}}^{+}\right)≔{lim}_{V\searrow V_{\mathrm{r}}}{q}_p\left(V\right)$ and $q_p\left(V_{\mathrm{r}}^{-}\right)≔{lim}_{V\nearrow V_{\mathrm{r}}}{q}_p\left(V\right)$, an absorbing boundary at V_s,

$$p(V_{\mathrm{s}},t)=0,$$ (31)

and a natural (reflecting) boundary condition,

$$\underset{V\to -\infty }{lim}{q}_p(V,t)=0.$$ (32)

Together with the initial membrane voltage distribution p(V, 0) = p₀(V) and mean adaptation current 〈w〉(0) = 0 the Fokker-Planck mean-field model is now completely specified.

Note that p(V, t) only reflects the proportion of neurons which are not refractory at time t, given by $P\left(t\right)={\int }_{-\infty }^{V_{\mathrm{s}}}p(v,t)dv=1-{\int }_{t-T_{\text{ref}}}^{t}r\left(s\right)ds$ (<1 for T_ref > 0 and r(t) > 0). The total probability density that the membrane voltage is V at time t is given by p(V, t) + p_ref(V, t) with refractory density p_ref(V, t) = (1 − P(t)) δ(V − V_r). At the end of the Methods section we describe how an (optional) spike shape extension for the aEIF model changes the calculation of p_ref and 〈V〉.

In practice we consider a finite reflecting lower barrier V_lb instead of negative infinite for the numerical solution (next section) and for the low-dimensional approximations of the Fokker-Planck PDE (cf. sections below). V_lb is chosen sufficiently small in order to not distort the free diffusion of the membrane voltage for values below the reset, i.e., V_lb ≪ V_r. The density p(V, t) is then supported on [V_lb, V_s] for each time t, and in all expressions above V → −∞ is replaced by V_lb.

Finite volume discretization

In this work we focus on low-dimensional approximations of the FP model. To obtain a reference for the reduced models it is, however, valuable to solve the (full) FP system, Eqs (24)–(32). Here we outline an accurate and robust method of solution that exploits the linear form of the FP model in contrast to previously described numerical schemes [61, 62] which both require rather small time steps due to the steeply increasing exponential current I_exp in the flux q_p close to the spike voltage V_s.

We first discretize the (finite) domain [V_lb, V_s] into N_V equidistant grid cells $[V_{m-\frac{1}{2}},V_{m+\frac{1}{2}}]$ with centers V_m (m = 1, …, N_V) that satisfy V₁ < V₂ < ⋯ < V_NV, where $V_{\frac{1}{2}}=V_{\text{lb}}$ and $V_{N_V+\frac{1}{2}}=V_{\mathrm{s}}$ are the outmost cell borders. Within each cell the numerical approximation of p(V, t) is assumed to be constant and corresponds to the average value denoted by p(V_m, t). Integrating Eq (24) combined with Eq (30) over the volume of cell m, and applying the divergence theorem, yields

$$\frac{\partial }{\partial t}p(V_m,t)=\frac{q_p(V_{m-\frac{1}{2}},t)-q_p(V_{m+\frac{1}{2}},t)}{\Delta V}+{\delta }_{m{m}_{\mathrm{r}}}\frac{1}{\Delta V}{q}_p(V_{N_V+\frac{1}{2}},t-T_{\text{ref}}),$$ (33)

where ΔV is the grid spacing and m_r corresponds to the index of the cell that contains the reset voltage V_r. To solve Eq (33) forward in time the fluxes at the borders of each cell need to be approximated. Since the Fokker-Planck PDE belongs to the class of drift-diffusion equations this can be accurately achieved by the first order Scharfetter-Gummel flux [63, 64],

$$q_p(V_{m+\frac{1}{2}},t)=v_{m+\frac{1}{2}}\frac{p(V_m,t)-p(V_{m+1},t)exp(-v_{m+\frac{1}{2}}\Delta V/D)}{1-exp(-v_{m+\frac{1}{2}}\Delta V/D)},$$ (34)

where $v_{m+\frac{1}{2}}\left(t\right)=[I_{\mathrm{L}}\left(V_{m+\frac{1}{2}}\right)+I_{\text{exp}}\left(V_{m+\frac{1}{2}}\right)]/C+{\mu }_{\text{tot}}(t,r_d\left(t\right),\langle w\rangle \left(t\right))$ and $D\left(t\right)=\frac{1}{2}{\sigma }_{\text{tot}}^2(t,r_d\left(t\right))$ denote the drift and diffusion coefficients, respectively (cf. Eq (25)). This exponentially fitted scheme [64] is globally first order convergent [65] and yields for large drifts, $|v_{m+\frac{1}{2}}|\Delta V\gg D$, the upwind flux, sharing its stability properties. For vanishing drifts, on the other hand, the centered difference method is recovered [64], leading to more accurate solutions than the upwind scheme in regimes of strong diffusion.

For the time discretization we rewrite Eq (33) (with Eq (34)) in vectorized form and approximate the involved time derivative as first order backward difference to ensure numerical stability. This yields in each time step of length Δt a linear system for the values pⁿ⁺¹ of the (discretized) probability density at t_n+1, given the values pⁿ at the previous time step t_n, and the spike rate at the time t_n+1−nref for which the refractory period has just passed,

$$(\mathbf{I}-\frac{\Delta t}{\Delta V}{\mathbf{G}}^n){\mathbf{p}}^{n+1}={\mathbf{p}}^n+{\mathit{g}}^{n+1-n_{\text{ref}}},$$ (35)

with vector elements $p_m^n=p(V_m,t_n)$, m = 1, …, N_V, and $g_m^{n+1-n_{\text{ref}}}={\delta }_{m{m}_{\mathrm{r}}}\frac{\Delta t}{\Delta V}r\left(t_{n+1-n_{\text{ref}}}\right)$. The refractory period in time steps is given by n_ref = ⌈T_ref/Δt⌉, where the brackets denote the ceiling function, and I is the identity matrix. This linear equation can be efficiently solved with runtime complexity $\mathcal{O}\left(N_V\right)$ due to the tridiagonal structure of ${\mathbf{G}}^n\in {\mathbb{R}}^{N_V\times N_V}$ which contains the discretization of the membrane voltage (cf. Eqs (33) and (34)), including the absorbing and reflecting boundary conditions (Eqs (31) and (32)). For details we refer to S1 Text.

The spike rate, Eq (29), in this representation is obtained by evaluating the Scharfetter-Gummel flux, Eq (34), at the spike voltage V_s, taking into account the absorbing boundary condition, Eq (31), and introducing an auxiliary ghost cell [66], with center V_NV+1, which yields

$$r\left(t_{n+1}\right)=q_p(V_{N_V+\frac{1}{2}},t_{n+1})=v_{N_V+\frac{1}{2}}\frac{1+exp(-v_{N_V+\frac{1}{2}}\Delta V/D)}{1-exp(-v_{N_V+\frac{1}{2}}\Delta V/D)}{p}_{N_V}^{n+1},$$ (36)

where the drift and diffusion coefficients, $v_{N_V+\frac{1}{2}}$ and D, are evaluated at t_n. The mean membrane voltage (of non-refractory neurons), Eq (28), used for the dynamics of the mean adaptation current, Eq (23), is calculated by $\langle V\rangle \left(t_n\right)={\sum }_{m=1}^{N_V}{V}_m{p}_m^n/{\sum }_{m=1}^{N_V}{p}_m^n$.

Practically, we use the initialization $p_m^0=p_0\left(V_m\right)$ and solve in each time step the linear system, Eq (35), using the function banded_solve from the Python library SciPy [67]. Note that (for a recurrent network or time-varying external input) the tridiagonal matrix Gⁿ has to be constructed in each time step t_n, which can be time consuming–especially for small ΔV and/or small Δt. Therefore, we employ low-level virtual machine acceleration for this task through the Python package Numba [24] which yields an efficient implementation.

Remark: for a vanishing refractory period T_ref = 0 the matrix Gⁿ would lose its tridiagonal structure due to the instantaneous reinjection, cf. Eq (36). In this case we enforce a minimal refractory period of one time step, T_ref = Δt, which is an excellent approximation if the time step is chosen sufficiently small and the spike rate does not exceed biologically plausible values.

Low-dimensional approximations

In the following sections we present two approaches of how simple spike rate models can be derived from the Fokker-Planck mean-field model described in the previous section, cf. Eqs (20), (21) and (23)–(32).

The derived models are described by low-dimensional ordinary differential equations (ODEs) which depend on a number of quantities defined in the plane of (generic) input mean and standard deviation (μ, σ). To explain this concept more clearly we consider, as an example, the steady-state spike rate, which is a quantity required by all reduced models. The steady-state spike rate as a function of μ and σ,

$$r_{\infty }(\mu ,\sigma )≔\underset{t\to \infty }{lim}r(t;{\mu }_{\text{tot}}=\mu ,{\sigma }_{\text{tot}}=\sigma ),$$ (37)

denotes the stationary value of Eq (29) under replacement of the (time-varying) total moments μ_tot and ${\sigma }_{\text{tot}}^2$ in the probability flux q_p, Eq (25), by (constants) μ and σ², respectively. Thus the steady-state spike rate r_∞ effectively corresponds to that of an uncoupled EIF population whose membrane voltage is governed by dV_i/dt = [I_L(V_i) + I_exp(V_i)]/C + μ + σξ_i(t) plus reset condition, i.e., adaptation and synaptic current dynamics are detached. For a visualization of r_∞(μ, σ) see Fig 6.

Fig 6. Steady-state spike rate and mean membrane voltage for a population of EIF neurons.

r_∞ and 〈V〉_∞ for an uncoupled population of EIF neurons (aEIF with a = b = 0) as a function of (generic) input mean μ and standard deviation σ, calculated from the (steady-state) Fokker-Planck equation, shown in two different representations (left and right, each).

When simulating the reduced models these quantities need to be evaluated for each discrete time point t at a certain value of (μ, σ) which depends on the overall synaptic moments μ_syn(t), ${\sigma }_{\text{syn}}^2\left(t\right)$ and on the mean adaptation current 〈w〉(t) in a model-specific way (as described in the following Sects.). An example trajectory of r_∞ in the (μ, σ) space for a network showing stable spike rate oscillations is shown in Fig 5.

Importantly, these quantities depend on the parameters of synaptic input (J, K, τ_d, μ_ext, σ_ext) and adaptation current (a, b, τ_w, E_w) only through their arguments (μ, σ). Therefore, for given parameter values of the EIF model (C, g_L, E_L, Δ_T, V_T, V_r, T_ref) we precalculate those quantities on a (reasonably large and sufficiently dense) grid of μ and σ values, and access them during time integration by interpolating the quantity values stored in a table. This greatly reduces the computational complexity and enables rapid numerical simulations.

The derived low-dimensional models describe the spike rate dynamics and generally do not express the evolution of the entire membrane voltage distribution. Therefore, the mean adaptation dynamics, which depends on the density p(V, t) (via 〈V〉, cf. Eq (23)) is adjusted through approximating the mean membrane voltage 〈V〉 by the expectation over the steady-state distribution,

$${⟨V⟩}_{\infty }=\frac{{\int }_{-\infty }^{V_{\mathrm{s}}}v{p}_{\infty }\left(v\right)dv}{{\int }_{-\infty }^{V_{\mathrm{s}}}{p}_{\infty }\left(v\right)dv},$$ (38)

which is valid for sufficiently slow adaptation current dynamics [48, 58]. The steady-state distribution is defined as p_∞(V) = lim_{t → ∞} p(V, t; μ_tot = μ, σ_tot = σ), representing the stationary membrane voltages of an uncoupled EIF population for generic input mean μ and standard deviation σ. The mean adaptation current in all reduced models is thus governed by

$$\frac{d⟨w⟩}{dt}=\frac{a({⟨V⟩}_{\infty }-E_w)-⟨w⟩}{{\tau }_w}+br\left(t\right),$$ (39)

where the evaluation of quantity 〈V〉_∞ in terms of particular values for μ and σ at a given time t is model-specific (cf. following Sects.). Note again that the calculation of 〈V〉_∞ slightly changes when considering an (optional) spike shape extension for the aEIF model, as described at the end of the Methods section.

The Fokker-Planck model does not restrict the form of the delay distribution p_d, except that the convolution with the spike rate r, Eq (20), has to be well defined. Here, however, we aim at specifying the complete network dynamics in terms of a low-dimensional ODE system. Exploiting the exponential form of the delay distribution p_d we obtain a simple ordinary differential equation for the delayed spike rate,

$$\frac{d{r}_d}{dt}=\frac{r-r_d}{{\tau }_d},$$ (40)

which is equivalent to the convolution r_d = r * p_d.

Note that more generally any delay distribution from the exponential family allows to represent the delayed spike rate r_d by an equivalent ODE instead of a convolution integral [68]. Identical delays, r_d(t) = r(t − d), are also possible but lead to delay differential equations. Naturally, in case of no delays, we simply have r_d(t) = r(t).

To simulate the reduced models standard explicit time discretization schemes can be applied–directly to the first order equations of the LN_exp model, and for the other models (LN_dos, spec₁, spec₂)–to the respective equivalent (real) first order systems. We would like to note that when using the explicit Euler method to integrate any of the latter three low-dimensional models a sufficiently small integration time step Δt is required to prevent oscillatory artifacts. Although the explicit Euler method works well for the parameter values used in this contribution, we have additionally implemented the method of Heun, i.e., the explicit trapezoidal rule, which is second order accurate.

Spectral models

Eigendecomposition of the Fokker-Planck operator

Following and extending [18] we can specify the Fokker-Planck operator $\mathcal{L}$

$$\mathcal{L}(\mu ,\sigma )\left[p\right]=-\frac{\partial }{\partial V}\left[\right(\frac{I_{\mathrm{L}}\left(V\right)+I_{\text{exp}}\left(V\right)}{C}+\mu \left)p\right]+\frac{{\sigma }^2}{2}\frac{{\partial }^{2}p}{\partial V^2},$$ (41)

for an uncoupled EIF population receiving (constant) input (μ, σ), cf. Sect. Low-dimensional approximations. This operator allows to rearrange the FP dynamics of the recurrent aEIF network, Eq (24), as

$$\frac{\partial p}{\partial t}=\mathcal{L}({\mu }_{\text{tot}}\left(t\right),{\sigma }_{\text{tot}}\left(t\right))\left[p\right]$$ (42)

which depends on the (time-varying) total input moments μ_tot(t, r_d, 〈w〉) and ${\sigma }_{\text{tot}}^2(t,r_d)$, cf. Eqs (26) and (27), in the drift and diffusion coefficients, respectively.

For each value of (μ, σ) the operator $\mathcal{L}$ possesses an infinite, discrete set of eigenvalues λ_n in the left complex half-plane including zero [18], i.e., Re{λ_n} ≤ 0, and associated eigenfunctions ϕ_n(V) (n = 0, 1, 2, …) satisfying

$$\mathcal{L}\left[{\varphi }_n\right]={\lambda }_n{\varphi }_n.$$ (43)

Furthermore, the boundary conditions, Eqs (30)–(32) have to be fulfilled for each eigenfunction ϕ_n separately, i.e., the absorbing boundary at the spike voltage,

$${\varphi }_n\left(V_{\mathrm{s}}\right)=0,$$ (44)

and the reflecting barrier at the (finite) lower bound voltage,

$$q_{{\varphi }_n}\left(V_{\text{lb}}\right)=0,$$ (45)

must hold. The eigenflux is given by

$$q_{{\varphi }_n}\left(V\right)=(\frac{I_{\mathrm{L}}\left(V\right)+I_{\text{exp}}\left(V\right)}{C}+\mu ){\varphi }_n\left(V\right)-\frac{{\sigma }^2}{2}\frac{\partial }{\partial V}{\varphi }_n\left(V\right),$$ (46)

i.e., the flux q_p of Eq (25) with eigenfunction ϕ_n and (constant) generic input moments (μ, σ²) instead of density p and (time-varying) total input moments, respectively. Moreover, the eigenflux q_ϕn has to be reinjected into the reset voltage, cf. Eq (30),

$$q_{{\varphi }_n}\left(V_{\mathrm{r}}^{+}\right)-q_{{\varphi }_n}\left(V_{\mathrm{r}}^{-}\right)=q_{{\varphi }_n}\left(V_{\mathrm{s}}\right),$$ (47)

where we have neglected the refractory period, i.e., T_ref = 0. Note that incorporating a refractory period T_ref > 0 is straightforward only for the simplified case of vanishing total input moment variations, ${\dot{\mu }}_{\text{tot}}\approx {\dot{\sigma }}_{\text{tot}}^2\approx 0$, which is described in the following section and is not captured here in general.

The spectrum of $\mathcal{L}$ is shown in Fig 7A and further discussed in Sect. Remarks on the spectrum.

Fig 7. Spectrum of the Fokker-Planck operator $\mathcal{L}$ and related quantities.

A: regular eigenvalues of $\mathcal{L}$ (blue) and diffusive ones (red) with real and imaginary part (top and bottom, respectively) as a function of the mean input μ for small noise intensity σ (left) and larger input fluctuations (right). The first two dominant eigenvalues λ₁, λ₂ are indicated together with discontinuities in the real (λ₂) and imaginary part (λ₁ and λ₂), respectively. The stationary eigenvalue λ₀ = 0 is shown in gray. Note that the value of the mean input μ at which the eigenvalue λ_n changes from real to complex values depends on the noise amplitude σ and the eigenvalue index n which is difficult to see in the figure. The narrow winding curves attached to the left side of the respective spectra represent the lower bound flux q(V_lb) for μ_min = −1.5 mV/ms as a function of (real) eigenvalue candidate λ. The flux axis has a logarithmic scale between the large ticks (absolute values between 10⁻¹⁰ and 10⁻² kHz) and is linear around the dashed zero value. The open circles denote the eigenvalues, i.e., those λ that satisfy q(V_lb) = 0. Note that q(V_lb) ranges over several orders of magnitude. B: stationary eigenfunction ϕ₀ = p_∞ (gray) and nonstationary eigenfunctions ϕ₁ of $\mathcal{L}$ and ψ₁ of ${\mathcal{L}}^{*}$ corresponding to the first dominant eigenvalue λ₁ for three different input parameter values indicated by the triangle and circles in A (same units of μ and σ as therein). The eigenfunctions are biorthonormalized, but (only) for visualization truncated at V = −100 mV and furthermore individually scaled to absolutely range within the unit interval of arbitrary units [a.u.]. C: first and second dominant eigenvalues λ₁, λ₂ with real and imaginary part (that are also indicated in A), as well as additional (real-valued) quantities of the model spec₂ (M, S, F_μ, F_σ²) as a function of input mean μ and noise strength σ in steps of $0.2\text{mV}/\sqrt{\text{ms}}$ from small values (black) to larger ones (green). The dots indicate identical parameter values to the spectra of A and the eigenfunctions of B (darker: $\sigma =1.5\text{mV}/\sqrt{\text{ms}}$, brighter green: $\sigma =3.5\text{mV}/\sqrt{\text{ms}}$).

Defining the non-conjugated [69] inner product $\langle \psi ,\varphi \rangle ={\int }_{V_{\text{lb}}}^{V_{\mathrm{s}}}\psi \left(v\right)\varphi \left(v\right)dv$ yields the corresponding adjoint operator ${\mathcal{L}}^{*}(\mu ,\sigma )$ given by [18]

$${\mathcal{L}}^{*}=(\frac{I_{\mathrm{L}}\left(V\right)+I_{\text{exp}}\left(V\right)}{C}+\mu )\frac{\partial }{\partial V}+\frac{{\sigma }^2}{2}\frac{{\partial }^2}{\partial V^2},$$ (48)

which satisfies $\langle \psi ,\mathcal{L}\varphi \rangle =\langle {\mathcal{L}}^{*}\psi ,\varphi \rangle$ for any complex-valued functions ψ and ϕ that are sufficiently smooth on [V_lb, V_s]. ${\mathcal{L}}^{*}$ has the same set of eigenvalues λ_n as $\mathcal{L}$ but distinct associated eigenfunctions ψ_n(V), i.e.,

$${\mathcal{L}}^{*}\left[{\psi }_n\right]={\lambda }_n{\psi }_n,$$ (49)

which have to satisfy three boundary conditions,

$${\psi }_n\left(V_{\mathrm{s}}\right)={\psi }_n\left(V_{\mathrm{r}}\right),$$ (50)

$$\frac{\partial {\psi }_n}{\partial V}\left(V_{\text{lb}}\right)=0,$$ (51)

$$\frac{\partial {\psi }_n}{\partial V}\left(V_{\mathrm{r}}^{+}\right)=\frac{\partial {\psi }_n}{\partial V}\left(V_{\mathrm{r}}^{-}\right)$$ (52)

that are determined by integrating $\langle {\psi }_n,\mathcal{L}\left[{\varphi }_n\right]\rangle$ by parts and equalling with $\langle {\mathcal{L}}^{*}\left[{\psi }_n\right],{\varphi }_n\rangle$ using the conditions of $\mathcal{L}$, Eqs (44)–(47). Note that the last condition, Eq (52), ensures a continuous derivative of ψ_n at V_r in contrast to the eigenfunctions ϕ_n of $\mathcal{L}$, that have a kink at the reset due to reinjection condition, Eq (47), as shown in Fig 7B.

The eigenfunctions of $\mathcal{L}$ and ${\mathcal{L}}^{*}$ are pairwise orthogonal and in the following (without loss of generality) assumed to be scaled according to the biorthonormality condition,

$$⟨{\psi }_n,{\varphi }_m⟩={\delta }_{nm}.$$ (53)

The membrane voltage probability density can now be expanded onto the (moving) eigenbasis of $\mathcal{L}$ [18, 70],

$$p(V,t)=\sum _{n=0}^{\infty }{\alpha }_n\left(t\right){\varphi }_n\left(V\right),$$ (54)

where each eigenfunction ϕ_n depends on time via the total input moments μ = μ_tot(t, r_d, 〈w〉), ${\sigma }^2={\sigma }_{\text{tot}}^2(t,r_d)$, and the projection coefficients are given by α_n = 〈ψ_n, p〉 and particularly α₀ = 1 [18]. Deriving α_n with respect to time (for n = 1, 2, …) and using the expansion, Eq (54), the Fokker-Planck Eq (42) as well as the definition of the adjoint operator yields an infinite-dimensional equation for the complex-valued projection coefficients α(t) = (α₁(t), α₂(t), …)^T,

$$\dot{\mathit{\alpha }}=(\mathbf{\Lambda }+{\mathbf{C}}_{\mu }{\dot{\mu }}_{\text{tot}}+{\mathbf{C}}_{{\sigma }^2}{\dot{\sigma }}_{\text{tot}}^2)\mathit{\alpha }+{\mathbf{c}}_{\mu }{\dot{\mu }}_{\text{tot}}+{\mathbf{c}}_{{\sigma }^2}{\dot{\sigma }}_{\text{tot}}^2.$$ (55)

This dynamics is initialized by α_n(0) = 〈ψ_n, p₀〉, and is complemented by (i) an expression for the spike rate,

$$r\left(t\right)=r_{\infty }+\mathbf{f}·\mathit{\alpha },$$ (56)

that is obtained from Eq (29) (using Eq (54)), and (ii) by the mean adaptation and delayed spike rate dynamics, Eqs (39) and (40). The dots in Eqs (55) and (56) denote time derivatives (e.g., ${\dot{\sigma }}_{\text{tot}}^2=d{\sigma }_{\text{tot}}^2/dt$) and a non-conjugated scalar product of complex vectors (i.e., $\mathbf{f}·\mathit{\alpha }={\sum }_{n=1}^{\infty }{f}_n{\alpha }_n$), respectively. The matrix Λ = diag(λ₁, λ₂, …) contains the eigenvalues of $\mathcal{L}$, the matrices C_μ and C_σ² have elements (C_x)_n,m = 〈∂_xψ_n, ϕ_m〉 for x ∈ {μ, σ²} and n, $m\in \mathbb{N}$ with partial derivative ∂_x = ∂/∂x, the vectors c_μ and c_σ² consist of components $c_n^x=\langle {\partial }_x{\psi }_n,{\varphi }_0\rangle$. The steady-state spike rate is given by r_∞ = q_ϕ0(V_s), i.e., the flux of the eigenfunction ϕ₀ = p_∞ that represents the stationary membrane voltage distribution p_∞ and corresponds to the (stationary) eigenvalue λ₀ = 0 [18]. The vector f contains the (nonstationary) eigenfluxes evaluated at the spike voltage, f_n = q_ϕn(V_s).

Note that the quantities (Λ, C_μ, C_σ², c_μ, c_σ², r_∞, f, 〈V〉_∞) all depend on time in Eqs (55), (56) and (39) via the total input moments (μ_tot, ${\sigma }_{\text{tot}}^2$). Particularly, at time t the (biorthonormal) solution of the eigenvalue problems for $\mathcal{L}$ and its adjoint ${\mathcal{L}}^{*}$, Eqs (43)–(47) and (49)–(52) with $n\in {\mathbb{N}}_0$, is required for μ = μ_tot(t, r_d, 〈w〉), σ = σ_tot(t, r_d). Because $\mathcal{L}$ is a real operator its spectrum contains only real eigenvalues and/or complex conjugated pairs depending on (μ, σ). This property carries over to the eigenfunctions and therefore also to the components of all quantities above implying for example that the scalar product f · α is always real-valued.

Although the spectral representation, Eqs (55) and (56), is fully equivalent to the original (partial differential) Fokker-Planck equation without refractory period and while it contains only time derivatives it is still an infinite-dimensional and furthermore generally an implicit model. Therefore, to derive an explicit low-dimensional ordinary differential model for the spike rate r(t), it is not sufficient to truncate the expansion in Eq (54) after, e.g., two terms but additional assumptions have to be considered.

Basic model: One eigenvalue, negligible input variations

The first and simpler, derived spectral model is based on [19] and requires the strong assumption of vanishing changes of the total input moments, ${\dot{\mu }}_{\text{tot}}\approx 0$, ${\dot{\sigma }}_{\text{tot}}^2\approx 0$. Under this approximation the projection coefficient dynamics, Eq (55), simplifies to

$${\dot{\alpha }}_n={\lambda }_n{\alpha }_n$$ (57)

for $n\in \mathbb{N}$. Considering only the dominant (nonzero) eigenvalue

$${\lambda }_1=\text{argmin}\left\{\right|\text{Re}\left\{{\lambda }_n\right\}|:{\lambda }_n\ne 0\},$$ (58)

i.e., the “slowest mode”, we obtain from Eqs (56) and (57) that r(t) = r_∞ + m₁ Re{f₁α₁} with m₁ = 1 if ${\lambda }_1\in \mathbb{R}$ and m₁ = 2 if ${\lambda }_1\in \mathbb{C}\setminus \mathbb{R}$. Here we have included for a complex eigenvalue λ₁ also its complex conjugate ${\overline{\lambda }}_1$ that has the projection coefficient $\overline{{\alpha }_1}\left(t\right)$. They jointly yield a zero imaginary part in the scalar product of Eq (56). Defining $\tilde{r}\left(t\right)=r_{\infty }+m_1{f}_1{\alpha }_1$ yields the complex first order equation for the spike rate [19],

$$\dot{\tilde{r}}={\lambda }_1(\tilde{r}-r_{\infty }),r\left(t\right)=\text{Re}\left\{\tilde{r}\right\}.$$ (59)

While this derivation is based on neglecting changes of the total input moments, their time-variation is effectively reintroduced in the two quantities (dominant eigenvalue and steady-state rate), i.e., λ₁(μ_tot, σ_tot) and r_∞(μ_tot, σ_tot) with μ_tot(t) and σ_tot(t) according to Eqs (26) and (27). Therefore, the spike rate evolution, Eq (59), is complemented with the dynamics of mean adaptation current 〈w〉 and delayed spike rate r_d, Eqs (39) and (40), where the former involves the third (μ_tot, σ_tot)-dependent quantity 〈V〉_∞. See Figs 6 and 7C for the involved quantities depending on (generic) input μ, σ.

We call this first derived low-dimensional spike rate model, i.e., Eqs (59), (39) and (40), spec₁. It is very simple in comparison with the full Fokker-Planck system in the spectral representation, Eqs (55)–(56), (39) and (40), in the sense that it does not depend on “nonstationary” quantities of $\mathcal{L}$ or its adjoint ${\mathcal{L}}^{*}$ except for the dominant eigenvalue λ₁.

Note that under the assumption of vanishing input moment variations the dynamics of the expansion coefficients α_n, Eq (55), is simply exponentially decaying in time, cf. Eq (57). This allows to incorporate a refractory period T_ref > 0 into the spectral decomposition framework by inserting the eigenbasis expansion of the membrane voltage distribution, Eq (54), into the reinjection condition of the Fokker-Planck model, Eq (30), using α_n(t) = α_n(0) exp(λ_nt) and the absorbing boundary, Eq (44). This generalizes the reinjection condition for the eigenfunctions ϕ_n of $\mathcal{L}$ from Eq (47) to

$$q_{{\varphi }_n}\left(V_{\mathrm{r}}^{+}\right)-q_{{\varphi }_n}\left(V_{\mathrm{r}}^{-}\right)=q_{{\varphi }_n}\left(V_{\mathrm{s}}\right)exp(-{\lambda }_n{T}_{\text{ref}}),$$ (60)

which was applied in [19], and the corresponding boundary condition for the adjoint operator’s eigenfunctions ψ_n from Eq (50) to

$${\psi }_n\left(V_{\mathrm{s}}\right)={\psi }_n\left(V_{\mathrm{r}}\right)exp(-{\lambda }_n{T}_{\text{ref}}).$$ (61)

Advanced model: Two eigenvalues, slow input moment variations

Another possibility to derive a low-dimensional spike rate model is based on ongoing work of Maurizio Mattia. He has recognized that under the weaker (compared to the basic spec₁ model above) assumption of small but not vanishing input moment changes a real-valued second order ordinary differential equation for the spike rate r(t) can be consistently derived [20]. Here we extend this approach to account for neuronal adaptation, time-varying external input moments and delay distributions.

The steps in a nutshell (for the detailed derivation see S1 Text) are (i) taking the derivative of Eq (55) (once) and of (56) (twice) w.r.t. time, (ii) considering only the first two dominant eigenvalues λ₁ and λ₂, i.e., neglecting all (faster) eigenmodes that correspond to eigenvalues with larger absolute real part (“modal approximation”), (iii) assuming slowly changing input moments, i.e., small ${\dot{\mu }}_{\text{tot}}$ and ${\dot{\sigma }}_{\text{tot}}^2$ that allow to consider projections coefficients of that order, i.e., ${\alpha }_n=\mathcal{O}\left({\dot{\mu }}_{\text{tot}}\right)$ and ${\alpha }_n=\mathcal{O}\left({\dot{\sigma }}_{\text{tot}}^2\right)$ for n = 1, 2, and therefore to consider only linear occurences of ${\dot{\mu }}_{\text{tot}}$, ${\dot{\sigma }}_{\text{tot}}^2$, α₁, α₂, and neglect terms of second and higher order. The slowness approximation implies that neither the external moments μ_ext(t), ${\sigma }_{\text{ext}}^2\left(t\right)$ nor the (delayed) spike rate r_d(t) nor the population-averaged adaptation current 〈w〉(t) should change very fast (cf. Eqs (26), (27) and (21). Note that the dynamics of 〈w〉 is assumed to be slow already, cf. Sect. Fokker-Planck system.

Under these approximations we obtain for the spike rate dynamics the following real second order ODE,

$${\beta }_2\ddot{r}+{\beta }_1\dot{r}+{\beta }_{0}r=r_{\infty }-r-{\beta }_c,$$ (62)

that is complemented with the mean adaptation current and delayed spike rate dynamics, Eqs (39) and (40), and we call the model spec₂. The coefficients

$${\beta }_2=D,$$ (63)

$${\beta }_1=-T+DM\frac{b}{C}-\frac{R}{{\tau }_d},$$ (64)

$${\beta }_0=-DM\frac{b}{{\tau }_{w}C}-\frac{b}{C}{H}_{\mu }+\frac{1}{{\tau }_d}(KJ{H}_{\mu }+K{J}^2{H}_{{\sigma }^2})+\frac{R}{{\tau }_d^2},$$ (65)

$$\begin{array}{ccc}\hfill {\beta }_{\text{c}}& \hfill =\hfill & r_d(-\frac{1}{{\tau }_d}(KJ{H}_{\mu }+K{J}^2{H}_{{\sigma }^2})-\frac{R}{{\tau }_d^2})\hfill \\ & -& ({\ddot{\mu }}_{\text{ext}}+\frac{a({⟨V⟩}_{\infty }-E_w)-⟨w⟩}{{\tau }_w^{2}C})DM-\ddot{{\sigma }_{\text{ext}}^2}DS\hfill \\ & +& ({\dot{\mu }}_{\text{ext}}-\frac{a({⟨V⟩}_{\infty }-E_w)-⟨w⟩}{{\tau }_{w}C})H_{\mu }+\dot{{\sigma }_{\text{ext}}^2}{H}_{{\sigma }^2},\hfill \end{array}$$ (66)

depend on the (lumped) quantities D = 1/λ₁ ⋅ 1/λ₂, T = 1/λ₁ + 1/λ₂, M = ∂_μr_∞ + f ⋅ c_μ, S = ∂_σ²r_∞ + f ⋅ c_σ², R = DMKJ + DSKJ², H_μ = TM + DMa∂_μ〈V〉_∞/(τ_wC) − DF_μ, H_σ² = TS + DMa∂_σ²〈V〉_∞/(τ_wC) − DF_σ², F_μ = f ⋅ Λ c_μ and F_σ² = f ⋅ Λ c_σ². Here the diagonal eigenvalue matrix Λ = diag(λ₁, λ₂) and the vectors f = (f₁, f₂)^T, ${\mathbf{c}}_{\mu }={(c_1^{\mu },c_2^{\mu })}^T$, ${\mathbf{c}}_{{\sigma }^2}={(c_1^{{\sigma }^2},c_2^{{\sigma }^2})}^T$ are two-dimensional in contrast to infinite as in the original dynamics, Eqs (55) and (56). These individual quantities, that also include 〈V〉_∞ (cf. Eq (39)), r_∞ and derivatives of both w.r.t. generic mean μ and variance σ², are evaluated at the total input moments μ = μ_tot(t, r_d, 〈w〉), ${\sigma }^2={\sigma }_{\text{tot}}^2(t,r_d)$. A relevant subset of individual and lumped quantities is shown in Figs 6 and 7C.

The four coefficients, Eqs (63)–(66), are real-valued because we define λ₂ as the second dominant eigenvalue conditioned that (λ₁, λ₂) compose either a real or a complex conjugated eigenvalue pair, i.e.,

$${\lambda }_2=\text{argmin}\left\{\right|\text{Re}\left\{{\lambda }_n\right\}|:{\lambda }_n\ne {\lambda }_1\text{s.t.}{\lambda }_n\ne 0,{\lambda }_1+{\lambda }_n\in \mathbb{R}\},$$ (67)

where λ₁ is obtained as for the (basic) spec₁ model, cf. Eq (58). This condition ensures that all related complex quantities occur in vectors of two complex conjugate components (for example, $f_1=\overline{f_2}$) implying that the scalar products above are real-valued (e.g., $\mathbf{f}·{\mathbf{c}}_{\mu }\in \mathbb{R}$) and therefore all (nine) lumped quantities, too. Note that this specific definition of λ₂ is required only for integrate-and-fire neuron models that have a lower bound different from the reset, i.e., V_lb < V_r, as discussed in the following section.

The coefficients of the spec₂ model require–in addition to eigenvalues λ_n and steady-state rate r_∞ (cf. basic spec₁ model, Eq (59))–quantities that involve the first and second eigenfunctions of the Fokker-Planck operator $\mathcal{L}$ and its adjoint ${\mathcal{L}}^{*}$. Additionally, β₁, β₀ and β_c contain the parameters of membrane voltage (C) and mean adaptation current dynamics (a, b, τ_w, E_w) as well as of the recurrent coupling (K, J, τ_d) and, importantly, explicit dependencies on the population-averaged adaptation current 〈w〉 and the delayed spike rate r_d (that is in addition to implicitly via μ_tot and σ_tot). Furthermore β_c depends on the first two time derivatives of the external input moments μ_ext(t) and ${\sigma }_{\text{ext}}^2\left(t\right)$. This explicit occurence of neuronal and coupling parameters, state variables and input moment derivatives in the coefficients is not expressed in the basic spectral model (spec₁), Eq (59). A consequence is that for the (advanced) model spec₂ the external moments have to be provided twice differentiable or in case of non-smooth time series to be filtered (e.g., see Sect. Performance for variations of the mean input).

The particular coefficients, Eqs (63)–(66), are specific for the choice of an exponential delay distribution, as indicated by the occurrence of the mean delay τ_d in β₁, β₀ and β_c. Other choices such as identical delays or no delays are described in the supplementary material S1 Text.

Note that the original (infinite-dimensional) spectral dynamics, Eqs (55) and (56), assumes for the refractory period a value of T_ref = 0 which carries over to the same choice for the spec₂ model, whereas T_ref > 0 is not supported (yet).

The spike rate is by definition nonnegative, however, the model spec₂, Eq (62), can yield negative rates r(t), especially for small total mean input μ_tot and fast (external) input changes, e.g., when ${\dot{\mu }}_{\text{ext}}$ is large. We explicitly avoid that behaviour by setting both the spike rate of this model, r, and its derivative, $\dot{r}$, to zero whenever r(t)<0 and continue the integration of the differential equation afterwards.

Remarks on the spectrum

In the previous two sections we have developed two spike rate models based on approximations of the Fokker-Planck system’s spectral representation, Eqs (55) and (56) under different slowness assumptions. In the derivation of the (simple) model spec₁, cf. Eq (59), temporal variations of the total input moments are completely neglected while the (advanced) model spec₂, cf. Eq (62), incorporates (slow) changes of the total input moments through linear terms (proportional to ${\dot{\mu }}_{\text{tot}}$ or ${\dot{\sigma }}_{\text{tot}}^2$) and neglects (faster) quadratic and higher order ones.

Both slowness approximations imply that the eigenvalue matrix Λ is the dominant term in the homogeneous part of Eq (55). Therefore the eigenvalues approximately correspond to decay time constants 1/|Re{λ_n}| in their real part and in case of complex eigenvalues they contribute damped oscillatory components with frequency |Im{λ_n}|/(2π) through their imaginary part to the dynamics. How the spectrum of the Fokker-Planck operator $\mathcal{L}$ depends on the input moments therefore gives insights into the behavior of the derived spike rate models spec₁ and spec₂, and of the FP model with slow adaptation in general.

Here we summarize the main properties of the eigenvalues λ_n(μ, σ) for the (uncoupled, nonadaptive) EIF neuron model as a function of generic input mean μ and standard deviation σ (cf. Sect. Low-dimensional approximations) that are shown in Fig 7A. Note that at time t the total input determines the particularly effective eigenvalue, λ_n(μ_tot(t), σ_tot(t)), in dependence of external input moments, (delayed) spike rate and adaptation current, cf. Eqs (26), (27) and (21. We thereby extend the findings of [18] concerning the perfect integrate-and-fire neuron with reflecting lower barrier at the reset voltage (PIFb), i.e., dV_i/dt = μ + σξ_i(t) with V_lb = V_r.

1. The eigenvalue λ₀ = 0 exists for all (generic) input moments μ, σ² with stationary membrane voltage distribution as corresponding eigenfunction, ϕ₀ = p_∞ (see Fig 7A and 7B), the respective adjoint eigenfunction is constant, ψ₀ = 1, cf. [18]. The other, nonstationary eigenvalues λ_n with n ≥ 1 have negative real parts for all (μ, σ) which yields stability of the stationary distribution for constant total input moments. For sufficiently small mean input μ_min the eigenvalues λ_n(μ_min, σ) are real-valued for all n and σ, i.e., no (damped) oscillatory spike rate transient in that regime are present (consistent with [18]), cf. Fig 7A.

2. Two classes of modes can be distinguished: eigenvalues of the first kind occur in couples which are real-valued at μ_min but merge for increasing mean input μ to become a complex conjugated pair of decreasing absolute real part and almost linearly increasing imaginary part (see Fig 7A). Thus they correspond in this situation to damped oscillatory dynamics with increased frequency |Im{λ_n}|/(2π) and decay time constant 1/|Re{λ_n}| for stronger mean input. Note that there is no single critical parameter for the real-to-complex transition, instead that depends on the input mean μ and standard deviation σ as well as on the eigenvalue index n in contrast to the PIFb neuron where μ = 0 (alone) induces the transition [18]. We call this type of eigenvalue regular because it is observed also for very simple integrate-and-fire models (such as PIFb).

   The second type of eigenvalue is real for the whole input parameter space of (μ, σ). The corresponding decay time constant is large if noise dominates while for stronger mean input the respective dynamics is negligibly fast. Note that by setting the lower bound V_lb equal to the reset voltage, this eigenvalue class is completely removed for the EIFb model, which is the EIF membrane voltage description with V_lb = V_r (not used here). This explains why this new type of eigenvalue has not been described in [18], and suggests a relationship to the diffusion of the membrane voltage for hyperpolarized neuronal states. The latter correspondence is further supported from the significant values of the respective eigenfunction ϕ_n below the reset voltage V_r in contrast to the eigenfunctions of regular eigenvalues (cf. Fig 7B). Therefore we label this second type of eigenvalue diffusive.

3. The input noise intensity σ controls the spectrum’s mixture of the two eigenvalue classes as follows: weak noise favors regular modes, i.e., the dominant two eigenvalues are pairs of real (for smaller mean input μ) or complex conjugated eigenvalues (for larger μ) while the diffusive modes are irrelevantly fast in this regime (cf. Fig 7A, left). Increased input fluctuations, i.e., a larger σ, on the other hand leads to a spectrum with dominant (“slowest” eigenvalue λ₁) of the diffusive kind for small mean input μ while for larger μ the dominant mode is from the regular class (being real or complex depending on μ, σ, n), see Fig 7A, right. Furthermore an increased noise strength σ leads to a smaller decay time constant 1/|Re{λ_n}| for regular modes, i.e., their respective contribution to the spike rate dynamics is faster in the fluctuation-dominated regime than in the drift-dominated one, whereas for diffusive modes the opposite holds true.

The specific definition of the second dominant eigenvalue λ₂, Eq (67), is necessary to ensure real coefficients in the spec₂ model, Eqs (63)–(66), for regions in (μ, σ)-space where the first dominant eigenvalue λ₁ is of the (real) diffusive type and the second is part of a complex conjugate couple (for an example see Fig 7A, right).

Extracting the required dominant eigenvalues λ₁ (for both spectral models: spec₁, spec₂) and λ₂ (for spec₂ model only) according to Eqs (58) and (67) in the (μ, σ)-plane leads to points of instantaneous changes in the imaginary part (λ₁ and λ₂) and the real part (only λ₂) due to transitions from a dominant diffusive mode for small mean input μ to a regular eigenvalue (or pair) becoming dominant for larger μ (see Fig 7A and 7C and the last property above). These discontinuities (which lie on a one-dimensional curve μ(σ)) could be avoided by either restricting to V_lb = V_r (no diffusive modes) or by deriving a third spectral model based on three eigenvalues: the dominant regular pair together with the dominant diffusive eigenvalue. Making the latter extension is straightforward by using the same steps and slowness approximation as for the model spec₂ and would yield a 3rd order ODE for the spike rate r(t) with smooth coefficients and is expected to show increased reproduction accuracy (especially for small mean input) compared to the model spec₂.

The properties above imply for the low-dimensional models spec₁ and spec₂, that both enable damped oscillatory spike rates for mean-dominated input (for large μ) since then the first two dominant eigenvalues (λ₁ and λ₂) have nonzero imaginary parts and are complex conjugates of each other (see Fig 7C). Furthermore in this case the effective time constant, 1/|Re{λ₁}| = 1/|Re{λ₂}|, is large (especially for small input standard deviation σ). For noise-dominated input, i.e., when σ (and not μ) is large, on the other hand, the corresponding spike rate dynamics is fast and does not contain an oscillatory component.

Numerical solver

Here we present a numerical solution method of the Fokker-Planck boundary eigenvalue problem (BEVP) for the operator $\mathcal{L}(\mu ,\sigma )$, Eqs (43)–(47), and its adjoint ${\mathcal{L}}^{*}(\mu ,\sigma )$, Eqs (49)–(52). The solution of the two BEVPs in terms of eigenvalues λ_n and associated (biorthonormal) eigenfunctions (ϕ_n(V), ψ_n(V)) as well as quantities that are derived from those and are required by the models spec₁ and spec₂, is obtained for a rectangle of the input mean μ and standard deviation σ (see Sect. Low-dimensional Approximations). Note that the numerical method is not restricted to the EIF neuron model and supports other integrate-and-fire models as well (e.g., perfect, leaky or quadratic).

The eigenequation $\mathcal{L}(\mu ,\sigma )\left[\varphi \right]=\lambda \varphi$, i.e., Eq (43) (omitting the eigenvalue index n), represents a second order ODE (cf. Eq (41)), that is equivalent to the following first order system for the eigenflux q_ϕ(V) and the eigenfunction ϕ(V),

$$-\frac{d}{dV}\left(\begin{array}{c}{q}_{\varphi }\\ \varphi \end{array}\right)=\underset{=\text{A}}{\underbrace{\left(\begin{array}{cc}0& \lambda \\ \frac{2}{{\sigma }^2}& -2\frac{g\left(V\right)+\mu }{{\sigma }^2}\end{array}\right)}}\left(\begin{array}{c}{q}_{\varphi }\\ \varphi \end{array}\right)$$ (68)

with coefficient matrix A(V) that has a nonlinear component through g(V) = [I_L(V) + I_exp(V)]/C that contains leak and exponential (membrane) currents. Here it was used the form of the eigenflux q_ϕ (cf. Eq (46)), and that $\mathcal{L}(\mu ,\sigma )\left[\varphi \right]=-{\partial }_V{q}_{\varphi }$ for generic input moments μ and σ² (cf. Eq (41)).

Basically a direct discretization, e.g., by a finite difference approximation, of the membrane voltage derivatives in the System (68) can be applied. In combination with the boundary conds., Eqs (44)–(47), this leads to a (sparse) matrix eigenvalue problem that allows for application of standard (Arnoldi iteration based) numerical solvers. However, the convergence properties of this approach are very poor in the sense that extremely small voltage steps ΔV have to be chosen for the finite differences. Thus, the technique is inefficient as huge systems appear or even inaccurate due to amplified round-off errors by ill-conditioned matrices.

Here we propose an alternative solution procedure which is based on a reformulation of the System (68) with corresponding boundary conditions, Eqs (44)–(47), as a complex-valued algebraic root finding problem

$$\lambda \mapsto q_{\varphi }(V_{\text{lb}};\lambda )\stackrel{!}{=}0$$ (69)

whose solutions are the eigenvalues λ_n. To evaluate the nonlinear function (the left hand side of this equation) for an arbitrary $\lambda \in \mathbb{C}$ (not necessary an eigenvalue) Eq (68) is integrated backward starting from the spike voltage V_s (initializing one component to satisfy the absorbing boundary cond., Eq (44), i.e., ϕ(V_s) = 0, and another component that can be chosen arbitrarily, $q_{\varphi }\left(V_{\mathrm{s}}\right)\in \mathbb{C}\setminus \left\{0\right\}$, due to the linearity of the problem) via the reset V_r (where the reinjection cond., Eq (47), is enforced, i.e., $q_{\varphi }\left(V_{\mathrm{r}}^{-}\right)=q_{\varphi }\left(V_{\mathrm{r}}^{+}\right)-q_{\varphi }\left(V_{\mathrm{s}}\right)$, which induces a discontinuity in q_ϕ at the reset voltage) finally to the lower bound voltage V_lb. There, a nonzero value of $q_{\varphi }\left(V_{\text{lb}}\right)\in \mathbb{C}$ indicates that λ is not an eigenvalue since in this case ϕ(V) violates the reflecting boundary condition, Eq (45). q_ϕ(V_lb) = 0 on the other hand shows that λ is an eigenvalue with corresponding eigenfunction ϕ(V) and eigenflux q_ϕ(V). Note that when considering a nonzero refractory period the generalized version of the reinjection cond., Eq (60), i.e., $q_{\varphi }\left(V_{\mathrm{r}}^{-}\right)=q_{\varphi }\left(V_{\mathrm{r}}^{+}\right)-q_{\varphi }\left(V_{\mathrm{s}}\right)e^{-\lambda T_{\text{ref}}}$, is enforced instead of the expression above. The latter is only valid for the spec₁ model, Eq (59) (see Sect. Basic model: one eigenvalue, negligible input variations) and makes the Fokker-Planck eigenvalue problem nonlinear due to the exponentiation of λ in Eq (60).

The (complex-valued) root finding problem, Eq (69) can be solved numerically to yield a target eigenvalue λ_n using an iterative procedure (for example a variant of Newton’s method) given that a sufficiently close initial approximation ${\tilde{\lambda }}_n\in \mathbb{C}$ is available. In our Python implementation we apply Powell’s hybrid method as implemented in MINPACK wrapped through SciPy [67] to the equivalent real system of the two variables Re{λ} and Im{λ}.

Appropriate initial approximations ${\tilde{\lambda }}_n$ can be achieved (i) by exploiting that for sufficiently small generic mean input μ_min all eigenvalues have zero imaginary part (see Sect. Remarks on the spectrum). In that case the eigenvalues are given by the roots {λ₀, λ₁, …} of q(V_lb; λ)–the one-dimensional function of the real-valued eigenvalue candidate λ ∈ (−∞, 0]–which are obtained, for example, by (dense) evaluation of that function in a sufficiently large (sub)interval below zero. Furthermore (ii) all eigenvalues depend continuously on the (input) parameters μ and σ (cf. Fig 7A), i.e., for a small step in the respective parameter space the solution λ_n of Eq (69) for the last parametrization is a very good initial value ${\tilde{\lambda }}_n$ for the current parametrization. With this initialization the solution of Eq (69) is typically found in a few steps of the chosen Newton-like method (see S1 Text for details).

To efficiently and accurately evaluate q_ϕ(V_lb; λ) in each iteration (of the root finding algorithm), we perform an exponential (backward) integration of the System (68). The resulting scheme is based on truncating the Magnus expansion of the exact solution after one term [71]. Here matrix exponential function evaluations of the form exp[A(V)ΔV] occur that are calculated using an analytic expression [72]. Note that ODE solvers that either do not exploit the linear structure of Eq (68) at all (such as the explicit Euler method but also higher order Runge-Kutta methods) or utilize linearity only in one variable (e.g., [73]) have poor convergence behaviour and thus require very small step sizes ΔV due to the strong nonlinearity g(V) as a consequence of the large value of the current I_exp close to V_s and significantly large absolute eigenvalues |λ_n|, respectively.

For more information on the numerical solver we refer to S1 Text, where details regarding the adjoint operator, the exponential integration, the initial eigenvalue determination and the (parallelized) treatment of the input parameters are included.

Cascade models

Linear-Nonlinear (LN) cascade models of neuronal activity are often applied in neuroscience, because they are simple and efficient, and the model components can be estimated using established experimental procedures [21, 74, 75]. Here we use the LN cascade as an ansatz to develop a low-dimensional model and we determine its components from the underlying Fokker-Planck model. This section builds upon [21] and extends that approach for recurrently coupled aEIF neurons; specifically, by taking into account an adaptation current and variations of the input variance. Furthermore, we consider an improved approximation of the derived linear filters and include an (optional) explicit description of the spike shape, cf. [23] (ch 4.2).

The cascade models considered here produce spike rate output by applying to the time-varying mean μ_syn and standard deviation σ_syn of the (overall) synaptic input, cf. Eq (21), separately a linear temporal filter, D_μ and D_σ, followed by a common nonlinear function F. That is,

$$r\left(t\right)=F({\mu }_{\mathrm{f}},{\sigma }_{\mathrm{f}},⟨w⟩),$$ (70)

$${\mu }_{\mathrm{f}}\left(t\right)=D_{\mu }*{\mu }_{\text{syn}}\left(t\right),$$ (71)

$${\sigma }_{\mathrm{f}}\left(t\right)=D_{\sigma }*{\sigma }_{\text{syn}}\left(t\right),$$ (72)

where μ_f and σ_f denote the filtered mean and filtered standard deviation of the input, respectively. $D_{\mu }*{\mu }_{\text{syn}}\left(t\right)={\int }_0^{\infty }{D}_{\mu }\left(\tau \right){\mu }_{\text{syn}}(t-\tau )d\tau$ is the convolution between D_μ and μ_syn. The filters D_μ, D_σ are adaptive in the sense that they depend on the mean adaptation current 〈w〉 and on the (arbitrary) baseline input in terms of baseline mean ${\mu }_{\text{syn}}^0$ and standard deviation ${\sigma }_{\text{syn}}^0$. For improved readability these dependencies are not explicitly indicated in Eqs (71) and (72). Note, that the nonlinearity F also depends on 〈w〉, which is governed by Eq (23). Since the mean adaptation current depends on the mean membrane voltage 〈V〉 we also consider a nonlinear mapping H for that population output quantity,

$$⟨V⟩\left(t\right)=H({\mu }_{\mathrm{f}},{\sigma }_{\mathrm{f}},⟨w⟩).$$ (73)

For the derivation below it is instructive to first consider an uncoupled population, i.e., the input moments do not depend on r_d for now. In particular, the input statistics are described by ${\mu }_{\text{syn}}\left(t\right)={\mu }_{\text{syn}}^0+{\mu }_{\text{syn}}^1\left(t\right)$ and ${\sigma }_{\text{syn}}\left(t\right)={\sigma }_{\text{syn}}^0+{\sigma }_{\text{syn}}^1\left(t\right)$. In the following, we derive the components F, D_μ and D_σ from the Fokker-Planck model for small amplitude variations ${\mu }_{\text{syn}}^1$, ${\sigma }_{\text{syn}}^1$ and for a slowly varying adaptation current (as already assumed). We then approximate the derived linear filter components using suitable functions such that the convolutions can be expressed in terms of simple ODEs. Finally, we account for time-varying baseline input (${\mu }_{\text{syn}}^0\left(t\right)$, ${\sigma }_{\text{syn}}^0\left(t\right)$) and for recurrent coupling in the resulting low-dimensional spike rate models.

Deriving the components of the cascade

We first expand F in Eq (70) around the baseline ${\mu }_{\mathrm{f}}={\mu }_{\text{syn}}^0$, ${\sigma }_{\mathrm{f}}={\sigma }_{\text{syn}}^0$, 〈w〉 = 〈w〉₀ to linear order, assuming that the amplitudes of ${\mu }_{\text{syn}}^1$ and ${\sigma }_{\text{syn}}^1$ are small, and the mean adaptation current varies slowly compared to the input moments, to obtain the approximation for Eq (70),

$$\begin{array}{ccc}\hfill r\left(t\right)\approx F({\mu }_{\text{syn}}^0,{\sigma }_{\text{syn}}^0,⟨w⟩)& \hfill +\hfill & D_{\mu }*{\mu }_{\text{syn}}^1\left(t\right)\frac{\partial }{\partial \mu }F({\mu }_{\text{syn}}^0,{\sigma }_{\text{syn}}^0,⟨w⟩)\hfill \\ & +& D_{\sigma }*{\sigma }_{\text{syn}}^1\left(t\right)\frac{\partial }{\partial \sigma }F({\mu }_{\text{syn}}^0,{\sigma }_{\text{syn}}^0,⟨w⟩).\hfill \end{array}$$ (74)

Due to slow adaptation (〈w〉(t) = 〈w〉₀ + 〈w〉₁(t) with vanishing 〈w〉₁) we have neglected the expansion term in the direction of 〈w〉 and replaced 〈w〉₀ = 〈w〉 in the approximation above. Note also that, without loss of generality, we have assumed normalized filters, ${\int }_0^{\infty }{D}_{\mu }\left(\tau \right)d\tau ={\int }_0^{\infty }{D}_{\sigma }\left(\tau \right)d\tau =1$. Under the same assumptions the output from the Fokker-Planck model (Eqs (23)–(32)) can be approximated as

$$r\left(t\right)\approx r_{\infty }({\mu }_{\text{tot}}^0,{\sigma }_{\text{tot}}^0)+R_{\mu }*{\mu }_{\text{syn}}^1\left(t\right)+R_{\sigma }*{\sigma }_{\text{syn}}^1\left(t\right),$$ (75)

$$\frac{d⟨w⟩}{dt}\approx \frac{a({⟨V⟩}_{\infty }-E_w)-⟨w⟩}{{\tau }_w}+br\left(t\right),$$ (76)

$${\mu }_{\text{tot}}^0\left(t\right)={\mu }_{\text{syn}}^0-⟨w⟩/C,{\sigma }_{\text{tot}}^0={\sigma }_{\text{syn}}^0,$$ (77)

where r_∞ and 〈V〉_∞ are the steady-state spike rate and mean membrane voltage of a population of EIF neurons in response to an input of total mean ${\mu }_{\text{tot}}^0$ plus Gaussian white noise with standard deviation ${\sigma }_{\text{tot}}^0$. In particular, 〈V〉_∞ reflects the mean over all nonrefractory neurons, cf. Eq (38). R_μ and R_σ are the so-called linear rate response functions of the population for weak modulations of the input mean and standard deviation around ${\mu }_{\text{tot}}^0$ and ${\sigma }_{\text{tot}}^0$, respectively [21, 73, 76]. Comparing Eqs (74) and (75) we obtain for the nonlinearity F and the linear filters D_μ, D_σ,

$$F(\mu ,\sigma ,⟨w⟩)=r_{\infty }(\mu -⟨w⟩/C,\sigma ),$$ (78)

$$D_{\mu }\left(t\right)=\frac{R_{\mu }\left(t\right)}{\frac{\partial }{\partial \mu }{r}_{\infty }({\mu }_{\text{tot}}^0,{\sigma }_{\text{tot}}^0)},$$ (79)

$$D_{\sigma }\left(t\right)=\frac{R_{\sigma }\left(t\right)}{\frac{\partial }{\partial \sigma }{r}_{\infty }({\mu }_{\text{tot}}^0,{\sigma }_{\text{tot}}^0)}.$$ (80)

Furthermore, the function H in Eq (73) is given by

$$H(\mu ,\sigma ,⟨w⟩)={⟨V⟩}_{\infty }(\mu -⟨w⟩/C,\sigma ),$$ (81)

ensuring that the mean membrane voltage corresponds with the instantaneous spike rate estimate of the model (in every time step). Note that R_μ and R_σ depend on ${\mu }_{\text{tot}}^0$ and ${\sigma }_{\text{tot}}^0$ (which is again not explicitly indicated for improved readability).

Fortunately, the quantities r_∞, 〈V〉_∞, R_μ, and R_σ can be calculated from the Fokker-Planck equation using an efficient numerical method [73]. In particular, for r_∞ and 〈V〉_∞ we need to solve a linear boundary value problem (BVP), and R_μ(t), R_σ(t) are calculated in the Fourier domain, where we need to solve two linear BVPs to obtain ${\widehat{R}}_{\mu }\left(f\right)$, ${\widehat{R}}_{\sigma }\left(f\right)$ for each frequency f. It is worth noting that the refractory period is included in a straightforward way and does not increase the complexity of the BVPs to be solved, see [23, 73] (and our provided code).

Approximating the filter components

To express the LN model with adaptation, Eqs (70)–(73), (23) and (78)–(81), in terms of a low-dimensional ODE system we next approximate the linear filters D_μ and D_σ using suitable functions. The shapes of the true filters (proportional to R_μ and R_σ, cf. Eqs (79) and (80)) for different input parameter values ($\mu ={\mu }_{\text{tot}}^0$, $\sigma ={\sigma }_{\text{tot}}^0$) are shown in Fig 8A and 8B.

Fig 8. Linear rate response functions and quantities for the cascade models.

Linear rate response functions of EIF neurons subject to white noise input for modulations of the input mean around μ with constant σ: R_μ(t; μ, σ) in kHz/V (A, gray) and for modulations of the input standard deviation around σ with constant μ: R_σ(t; μ, σ) in $\text{kHz}/(\text{V}·\sqrt{\text{ms}}$) (B, gray). These functions are calculated in the Fourier domain for a range of modulation frequencies [${\widehat{R}}_{\mu }(f;\mu ,\sigma )$ in 1/V and ${\widehat{R}}_{\sigma }(f;\mu ,\sigma )$ in $1/(\text{V}·\sqrt{\text{ms}}$)] (insets, gray; absolute values are shown), and fit using an exponential function exploiting asymptotic results for ${\widehat{R}}_{\mu }$ (A, red dashed), as well as considering a range of frequencies (A and B, red solid). In addition, ${\widehat{R}}_{\mu }$ is fit using a damped oscillator function (A, violet). The details of the fitting procedures are described in the text. C: quantities (τ_μ, τ_σ, τ and ω) from the linear filter approximations (cf. A, B), required for the LN_exp and LN_dos model variants (Eqs (85), (87) and (89)), as a function of μ and σ.

We first consider the linear filter D_μ (Eq (79)) and apply the approximation

$$D_{\mu }\left(t\right)\approx A_{\mu }exp(-t/{\tau }_{\mu }),$$ (82)

motivated by the exponential decay exhibited by R_μ, particularly for large input variance compared to its mean. Note that D_μ depends on ${\mu }_{\text{tot}}^0$, ${\sigma }_{\text{tot}}^0$ and therefore the scaling parameter A_μ and the time constant τ_μ both depend on ${\mu }_{\text{tot}}^0$, ${\sigma }_{\text{tot}}^0$, which is not explicitly indicated. A_μ and τ_μ may be determined analytically using asymptotic results for the Fourier transform ${\widehat{R}}_{\mu }$ of the linear rate response for vanishing and very large frequencies, respectively [21, 76],

$$\underset{f\to 0}{lim}{\widehat{R}}_{\mu }\left(f\right)=\frac{\partial }{\partial \mu }{r}_{\infty }({\mu }_{\text{tot}}^0,{\sigma }_{\text{tot}}^0),\underset{f\to \infty }{lim}{\widehat{R}}_{\mu }\left(f\right)=\frac{r_{\infty }({\mu }_{\text{tot}}^0,{\sigma }_{\text{tot}}^0)}{i2\pi f{\Delta }_{\mathrm{T}}}.$$ (83)

To guarantee that these asymptotics are matched by the Fourier transform A_μτ_μ/(1 + i2πfτ_μ) of the exponential, taking into account the scaling factor in Eq (79), we obtain

$$A_{\mu }=\frac{1}{{\tau }_{\mu }},{\tau }_{\mu }=\frac{{\Delta }_{\mathrm{T}}}{r_{\infty }({\mu }_{\text{tot}}^0,{\sigma }_{\text{tot}}^0)}\frac{\partial }{\partial \mu }{r}_{\infty }({\mu }_{\text{tot}}^0,{\sigma }_{\text{tot}}^0).$$ (84)

Note that matching the zero frequency limit in the Fourier domain is equivalent to the natural requirement that the time integral ${\int }_0^{\infty }{D}_{\mu }\left(\tau \right)d\tau$ of the linear filter is reproduced exactly, that is, the approximation is normalized appropriately. An advantage of this approximation is that it is no longer required to calculate the linear rate response function R_μ explicitly. On the other hand, as only the limit f → ∞ is used for fitting in addition to the normalization constraint, the approximation of the linear filter can be poor for a range of intermediate frequencies (in the Fourier domain), in particular for small input mean and standard deviation (see Fig 8A here, and Fig. 4B,C in [21]). To improve the approximation for intermediate frequencies we use the same normalization condition, which fixes the parameter A_μ = 1/τ_μ, and we determine τ_μ by a least-squares fit of ${\widehat{D}}_{\mu }$ over the range of frequencies f ∈ [0, 1] kHz. In both cases, using the approximation Eq (82) the filtered mean input μ_f(t) = D_μ * μ_syn(t) can be equivalently obtained by solving the simple scalar ODE,

$$\frac{d{\mu }_{\mathrm{f}}}{dt}=\frac{{\mu }_{\text{syn}}-{\mu }_{\mathrm{f}}}{{\tau }_{\mu }}.$$ (85)

Recall that τ_μ depends on ${\mu }_{\text{tot}}^0$, ${\sigma }_{\text{tot}}^0$. This exponentially decaying filter is part of the LN_exp cascade model variant.

A shortcoming of the approximation Eqs (82) and (85) above is that it cannot reproduce damped oscillations exhibited by the true linear filter, in particular, for large input mean and small variance (see Fig 8A). Therefore, we introduce an alternative approximation using a damped oscillator function,

$$D_{\mu }\left(t\right)\approx B_{\mu }exp(-t/\tau )cos\left(\omega t\right).$$ (86)

Note that here B_μ, τ and ω depend on ${\mu }_{\text{tot}}^0$, ${\sigma }_{\text{tot}}^0$, which is not explicitly indicated. We fix the scaling parameter B_μ by (again) requiring that the approximation is normalized to reproduce the time integral of the true linear filter D_μ, which yields B_μ = (1 + (τ² ω²))/τ. The remaining two parameters τ and ω are determined such that the dominant oscillation frequency is reproduced. Specifically, the approximation should match ${\widehat{D}}_{\mu }$ at the frequencies $f_{\mathrm{R}}={\text{argmax}}_f\mid \text{Re}\left\{{\widehat{D}}_{\mu }\left(f\right)\right\}\mid$ and $f_{\mathrm{I}}={\text{argmax}}_f\mid \text{Im}\left\{{\widehat{D}}_{\mu }\left(f\right)\right\}\mid$ in the Fourier domain as close as possible. We would like to note that using the method of least-squares over a range of frequencies instead can generate approximated filters which decay to zero instantly, particularly for large input mean and small variance (not shown). For such inputs a damped oscillator with a single decay time constant is too simple to fit the complete, rather complex linear filter shape. With Eq (86) the filtered mean input can be obtained by solving the second order ODE

$${\ddot{\mu }}_{\mathrm{f}}+\frac{2}{\tau }{\dot{\mu }}_{\mathrm{f}}+(\frac{2}{{\tau }^2}+{\omega }^2){\mu }_{\mathrm{f}}=\frac{1+{\tau }^2{\omega }^2}{\tau }(\frac{{\mu }_{\text{syn}}}{\tau }+{\dot{\mu }}_{\text{syn}}).$$ (87)

Using this damped oscillator filter gives rise to the LN_dos cascade model variant.

Considering the linear filter D_σ (Eq (80)) we use the approximation D_σ(t) ≈ A_σ exp(−t/τ_σ), with A_σ = 1/τ_σ and τ_σ determined by a least-squares fit of ${\widehat{D}}_{\sigma }$ for frequencies f ∈ [0, 1] kHz, as long as $\frac{\partial }{\partial \sigma }{r}_{\infty }({\mu }_{\text{tot}}^0,{\sigma }_{\text{tot}}^0)>0$. When this condition is not fulfilled, which occurs for large input mean and small variance, the full linear filter cannot be properly fit by an exponential function. This may be seen by the asymptotic behavior [77]

$$\underset{f\to \infty }{lim}{\widehat{R}}_{\sigma }\left(f\right)=\frac{{\sigma }_{\text{tot}}^0{r}_{\infty }({\mu }_{\text{tot}}^0,{\sigma }_{\text{tot}}^0)}{i2\pi f{\Delta }_{\mathrm{T}}^2},$$ (88)

which implies negative ${lim}_{f\to \infty }{\widehat{D}}_{\sigma }\left(f\right)$ (cf. Eq (80)) that cannot be approximated for nonnegative τ_σ. In this case we use τ_σ → 0 which effectively yields D_σ(t) ≈ δ(t), justified by the observation that the full filter rapidly relaxes to zero (see Fig 8B). The linear filter application can be implemented by solving

$$\frac{d{\sigma }_{\mathrm{f}}}{dt}=\frac{{\sigma }_{\text{syn}}-{\sigma }_{\mathrm{f}}}{{\tau }_{\sigma }},$$ (89)

where σ_f(t) = D_σ * σ_syn(t) is the filtered input standard deviation. This filter is used in both model variants (LN_exp and LN_dos). Note (again) that τ_σ depends on ${\mu }_{\text{tot}}^0$, ${\sigma }_{\text{tot}}^0$.

Extension for changing input baseline and recurrent coupling

In the derivation above we considered that the synaptic input mean and standard deviation, μ_syn(t) and σ_syn(t), vary around ${\mu }_{\text{syn}}^0$ and ${\sigma }_{\text{syn}}^0$ with small magnitudes. To extend the LN cascade model(s) to inputs that show large deviations from their baseline values we let the linear filters adjust to a changing input baseline in the following way: using the exponentially decaying mean input filter (LN_exp model) the filter parameters τ_μ and τ_σ are evaluated at

$${\mu }_{\text{eff}}\left(t\right)={\mu }_{\mathrm{f}}-⟨w⟩\left(t\right)/C,{\sigma }_{\text{eff}}\left(t\right)={\sigma }_{\mathrm{f}}$$ (90)

in every time step, i.e., these parameters adapt to the effective input moments. Using the damped oscillating mean input filter (LN_dos model), on the other hand, the filter parameters τ, ω and τ_σ are evaluated at μ_tot(t), σ_tot(t) given by Eqs (26) and (27), i.e., these parameters adapt directly to the total input moments, assuming that these moments do not fluctuate too vigorously. Note that because of these adjustments we do not need to consider a (particular) input baseline. Two remarks are in place: (i) the parameters of the damped oscillator cannot be adapted to a changing input baseline using the effective input mean (with μ_f given by Eq (87)), because this can lead to stable oscillations (for an uncoupled population) and thus decreased reproduction performance (not shown); (ii) for input moments that change very rapidly the reproduction performance of the LN_dos model variant may be improved by alternatively evaluating the parameters τ, ω and τ_σ at ${\mu }_a\left(t\right)={\mu }_{\mathrm{f}}^a-\langle w\rangle /C$, σ_eff(t), with ${\mu }_{\mathrm{f}}^a$ governed by Eq (85) (combining LN_exp and LN_dos), cf. [23] (ch. 4.2).

Finally, recurrent coupling within the population is included (in both model variants) by the dependence of the synaptic input moments on the delayed spike rate, μ_syn(t, r_d), ${\sigma }_{\text{syn}}^2(t,r_d)$, with r_d given by Eq (40). For the LN_dos model we can then replace ${\dot{\mu }}_{\text{syn}}$ in Eq (87) by

$${\dot{\mu }}_{\text{syn}}={\dot{\mu }}_{\text{ext}}+JK{\dot{r}}_d={\dot{\mu }}_{\text{ext}}+JK\frac{r-r_d}{{\tau }_d}.$$ (91)

In case of identical (constant) propagation delays within the population this term would be ${\dot{\mu }}_{\text{syn}}={\dot{\mu }}_{\text{ext}}+JK\dot{r}(t-d)$ and in case of recurrent coupling without delays we would have

$${\dot{\mu }}_{\text{syn}}={\dot{\mu }}_{\text{ext}}+JK\left[\frac{\partial r_{\infty }}{\partial \mu }\right({\dot{\mu }}_{\mathrm{f}}-\frac{a({⟨V⟩}_{\infty }-E_w)-⟨w⟩}{C{\tau }_w}+\frac{br}{C})+\frac{\partial r_{\infty }}{\partial \sigma }\frac{{\sigma }_{\text{syn}}-{\sigma }_{\mathrm{f}}}{{\tau }_{\sigma }}].$$ (92)

To summarize both LN cascade models, the population spike rate and mean membrane voltage are described by Eqs (70) and (73), using Eqs (78), (81) and (23), respectively. For the LN_exp model input filtering is governed by Eqs (85) and (89), where the filter parameters are evaluated at μ_eff(t), σ_eff(t) (Eq (90)). For the LN_dos model input filtering is described by Eqs (87) and (89), where the filter parameters are evaluated at μ_tot(t), σ_tot(t) given by Eqs (26) and (27). The system for the recurrent network under consideration is closed by Eqs (21) and (40) which relate population spike rate output and overall synaptic input moments. Note that both LN systems here are fully equivalent to the respective ones specified in the Sect. Model reduction.

Further remarks

In order to efficiently simulate the derived cascade rate models it is highly recommended to precalculate the quantities which are needed in each time step on a (μ, σ)-rectangle (see Sect. Low-Dimensional Approximations above). For the LN_exp model these quantities are the filter time constants τ_μ, τ_σ, and for the LN_dos model we need the quantities τ, ω and τ_σ (all displayed in Fig 8C). Both models additionally require the steady-state quantities r_∞ and 〈V〉_∞ shown in Fig 6. An efficient implementation to obtain these quantities using Python with the package Numba for low-level virtual machine acceleration is available. Recall, that changing any parameter value of the external input, the recurrent synaptic input or the adaptation current does not require renewed precomputations.

If desired, it is also possible to obtain initial values for the variables of the cascade models (LN_exp and LN_dos variants) that correspond to a given initial distribution of membrane voltage and adaptation current values {V_i(0)}, {w_i(0)} of a population of N aEIF neurons. We can calculate 〈w〉(0) = 1/N∑_i w_i(0) and determine μ_f(0), σ_f(0) by requiring that the initial membrane voltage distribution of the respective LN model p_∞(V; μ_f(0) − 〈w〉(0)/C, σ_f(0)) matches the initial voltage distribution from the aEIF population as close as possible (e.g., using the Kolmogorov–Smirnov statistic). For the LN_dos model we additionally set ${\dot{\mu }}_{\mathrm{f}}\left(0\right)=0$. This means we assume vanishing changes in the input history which underlies the initial membrane voltage distribution and filter parameters in the LN models (i.e., ${\dot{\mu }}_{\text{syn}}\approx 0$, ${\dot{\sigma }}_{\text{syn}}\approx 0$ for a sufficiently long time interval prior to t = 0).

The components of the LN model are derived in the limit of small amplitude variations of μ_syn and σ_syn. However, the approximation also provides an exact description of the population dynamics for very slow variations of μ_syn and σ_syn, where the spike rate, mean membrane voltage and adaptation current are well approximated by their steady-state values in each time step.

Here we approximated the derived linear filters using exponential and damped oscillator functions. We would like to note that, for a given baseline input (${\mu }_{\text{tot}}^0$, ${\sigma }_{\text{tot}}^0$) the filter application using the latter function (Eq (86)) can be equivalently described by a complex-valued ODE [23] (ch. 4.2). Furthermore, the true linear filters D_μ and D_σ can be substantially better approximated by a damped oscillator function with two time scales (i.e., two exponentials) each. In these three cases, however, the ODE representation for the filter application can lead to decreased reproduction performance when the baseline input changes very rapidly (due to increased sensitivity to variations of the filter parameters).

Spike shape extension (optional)

In this contribution the membrane voltage spike shape has been neglected (typical for IF type neuron models) by clamping V_i and w_i during the refractory period, justified by the observation that it is rather stereotyped and its duration is very brief. Furthermore, the spike shape is believed to contain little information compared to the time at which the spike occurs. Nevertheless, it can be incorporated in the aEIF model in a straightforward way using the following reset condition, as suggested in [43]: When V_i reaches the spike voltage V_s from below we let V_i decrease linearly from V_s to V_r during the refractory period and increment the adaptation current w_i ← w_i + b at the onset of that period. That is, V_i and w_i are not clamped during the refractory period, instead, V_i has a fixed time course and w_i is incremented by b and then governed again by Eq (15). This modification implies that the average membrane voltage in Eq (23) needs to be calculated over all neurons (and not only the nonrefractory ones), that is, 〈V〉 is calculated with respect to p + p_ref, where $p_{\text{ref}}(V,t)={\int }_0^{T_{\text{ref}}}r(t-s)\delta (V-V_{\text{sp}}\left(s\right))ds$ with spike trajectory V_sp(t) = V_s + (V_r − V_s)t/T_ref, cf. [43]. The same applies to the steady-state mean membrane potential in Eqs (1), (39) and (76), i.e., 〈V〉_∞ is then given by

$${⟨V⟩}_{\infty }={\int }_{-\infty }^{V_{\mathrm{s}}}v{p}_{\infty }\left(v\right)dv+(1-{\int }_{-\infty }^{V_{\mathrm{s}}}{p}_{\infty }\left(v\right)dv)\frac{V_{\mathrm{r}}+V_{\mathrm{s}}}{2},$$ (93)

instead of Eq (38). Notably, the accuracy of the adiabatic approximation (Eq (15)) does not depend on the refractory period T_ref in this case. That type of spike shape can therefore be considered in the FP model and the low-dimensional models in a straightforward way without significant additional computational demand. Note, however, that for the spec₂ model a nonzero refractory period is not supported (see above). For an evaluation of the spike shape extension in terms of reproduction accuracy of the LN models see [23] (Fig. 4.15 in [23]).

Supporting information

S1 Text. Supplementary methods.

A) Numerical integration of the time-dependent Fokker-Planck equation. B) Derivation of the model spec₂ based on the Fokker-Planck operator. C) Numerical solver for the nonlinear Fokker-Planck eigenvalue problem.

(PDF)

S1 Fig. Fast changes of the input variance.

(PDF)

Data Availability

Numerical implementations of the considered models written in the Python programming language are available under a free license at: https://github.com/neuromethods/fokker-planck-based-spike-rate-models.

Funding Statement

The author JL was funded via German Research Foundation (Collaborative Research Center no. 910), http://www.itp.tu-berlin.de/collaborative_research_center_910/. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.