The Volterra Functional Series is a Viable Alternative to Kinetic Models for Synaptic Modeling -Calibration and Benchmarking

Abstract

Synaptic transmission is governed by a series of complex and highly nonlinear mechanisms and pathways in which the dynamics have a profound influence on the overall signal sent to the postsynaptic cell. In simulation, these mechanisms are often represented through kinetic models governed by state variables and rate law equations. Calculations of such ordinary differential equations (ODEs) in kinetic models can be computationally intensive, and although algorithms have been optimally developed to handle ODEs efficiently, simulation of numerous, large and complex kinetic models requires a prohibitively large amount of computational power. Here we present an alternative representation of ionotropic glutamatergic receptors AMPAr and NMDAr kinetic models consisting of input-output surrogates of the receptor models which can capture the nonlinear dynamics seen in the kinetic models. We benchmark this Input-Output (IO) synapse model and compare it with kinetic receptor models to evaluate the simulation time required when using either synapse model, as well as the number of time steps each model needs for simulation. While remaining faithful to the original dynamics of the model, our results indicate that the IO synapse model requires less simulation time than the kinetic models under conditions which elicit normal physiological responses, thereby improving computational efficiency while preserving the complex non-linear dynamics of the receptors. These IO surrogates therefore constitute an appealing alternative to kinetic models in large scale networks simulations.

I. Introduction

Chemical synapses are comprised of vastly complex and intricate pathways and mechanisms that shape signal transmission between neurons. These mechanisms are characterized by nonlinear properties at widely varying time scales (e.g. neurotransmitter diffusion is achieved in the microseconds scale, while activation of second-messenger pathway is in the seconds time scale), which can prove computationally difficult to capture and model accurately. To further complicate the matter, synapses are nonstationary in nature and their small size makes it difficult to study experimentally, thus further complicating accurate development and validation of computational models. Fortunately, advancements have been made in physiological studies [1], [2] that have led to a better understanding of synapse properties and dynamics. These studies have helped in developing more detailed and accurate synapse computational models, and simulations in turn have helped further elucidate processes that are difficult to determine experimentally [3]–[5].

A large contribution of the aforementioned complexity inherent to the various mechanics of chemical synapses can be represented as changes in the internal states of the models over the time course of the simulation. This is generally defined through kinetic models by using ordinary differential equations (ODEs). ODEs represent the rate at which the various states of the models change over time. These states are interdependent on each other, which gives arise to nonlinear dynamics during simulation that are characteristic of the complex systems being modeled. In simulations, ODE solvers are algorithms used for calculating the differentials and states based on past state values[6]. Generally, changes in the state values are calculated incrementally over the simulated time course, where one increment is defined as a “time step” in the simulation. A special category of ODE solvers - variable time step algorithms - can actually change the size of these increments depending on the amount of activity that is occurring in the states. This can lead to more accurate calculations of the states when the rate of change is high, while reducing the number of calculations required when the rate of change is low thereby reducing computational cost. As a result, variable time step algorithms have made longer simulations possible while improving the validity of results produced (see Figure 1 for a detailed schematic).

Figure 1. Schematic detailing the variable time step algorithm.

Left: In variable time step, the time points in simulated time are determined by the rate of change in the dynamics of the model - more change causes the points in simulated time to be calculated more close to each other, while less change leads to larger time steps in between calculations. For each simulated point in time that is calculated, the simulation itself takes T_i seconds, and total calculation time for the simulation is the summation of all the calculation time per time step. Right: The calculation time is determined by the differential equations and algebraic equations in the synapse model, and the differential equations in the neuron model. Differential equations require a previous time point as a reference and the change in time between the previous time point and the current time point. Algebraic equations require only the current time point to calculate. Kinetic models comprise mainly of differential equations, while the IO synapse model contains mostly algebraic equations.

As of yet, there are still some limitations imposed by usage of variable time-step algorithms. One particular limitation is the number of differential equations can affect the performance of the variable time-step algorithm. Specifically, if certain differentials are fast-acting while others have slower dynamics at the same time period, the algorithm must compromise and determine the optimal step size for all rate values being calculated. This ultimately leads to decreased efficiency when computing larger models. To address this issue, in this study we present an alternative model that utilizes less differential rate equations and more algebraic equations, which do not depend on past time values to calculate. The model proposed uses the functional Volterra power series with Laguerre Expansions to capture the input-output relationships of the kinetic state based models and is designated as an Input-Output (IO) synapse model. Here we compare the performance of the IO synapse model to the kinetic model, where we find a difference in simulation time and the number of steps required between the two models.

II. Methods

Volterra Series with Laguerre Expansions - The IO synapse model utilizes the Volterra Functional Power Series with Laguerre Expansions. The Volterra model was fitted using procedures as described in [7], while the Laguerre Expansions were calculated in the time domain with equations. See[8] for details.

In the NEURON simulation environment[9], the Laguerre basis functions were implemented both in algebraic form converted into a lookup table, and as state equations to make use of the optimized ODE solvers provided by NEURON. The rate form of the Laguerre Expansion is shown as the following:

$$\{\begin{array}{c}\frac{d{l}_0}{dt}=-p(l_0(t))\\ \frac{d{l}_1}{dt}=-p(l_1(t)-2l_0(t))\\ \frac{d{l}_2}{dt}=-p(l_2(t)-2l_1(t)+l_0(t))\\ \cdots \end{array}$$

The input to the system, defined as x(t), is a series of ones and zeros where a value of one indicates an event at time t, and a value of 0 indicates an absence of any event at time t. The final form of the basis functions, v(t), is presented as the following:

$$\{\begin{array}{c}\frac{d{v}_0}{dt}=-p(v_0(t))+x(t)\sqrt{2p}\\ \frac{d{v}_1}{dt}=-p(v_1(t)-2v_0(t))-x(t)\sqrt{2p}\\ \frac{d{v}_2}{dt}=-p(v_2(t)-2v_1(t)+v_0(t))+x(t)\sqrt{2p}\\ \cdots \end{array}$$

Here we consider two types of input-output receptor models, one for the AMPA receptor and other for the NMDA receptor[4]. The IO synapse model was implemented into NEURON as a module (.mod) file representing as a synapse. The leaky integrate and fire model [10] was used as a single compartment neuron model. All synapse models are instantiated directly onto the neuron compartment.

Kinetic Models of the two receptors were also adapted to NEURON using SBML 2 NEURON, a freeware developed by the NeuroML team[11]. Similarly, modifications were also made to adapt the kinetic models of the EONS platform to the NEURON simulation environment.

Simulation details - 2 Hz Poisson random interval train was used as input and was delivered to all synapses. Synapses were deterministic, i.e. fired in response to each stimulation event. All simulations used 1000 synapse instances placed on a single neuron.

III. Results

In Evaluation of Table Format Against State Equations

The Laguerre expansion can be represented as a lookup table or can be expressed as a set of rates. We compared the two methods to determine which method yields better performance. Simulations were run with 1000 synapse instances of the IO synapse model. In these simulations, the number of AMPA was set to 80 while number of NMDA was set to 0. Each synapse had an EPSC amplitude of about 0.02 nA, which is the average expected EPSC of a synapse in response to a presynaptic stimulation[4]. Results of the comparison show that the table lookup method takes approximately 100 seconds to simulate 10 seconds of simulated time, while the Laguerre basis functions defined as rate equations took 29 seconds, indicating that state equations perform better in the NEURON simulation environment than the table representation of the Laguerre expansions for the IO synapse model.

Comparisons Between the IO Receptor Models and Kinetic Models in NEURON

The IO receptor models were fitted to the EONS synaptic platform results and encompass synaptic properties such as glutamate diffusion and receptor kinetics. The kinetic receptor models implemented only reflect the receptor kinetics without consideration for presynaptic factors and glutamate diffusion. Nonetheless, the kinetic models are considered one of the more time-consuming parts of the simulation process; therefore comparing between the kinetic receptor models within NEURON with the IO synapse model can provide insight into the performance and efficiency of the IO synapse model. In all simulations, 1000 synapse instances were used. The benchmarking studies are split into three categories: one, where all the synapses fire at their full weight (0.02 nA); two, synapses fire at a rate that is characteristic of physiological firing; and three, synapses fire with no input delivered into the neuron, strictly limiting the calculations to the synapses themselves and not to the neuron model. Results show that when all synapses were firing at their full potential, the kinetic models outperformed the IO synapse model in terms of speed. On the other hand, in physiological conditions or no weight, the IO synapse model was faster. In all cases, the IO synapse model required less steps to simulate than in the kinetic model.

IV. Conclusion

In this study we present an input-output synapse model and its overall performance when multiple instances are placed on a simple integrate-and-fire neuron model[10]. The foundation of the IO synapse model is the Volterra functional power series, which is mainly composed of algebraic equations. The Laguerre expansion, used as basis functions for the Volterra functional power series, can be represented either in algebraic format or in the form of state equations. The results presented here shows the simulation time for both forms of the Laguerre expansions and suggests that the simulation time is significantly decreased when using the Laguerre basis functions in the form of state equations. A possible explanation is that calculation of differentials is considered more efficient than calculation of complex operations such as exponentials and power functions within the NEURON environment. As many of the neuron models are compartment-based and contain differential equations, NEURON therefore uses well optimized ODE solvers for efficient calculation of the differentials in varying models. Exponential and power functions are generally represented in tabular format so that values may be accessed through the tables rather than calculated each time. However, table construction can still be time consuming and values may still require interpolation for the intermediate values in between table entries. These factors could account for the increased calculation time for using Laguerre expansions in table format.

This study presents the IO synapse model as an alternative to the traditional kinetic receptor models for multiscale and large-scale simulations. The results outline the performance of the IO synapse model and two simple kinetic models, the AMPA receptor and the NMDA receptor [4], with 1000 synapse instances on a single neuron. Three conditions were measured -- Full Weight, where all synapses fired synchronously at their full potential during an event; Physiological Conditions , where synapse weight was recalibrated to induce an average response of one spike from the postsynaptic cell to a presynaptic event; and No Weight, where synapse activity does not trigger neuron activity in the simulation. Full weight caused excessive neuron spiking and large changes in the response within a short amount of time, thus the simulation required a large number of steps to calculate, whether using the IO synapse model or the kinetic models. Physiological Conditions, with moderate neuron activity, and No Weight, with no neuron activity, required significantly less steps, especially for the IO synapse. This discrepancy in the number of steps could therefore be attributed to the activity of the neuron, which in itself utilizes rate equations and influences the step size of the simulation. When neuron activity is reduced, the synapse model retains more influence over step size and number of steps. Here, the kinetic models, which contain more differential equations than the IO models, were shown to require three times as many steps for calculation than the IO synapse model -- as a result, the kinetic models yielded longer simulation time than the IO synapse model.

It is notable, however, that the simulation time per step size is higher for the IO synapse model than the kinetic models. The IO synapse model contains many algebraic equations for calculating nonlinear dynamics -- these algebraic equations are not dependent on time steps and do not influence the step sizes on their own. The kinetic models, on the other hand, have only a few rate equations to solve compared to the number of algebraic equations in the IO synapse model. On their own, these differential equations are not computationally intensive. Instead, the equations are more reliant on step size to solve the states in the model efficiently and accurately. More differential equations would require more precise step sizes, leading to more steps required for the overall simulation. As a result, even though kinetic models take less time to calculate per step, the simulation time is longer for kinetic models in general because they require more steps to calculate compared to the IO synapse model. Large-scale neuron networks may have millions of neurons with thousands of synapses on each neuron. The simulation time required relative to the number of synapses is seen to be linear (results not shown here) regardless of what type of synapse is used. As a result, it is imperative that the computational complexity of each synapse is low in order for large scale simulations to be feasible. Here we have documented the performance of two types of synapse models. The models compared here represent the ionotropic AMPA receptor only; yet, our results outline the differences in terms of calculation time depending on the underlying model methodology. Future renditions will have more complex mechanisms such as metabotropic glutamate receptor activation and calcium dynamics; in these kinetic models, the number of ODEs is significantly larger, thereby resulting in even more time per step to calculate than their input-output surrogates, and potentially result in an even smaller step size. Further advances in the IO synapse model can therefore provide a way to enable implementation of highly complex synaptic mechanisms into large-scale models. By doing so, more complex physiological and pathological conditions can be studied and analyzed to better our understanding of synapse dynamics and their effects on overall neuron network activity and function.

Figure 2.

Comparison between 2 representations of the Laguerre Expansion: algebraic equations (in the form of a table) or state equations for the IO Synapse Model. Using algebraic equations required about 100 seconds to simulate while the state equations required 29 seconds.

Figure 3.

Comparisons between the IO Synapse Model and kinetic receptor models based on speed and number of steps. Three conditions are presented: Full weight, where each synapse fires at their full weight leading to a current amplitude of 0.02 nA in response to an event; Physiological conditions, where synapse weights are calibrated to reproduce standard neural activity; and no weight, where synapses are active but there is no neural activity. A.) Simulation time (in seconds) required based on synapse type and condition. At full weight, kinetic models are faster than the IO synapse model; for physiological conditions and no weight, IO synapses are faster. B.) Number of time steps (calculation points) required based on synapse type and condition. IO synapse always requires less steps to calculate than the kinetic model. C.) Simulation time (in milliseconds) required per time step of the simulation, based on synapse type and condition. IO synapse model requires more time than the kinetic model to calculate each time step. D.) Linear fit plot detailing the simulation time required based on number of steps, for the IO synapse model and the Kinetic model. Note that while overall IO Synapse model requires more time to simulate for the same number of steps, the number of steps required in a simulation for using the IO Synapse Model is overall reduced.