Skip to main content
NCBI home page
Cognitive Neurodynamics logoLink to Cognitive Neurodynamics
. 2021 Sep 15;16(2):401–409. doi: 10.1007/s11571-021-09698-7

Modeling plasticity during epileptogenesis by long short term memory neural networks

Marzieh Shahpari 1, Morteza Hajji 2, Javad Mirnajafi-Zadeh 3, Peyman Setoodeh 2,✉
PMCID: PMC8934824  PMID: 35401870

Abstract

Understanding the pathogenesis of epilepsy including changes in synaptic pathways can improve our knowledge about epilepsy and development of new treatments. In this regard, data-driven models such as artificial neural networks, which are able to capture the effects of synaptic plasticity, can play an important role. This paper proposes long short term memory (LSTM) as the ideal architecture for modeling plasticity changes, and validates this proposal via experimental data. As a special class of recurrent neural networks (RNNs), LSTM is able to track information through time and control its flow via several gating mechanisms, which allow for maintaining the relevant and forgetting the irrelevant information. In our experiments, potentiation and depotentiation of motor circuit and perforant pathway as two forms of plasticity were respectively induced by kindled and kindled + transcranial magnetic stimulation of animal groups. In kindling, both procedure duration and gradual synaptic changes play critical roles. The stimulation of both groups continued for six days. Both after-discharge (AD) and seizure behavior as two biologically measurable effects of plasticity were recorded immediately post each stimulation. Three classes of artificial neural networks—LSTM, RNN, and feedforward neural network (FFNN)—were trained to predict AD and seizure behavior as indicators of plasticity during these six days. Results obtained from the collected data confirm the superiority of LSTM. For seizure behavior, the prediction accuracies achieved by these three models were 0.91 ± 0.01, 0.77 ± 0.02, and 0.59 ± 0.02%, respectively, and for AD, the prediction accuracies were 0.82 ± 0.01, 0.74 ± 0.08 and 0.42 ± 0.1, respectively.

Keywords: Deep learning, LSTM, Kindling, Epilepsy model

Introduction

Epilepsy is one of the most common neurological diseases that has affected fifty million people worldwide (Beghi 2020). Epilepsy affects all genders and social classes in a similar manner, and occurs at any age (Kale 1997), (Svalheim et al. 2006). It is associated with recurrent electrical discharges that spread over cortices accompanied by molecular and cellular pathogenesis in different brain regions (Rizvi et al. 2017). Although the first reports of epilepsy and seizures belong to several thousand years ago, still there are no definitive therapeutic solutions. In addition, about thirty percent of patients are still pharmacoresistant (Dalic and Cook 2016). Due to multiple physiopathological conditions in the brain as a complex dynamic system, finding an absolute treatment for epilepsy is not a straightforward task. Many experimentally-validated models have been proposed to advance our knowledge about epilepsy and its associated mechanisms.

This paper aims at taking a step towards bridging the gap between a descriptive analysis and a formal theory of brain’s response to perturbations that negatively impact the brain function and cause illness. The proposed approach is an effort towards integrating data analysis, computational modeling, and system theory for understanding the effects of such perturbations on brain by capturing the relations between perturbations and functionality. Such modeling approaches will hopefully pave the way for moving from local treatments to network treatments for restoring mental health (Lydon-Staley et al. 2021).

It is important to build computational models that are capable of revealing dynamic mechanisms, which are testable in biological systems. In this way, the clinical phenomenology can be associated with the corresponding dynamic mechanisms. Hence, in order to build computational models for epilepsy, the following points deserve special attention (Richardson 2012):

  • The fact that seizures emerge from normal brain states and usually self-terminate reveals that normal brain function and seizure share the same neuronal machinery, and in effect therefore, epilepsy emerges from dynamic properties of the brain. In many patients, seizure onset is driven by a complex dynamic network rather than a single focal region.

  • Since patients with seizure onsets from different brain regions may show seemingly identical seizures, we can conclude that a wide range of mechanisms lead to a relatively small number of seizure types. Hence, different micro-scale mechanisms of seizure onset may result in common large-scale behavior.

Regarding the fact that there is a small range of such common large-scale behaviors, from the system and control theoretic point of view, prediction of these different categories of behavior can be formulated as a classification problem (Gao et al. 2017). As mentioned previously, such common large-scale behaviors are direct effects of brain’s complex network dynamics, which can be tested and measured in biological subjects. Brain network dynamics is in turn a function of plasticity. Hence, in this research, AD and seizure behavior as two biologically measurable effects of plasticity are predicted by building proper classifiers. In epilepsy, brain dynamically switches between normal state and seizure. This paper is focused on building fairly simple models that can capture global behavior of brain networks. Such models must encompass different micro-scale mechanisms of seizure onset and reveal common emergent macro-scale behavior in order to allow for epilepsy classification based on description of mechanisms. These models provide key insights into epilepsy and pave the way for new approaches to treatment (Richardson 2012).

In summary, to model synaptic potentiation and depotentiation as two important types of synaptic plasticity, we use brain signals during kindling (as an abnormal synaptic potentiation) and kindling + repetitive transcranial magnetic stimulation (rTMS) (as a model of depotentiation or synaptic depression) induced in rats. Local field potentials (LFP) and seizure behavior are considered as indicators of synaptic plasticity. To model the plasticity, artificial neural networks are used to predict these indicators of the plasticity regarding the fact that such a prediction task can be formulated as a classification problem. Therefore, a properly-trained neural network that performs well on predicting these indicators of plasticity provides a computational model that reveals the underlying dynamic mechanisms in a biologically-testable manner.

Neuro-computational modeling can be used as an integrative approach to capture the complexity of epileptic phenomena (Wendling 2008). In addition, computational models can be considered as effective tools to examine hypotheses, which can be subsequently tested experimentally. A wide spectrum of computational models have been used for automatic detection or prediction of ictal and seizures from EEG recordings (Akbarian and Erfanian 2018; Fu et al. 2020; Sun et al. 2019). However, deploying machine learning to model the physiology and pathophysiology of brain has been a less-explored area, which is gaining momentum regarding the impressive success of deep-learning algorithms in different fields.

Deep neural networks are brain-inspired architectures, which can learn accurate approximations of functions with arbitrary levels of complexity (Bengio 2009). In recent years, such algorithms have been widely used in different engineering disciplines. Deep learning provides a success story on how machine learning can benefit from neuroscience. Similarly, neuroscience can benefit from the ongoing research in the field of artificial intelligence (Fellous et al. 2019; Shapshak 2018). Following this line of thinking, here, we use deep neural networks for modelling the neuronal activity and plasticity during kindling.

While FFNNs are suitable for learning static functions, RNNs are able to capture dynamic behavior due to their feedback connections. Therefore, they provide a more rational choice for modeling biological neuronal networks (Kriegeskorte 2015). Hence, RNNs can be viewed as potential candidates for modelling plasticity in the brain (Pfeiffer and Pfeil 2018). However, the vanishing gradient problem in standard RNNs must be addressed in order to achieve a reliable model for plasticity, when we are dealing with sequences of events. In this study, we propose to use RNNs in general and especially a sophisticated class of RNNs known as LSTM to model short-term plasticity and neuronal activity during kindling in rat. Results demonstrate their advantages over feedforward networks.

The main building block of an LSTM network is a computational cell that maintains a separate cell state from the output of the cell. Each LSTM cell deploys a set of gates to control the flow of information. LSTM networks are distinguished from simple RNNs by such gating mechanisms. The forget gate filters the irrelevant information. Then, relevant information is stored and the cell state is selectively updated based on the information passed through the input gate regarding the input and the recurrent output. Finally, the cell output is constructed from a filtered version of the cell state by the output gate. Such networks are trained by backpropagation through time with uninterrupted gradient flow.

Deep learning has been widely used to model vision and other cognitive processes. However, in the field of epilepsy, most of the studies have been focused on predicting seizure by extracranial EEG. In this study, we provide models that are able to predict the response of local specific synapses in dentate gyrus. Modeling perforant path could be used as a template for testing various drugs in temporal lobe epilepsy (TLE). Another novelty of this work is modeling seizure behavior and motor circuit during potentiation and depotentiation. A reliable and valid computational model that can predict the synaptic plasticity of the limbic system and motor circuit can be used to predict the effects of various therapeutic agents including antiepileptic drugs as well as electrical and magnetic stimulations.

This paper is organized as follows. Next section provides the required background on kindling. “Model and methods” section covers experiment design, data collection, and building neural network-based predictive models. Results are presented in “Results” section. Finally, “Discussion” section and “Conclusion” section are dedicated to discussion and conclusion, respectively.

Kindling

Kindling is a chronic model in which the sequential stimulations enhance seizure susceptibility (Bertram 2007). It generates pathophysiological variations in brain functions and morphologies, which are similar to those manifesting in TLE patients (the most abundant epileptic type) (Morimoto et al. 2004; Schmoll et al. 2003). There are two categories of kindling (Wasterlain et al. 1982):

  • Electrical kindling is induced by electrical stimulation of various brain regions (Goddard et al. 1969; Racine 1972).

  • Chemical kindling uses diverse chemical agents either systemically or in particular brain regions (Samokhina and Samokhin 2018; Lévesque and Avoli 2013; Kim and Cho 2018).

For studying cellular and molecular mechanisms of epileptogenesis, electrical kindling is usually preferred to chemical kindling, because it provides more accurate and valid experimental results. Electrical kindling models of temporal lobe epilepsy are induced by repeated stimulation pulses in limbic structures, especially in amygdala and perforant path, leading to the gradual induction of limbic system seizures, which ultimately provide the condition for studying network mechanisms (Goddard et al. 1969; Racine 1972).

During kindling, pathological changes progressively occur at various levels of the nervous system structure, ranging from altered molecular construction in discrete neurons to reorganization of synaptic connections or even loss of specific neuronal populations (Mody 1993; Szyndler et al. 2012; Faas et al. 1996). These changes result in an abnormal synaptic plasticity (Hughes 1958). Two forms of synaptic plasticity are potentiation and depotentiation that seem to change in various diseases (Citri and Malenka 2008). LFPs recorded from animal brain are considered as good indicators representing these two types of synaptic plasticity and neuronal activation during kindling (Herreras 2016).

Kindling and long-term potentiation (LTP) strongly resemble in many aspects. Both are considered as a central nervous system (CNS) plasticity model. In addition, applying brief high-frequency trains of electrical pulses through implanted electrodes leads to excessive and susceptible response to a constant stimulus. Furthermore, these two models are similar regarding the basic neural mechanisms. This similarity has led to the hypothesis that LTP may establish the cellular mechanism of kindling (Cain 1989).

The reversal of LTP is known as depotentiation, in which synapses that have recently experienced LTP, reverse the synaptic strengthening process (Cain 1989; Chen et al. 2016). Depotentiation can be induced by medical drugs, low-frequency stimulation, or rTMS after induction of potentiation (Yadollahpour et al. 2014). In the current study, high-frequency stimulation and rTMS are used to induce potentiation and depotentiation, respectively. Then, neuro-computation is deployed to model these two forms of synaptic plasticity based on the collected data.

Model and methods

Rapid kindling procedure

The physiological experiments were performed in the Epilepsy Lab in the Department of Physiology, Tarbiat Modares University, Tehran, Iran. The data used in this study was gathered from 27 male rats (250-270 gr). The dataset was collected by placing the stimulating electrodes in the perforant path and the recording electrode in the dentate gyrus of rats (Fig. 1). The reference and ground electrodes were fixed above the skull via stainless steel pins. After fixing the electrodes in the miniature socket, it was consolidated using dental acrylic. The head-stage of the rat was connected to a flexible shielded cable. Recording was carried out after transferring animals to the recording box.

Fig. 1.

Fig. 1

Open in a new tab

The stimulating electrodes are located in the perforant path and the recording electrode is implanted in the dentate gyrus

After at least two weeks of post-surgical recovery, the AD threshold was defined as follows. Initially, a stimulating current of 30A was delivered and it was gradually increased by 10A until the AD of 10s was recorded from the dentate gyrus (Shahpari et al. 2012). ADs were regarded as spikes, which contained frequencies of minimum 1Hz and amplitudes of at least twice the baseline activity initiated after the stimulation (Fig. 2). AD threshold was determined as the minimum current essential to induce an AD ≥ 10s. The considered AD threshold intensity for different animals in this study was the interval of 80 to 150A. Animals were divided into kindled and kindled anticonvulsant agent groups as the indicators of potentiation and depotentiation groups, respectively. In the experiments, rTMS was used as the anticonvulsant agent.

Fig. 2.

Fig. 2

Open in a new tab

After-discharge of potentiation and depotentiation groups

To induce potentiation and depotentiation, animals were divided into two groups including kindled and kindled + rTMS, respectively. For the rats in the kindled group, eleven stimulations per day were performed and the ADs were recorded after each one of the kindling stimulations. In the kindled + rTMS group, rTMS was carried out using different frequencies and coil shapes. The stimulation continued for six days. Regarding the Racine score, the seizure behavior of each animal was recorded immediately post each stimulation (Racine 1972). Changes in seizure behavior during kindling are due to the synaptic plasticity in the brain motor circuit (Schubert et al. 2005).

Experiment design

The AD duration after each stimulation on each day was measured for modeling perforant path in kindling development. AD duration was categorized into 4 classes as: 0 ∼ (y < 4550), 1 ∼ (4550 < y < 5750), 2 ∼ (5750 < y < 13500) ∪ (13500 < y < 31500), and 3 ∼ (y > 31500). The seizure behavior was categorized into 6 classes: No seizure as class 0, and stages 1, 2, 3, 4, and 5 based on Racine score (Racine 1972) as classes 1 to 5, respectively. We considered two scenarios. In the first scenario, inputs were ADs and seizure behaviors corresponding to the first ten stimulations and output was AD and seizure behavior of the last stimulation on each day as shown in Fig. 3a. In the second model, performance of the learning machine was examined based on its ability to predict the AD and the seizure behavior on the sixth day (the last day of experiments) even when the data from the fifth day was not fed to the network. Inputs were ADs and seizure behaviors recorded during the first four days of experiment and outputs were ADs and seizure behaviors of all eleven stimulations on the sixth day as shown in Fig. 3b. The second scenario can be viewed as a proof of concept that the proposed method is capable of handling missing data (Cismondi et al. 2013).

Fig. 3.

Fig. 3

Open in a new tab

Two scenarios: a Inputs are ADs and seizure behaviors corresponding to the first ten stimulations per day, and output is the AD and the seizure behavior of the last (eleventh) stimulation on each day. b Inputs are ADs and seizure behaviors recorded during the first four days of experiment, and outputs were ADs and seizure behaviors of all eleven stimulations on the sixth day, when the data from the fifth day was assumed to be missing

Neural network design

  • Feedforward neural network: A neural network with three layers was constructed with 50 nodes in the first (input) layer, 5 to 50 nodes in the hidden layer, and 4 nodes in the output layer. To avoid over-fitting and under-fitting, the number of neurons in the hidden layer was varied to obtain the best architecture. Finally, the best performance was obtained using 50 neurons in the hidden layer. In a fully-connected FFNN, each neuron is connected to all of the neurons in both the previous and the next layers. In our model, the summations of weighted inputs were propagated through rectified linear unit (ReLU) activation functions in the hidden layer. The output layer provides a probability density function over multiple classes using the softmax function. To mitigate overfitting, weight regularization was used in the hidden layer using the 2-norm known as weight decay with a regularization parameter of 0.01. In addition, dropout with a rate of 0.5 was used in the hidden layer. The Model was compiled and trained using Adam as the optimizer and categorical cross-entropy as the loss function.

  • Recurrent neural network: A four-layer RNN was built with one input layer, two hidden layers, and one output layer. Figure 4 shows an RNN cell with a feedback path from its output to its input. The first fully-connected layer had 50 hidden neurons, followed by a ReLU activation layer, and the final classification output layer used softmax. Training was regularized by 2-norm with a weight decay of 0.01, and to overcome the overfitting issue, dropout with a rate of 0.5 was performed after the hidden layers.

  • Long short term memory network: For the third model, we used LSTM, which is an extended version of RNN with three gates: input, output, and forget gates (Fig. 5). A three-layer LSTM was designed with one input layer, one hidden layer, and one output layer (Fig. 6). These layers have the following characteristics: the LSTM blocks were deployed in the first layer, a fully connected layer of neurons was chosen for the hidden layer, and softmax was used in the output layer. To prevent overfitting, dropout was used after the hidden layer. To compile and train the model, RMSprop and categorical cross-entropy were selected as the optimizer and the loss function, respectively. RMSprop is an adaptive learning rate method. It has the effect of balancing the step size by decreasing the step for large gradient values to avoid the exploding gradient problem, and increasing the step for small gradient values to avoid the vanishing gradient problem (Hinton 2002).

Fig. 4.

Fig. 4

Open in a new tab

A simple recurrent cell

Fig. 5.

Fig. 5

Open in a new tab

Structure of an LSTM cell, where σ and ψ denote sigmoid and hyperbolic tangent functions, respectively

Fig. 6.

Fig. 6

Open in a new tab

The long short term memory network architecture

Results

In this study, we examined the effectiveness of various neural networks on predicting the seizure behavior and AD response of dentate gyrus neurons to the stimulation of perforant path. Different algorithms and neural network architectures were deployed to simulate the motor circuit and perfornat path in different conditions. Input dimensions of the designed classifiers were 10×1 and 4×11 for the first and the second scenarios, respectively. The studied models were all trained on an NVIDIA GeForce GTX 1050 GPU with 16GB memory. Neural networks were implemented using Keras and TensorFlow, which are Python libraries for deep learning.

Table 1 summarizes the results for all trained models to predict the AD response of dentate gyrus neurons to the last (eleventh) stimulation on each day based on ADs and seizure behaviors corresponding to the first ten stimulations on that day as shown in Fig. 3a. LSTM performed slightly better than simple RNN in predicting the seizure behavior and AD responses of dentate gyrus to the last stimulation. FFNN displayed a relatively good performance as well. In this scenario, inputs were seizure behavior and AD of 10 stimulations, and outputs were seizure behavior and AD of the immediate next stimulation. Therefore, there was no need for a neural network with a sophisticated memory.

Table 1.

Accuracy achieved by different models in predicting the AD response of dentate gyrus neurons to the last (eleventh) stimulation on each day based on ADs and seizure behaviors corresponding to the first ten stimulations on that day

Model AD responses Seizure behavior
FFNN 0.68 0.74
RNN 0.76 0.78
LSTM 0.78 0.80
Open in a new tab

In the second scenario, the three studied models were trained to predict the effects of each one of the eleven daily stimulations. These eleven AD time series were recorded within 1 hour. Tables 2 and 3 respectively show the results obtained by the three trained classifiers for prediction of seizure behaviors and AD responses of dentate gyrus on the sixth day (the last day of experiments). The first columns in these two tables reflect the temporal order of stimulations on the sixth day. Inputs to the models were the seizure behaviors and all AD responses of dentate gyrus to the stimulation of the perforant path during the first four days of experiments. Performance of the learning machine was examined based on its ability to predict the AD and the seizure behavior on the sixth day (last day of experiments) even when the data from the fifth day was missing, and therefore, was not fed to the network. Inputs were ADs and seizure behaviors recorded during the first four days of experiment and outputs were ADs and seizure behaviors of all eleven stimulations on the sixth day. As shown in Table 2, for prediction of seizure behavior, on average, LSTM provided the accuracy of 0.91±0.01, which is significantly better than the accuracy of simple RNN (0.77±0.02) and FFNN (0.59±0.02). For prediction of AD, as shown in Table 3, on average, LSTM, simple RNN, and FFNN provided the accuracy of 0.82±0.01, 0.74±0.08, and 0.42±0.1, respectively. In the second scenario, FFNN demonstrated a very poor performance compared to RNN and LSTM. In this scenario, dynamics plays a key role and the poor performance of the feedforward network is due to the absence of feedback paths. In feedforward networks, information moves in only one direction (forward path) from the input layer through the hidden layers to the output layer (Schmidhuber 2015). Therefore, FFNN cannot capture the dynamics of plasticity. Unlike FFNN, LSTM benefits from both feedback and memory, which enable it to achieve the best performance in prediction. Regarding these experiments, LSTM can be suggested as the method of choice among data-driven methods for epilepsy studies.

Table 2.

Performance of models for predicting all seizure behaviors on day 6

Stimulations FFNN RNN LSTM
1 0.41 0.67 0.80
2 0.70 0.68 0.91
3 0.58 0.71 0.91
4 0.55 0.68 0.90
5 0.55 0.75 0.90
6 0.64 0.71 0.91
7 0.64 0.86 0.96
8 0.56 0.85 1
9 0.68 0.94 1
10 0.69 0.89 0.94
11 0.58 0.78 0.80
Mean ± SD 0.59 ± 0.02 0.77 ± 0.02 0.91 ± 0.01
Open in a new tab

Table 3.

Performance of models for predicting all AD responses of dentate gyrus on day 6

Stimulations FFNN RNN LSTM
1 0.38 0.71 0.80
2 0.46 0.76 0.83
3 0.40 0.75 0.82
4 0.36 0.65 0.80
5 0.39 0.66 0.75
6 0.25 0.65 0.76
7 0.27 0.81 0.82
8 0.53 0.71 0.82
9 0.50 0.91 0.94
10 0.67 0.85 0.87
11 0.47 0.78 0.84
Mean ± SD 0.42 ± 0.10 0.74 ± 0.08 0.82 ± 0.01
Open in a new tab

Discussion

Brain contains a large-scale network of neurons with a massive number of connections and feedback loops. Such a network is a complex system that exhibits a wide range of dynamic behaviors due to the interactions between its different subsystems (Skarda and Freeman 1990). Complexity level of the brain does not allow to model it in a static manner using small-scale networks. However, complex models built on deep neural networks may be able to predict the function of brain to some extent. For instance, RNNs have the architectures similar to biological neural networks in which lateral and feedback connections are all over the circuits. Unlike FFNNs, RNNs can exploit their internal state (memory), which makes them capable of learning the stimulus history and discovering temporal patterns (Schäfer and Zimmermann 2007).

Whenever the gradient of the error function associated with a neural network is backpropagated through a unit of the network, it gets scaled by a certain factor, which is either greater or smaller than one. Therefore, the gradient may exponentially decay over time in a neural network, especially when the number of layers increases. Moreover, in synaptic plasticity, elapsing the time is an important issue because the strengthening and weakening of synapses occur over time. Thus, to simulate the synaptic plasticity, we need a model with memory that does not suffer from the vanishing gradient problem. LSTM by design addresses these two issues (Hochreiter and Schmidhuber 1997). In fact, if the computational graph of an RNN is unrolled across time, it will represent a deep neural network.

In this study, while changes in the seizure behavior indicated the plasticity in the motor circuit including motor cortex, basal ganglia, and cerebellum (Bostan and Strick 2018), the ADs indicated the plasticity in the perforant path. Most of the computationally-oriented research on epilepsy has been focused on epilepsy classification based on extracranial EEG in which accuracy of results heavily relies on the extracted features. Another category of computational work aims at predicting seizure based on intracranial and extracranial EEG, where emphasis would be on preictal features (Yuan et al. 2018). In both of these families of models, there is no need for deep learning architectures with memory and sophisticated gating mechanisms that control the flow of information, and even traditional machine learning algorithms can achieve good results (Li et al. 2018). Our study can be distinguished from the reported work in the literature from various aspects:

  • Here, LFP was used rather than EEG for modeling the plasticity of the perforant path. LFP is the measure of the electrical activity in the extracellular environment in the brain tissue by electrodes (Herreras 2016). While LFP is recorded in depth of the brain within the cortical tissue or other deep brain structures, EEG is the record of electrical activity at the surface of the scalp.

  • The LFP used in this study was the response of synapses in dentate gyrus to the perforant path stimulation. Unlike many studies, here, specific features were not extracted from LFP, but LFP was recorded to measure the duration of AD. The perforant path that has been modeled here is the part of hippocampal formation, which plays a critical role in memory and plasticity (Hyman et al. 1986). Its pathophysiological condition can be seen not only in epilepsy but also in other neural diseases such as stroke and Alzheimer’s disease (Thal et al. 2000; Bumanglag and Sloviter 2008). The stellate and pyramidal cells in entorhinal cortex, which give rise to the subiculum of the hippocampus, convey the processed information to the perforant pathway. These glutamatergic fibers finally terminate in the dentate gyrus (Witter et al. 2000). Predicting and modeling the function of perforant path can help in prediction of perforant behavior against new drugs and treatments.

  • Here, the motor circuit was modeled in epileptogenesis. Modeling the seizure behavior in epileptogenesis is a less-explored topic. In motor circuits, various brain regions are involved including motor cortex, basal ganglia, and cerebellum (Jueptner and Weiller 1998). Therefore, sophisticated computational tools are needed to cope with the challenging task of modeling their high-level and complex behavior. Predicting and modeling the seizure behavior of epileptic patients with different drugs are ideal for physicians and pharmacists.

Our dataset included the kindled group as the indicator of potentiation plasticity and the kindled +rTMS group as the indicator of depotentiation plasticity (Li et al. 2018). Results suggest that the memory mechanism implemented in the LSTM unit could model the synaptic plasticity of both potentiation and depotentiation in the brain motor circuits as well as changes in the plasticity of perforant path neurons during the kindling procedure. Our model could predict the AD of the sixth day even without using the fifth day’s measured data. This model can also be used for initial testing before experimental research. Therefore, such RNNs can be used as computational models for epilepsy, which is a topic for future work.

Conclusion

Modeling plasticity during epileptogenesis facilitates predicting the prolonged effect of antiepileptic drugs on kindling developments only by knowing their effects in primary days. This ultimately reduces the laboratory costs and saves time. The results of computational models can have a great impact on scientists’ decisions in choosing the type of intervention.

FFNN, simple RNN, and LSTM were examined for predicting motor circuit and the discharge responses of dentate gyrus following high-frequency stimulation of perforant pathway during epileptogenesis. Results of this study indicated that LSTM had the best performance on modeling the synaptic plasticity of both potentiation and depotentiation among the studied neural networks. LSTM could efficiently predict seizure stages based on AD and seizure behavior in a six-day period. This is due to the nature of kindling procedure, which is a time-dependent process. Synaptic plasticity is induced due to various changes taking place in synapses and their accumulated effects over time. Therefore, in order to capture the history and dynamics of this process, the neural network-based model needs to have a sophisticated memory mechanism and benefit from feedback as a facilitator of intelligence. Thus, LSTM with three gating mechanisms provides a reliable computational building block for modeling the kindling development.

Acknowledgements

Authors would like to express their gratitude to Dr. Zahra Ehsaei. They also thank the reviewers for their valuable comments, which have contributed to the improvement of the manuscript.

Data Availability

Data will be made available on reasonable request.

Declarations

Conflict of interest

Authors declare that they have no conflict of interest.

Footnotes

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. Akbarian B, Erfanian A. Automatic seizure detection based on nonlinear dynamical analysis of EEG signals and mutual information. Basic Clin Neurosci. 2018;9(4):227. doi: 10.32598/bcn.9.4.227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Beghi E. The epidemiology of epilepsy. Neuroepidemiology. 2020;54(2):185–191. doi: 10.1159/000503831. [DOI] [PubMed] [Google Scholar]
  3. Bengio Y. Learning deep architectures for AI. Norwell: Now Publishers Inc; 2009. [Google Scholar]
  4. Bertram E. The relevance of kindling for human epilepsy. Epilepsia. 2007;48:65–74. doi: 10.1111/j.1528-1167.2007.01068.x. [DOI] [PubMed] [Google Scholar]
  5. Bostan AC, Strick PL. The basal ganglia and the cerebellum: nodes in an integrated network. Nat Rev Neurosci. 2018;19(6):338–350. doi: 10.1038/s41583-018-0002-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bumanglag AV, Sloviter RS. Minimal latency to hippocampal epileptogenesis and clinical epilepsy after perforant pathway stimulation-induced status epilepticus in awake rats. J Comparat Neurol. 2008;510(6):561–580. doi: 10.1002/cne.21801. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Cain DP. Long-term potentiation and kindling: how similar are the mechanisms? Trends Neurosci. 1989;12(1):6–10. doi: 10.1016/0166-2236(89)90146-x. [DOI] [PubMed] [Google Scholar]
  8. Chen M, Ren W, Wang X. Depotentiation from potentiated synaptic strength in a tristable system of coupled phosphatase and kinase. Front Comput Neurosci. 2016;10:104. doi: 10.3389/fncom.2016.00104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cismondi F, Fialho AS, Vieira SM, Reti SR, Sousa JM, Finkelstein SN. Missing data in medical databases: impute, delete or classify? Artif Intell Med. 2013;58(1):63–72. doi: 10.1016/j.artmed.2013.01.003. [DOI] [PubMed] [Google Scholar]
  10. Citri A, Malenka RC. Synaptic plasticity: multiple forms, functions, and mechanisms. Neuropsychopharmacology. 2008;33(1):18–41. doi: 10.1038/sj.npp.1301559. [DOI] [PubMed] [Google Scholar]
  11. Dalic L, Cook MJ. Managing drug-resistant epilepsy: challenges and solutions. Neuropsychiatric Dis Treatment. 2016;12:2605. doi: 10.2147/NDT.S84852. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Faas GC, Vreugdenhil M, Wadman WJ. Calcium currents in pyramidal CA1 neurons in vitro after kindling epileptogenesis in the hippocampus of the rat. Neuroscience. 1996;75(1):57–67. doi: 10.1016/0306-4522(96)00254-0. [DOI] [PubMed] [Google Scholar]
  13. Fellous J-M, Sapiro G, Rossi A, Mayberg HS, Ferrante M. Explainable artificial intelligence for neuroscience: behavioral neurostimulation. Front Neurosci. 2019;13:1346. doi: 10.3389/fnins.2019.01346. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Fu R, Tian Y, Shi P, Bao T. Automatic detection of epileptic seizures in EEG using ssparse CSP and fisher linear discrimination analysis algorithm. J Med Syst. 2020;44(2):1–13. doi: 10.1007/s10916-019-1504-1. [DOI] [PubMed] [Google Scholar]
  15. Gao, W., Sun, L., and Tomizuka, M. (2017). A guided search framework in multiple model control. In: 2017 IEEE 56th annual conference on decision and control (CDC), IEEE, pp 3403–3410
  16. Goddard GV, McIntyre DC, Leech CK. A permanent change in brain function resulting from daily electrical stimulation. Exp Neurol. 1969;25(3):295–330. doi: 10.1016/0014-4886(69)90128-9. [DOI] [PubMed] [Google Scholar]
  17. Herreras O. Local field potentials: myths and misunderstandings. Front Neural Circuits. 2016;10:101. doi: 10.3389/fncir.2016.00101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Hinton GE. Training products of experts by minimizing contrastive divergence. Neural Comput. 2002;14(8):1771–1800. doi: 10.1162/089976602760128018. [DOI] [PubMed] [Google Scholar]
  19. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9(8):1735–1780. doi: 10.1162/neco.1997.9.8.1735. [DOI] [PubMed] [Google Scholar]
  20. Hughes JR. Post-tetanic potentiation. Physiol Rev. 1958;38(1):91–113. doi: 10.1152/physrev.1958.38.1.91. [DOI] [PubMed] [Google Scholar]
  21. Hyman BT, Van Hoesen GW, Kromer LJ, Damasio AR. Perforant pathway changes and the memory impairment of Alzheimers disease. Ann Neurol Official J Am Neurol Assoc Child Neurol Soc. 1986;20(4):472–481. doi: 10.1002/ana.410200406. [DOI] [PubMed] [Google Scholar]
  22. Jueptner M, Weiller C. A review of differences between basal ganglia and cerebellar control of movements as revealed by functional imaging studies. Brain J Neurol. 1998;121(8):1437–1449. doi: 10.1093/brain/121.8.1437. [DOI] [PubMed] [Google Scholar]
  23. Kale R (1997) Bringing epilepsy out of the shadows: Wide treatment gap needs to be reduced. British Med J, vol 315, p 2 [DOI] [PMC free article] [PubMed]
  24. Kim J-E, Cho K-O. The pilocarpine model of temporal lobe epilepsy and EEG monitoring using radiotelemetry system in mice. J Visual Exp. 2018;132:e56831. doi: 10.3791/56831. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Kriegeskorte N. Deep neural networks: a new framework for modeling biological vision and brain information processing. Ann Rev Vis Sci. 2015;1:417–446. doi: 10.1146/annurev-vision-082114-035447. [DOI] [PubMed] [Google Scholar]
  26. Lévesque M, Avoli M. The kainic acid model of temporal lobe epilepsy. Neurosci Biobehav Rev. 2013;37(10):2887–2899. doi: 10.1016/j.neubiorev.2013.10.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Li Y, Cui W, Luo M, Li K, Wang L. Epileptic seizure detection based on time-frequency images of EEG signals using gaussian mixture model and gray level co-occurrence matrix features. Int J Neural Syst. 2018;28(07):1850003. doi: 10.1142/S012906571850003X. [DOI] [PubMed] [Google Scholar]
  28. Lydon-Staley DM, Cornblath EJ, Blevins AS, Bassett DS. Modeling brain, symptom, and behavior in the winds of change. Neuropsychopharmacology. 2021;46(1):20–32. doi: 10.1038/s41386-020-00805-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Mody I. The molecular basis of kindling. Brain Pathol. 1993;3(4):395–403. doi: 10.1111/j.1750-3639.1993.tb00767.x. [DOI] [PubMed] [Google Scholar]
  30. Morimoto K, Fahnestock M, Racine RJ. Kindling and status epilepticus models of epilepsy: rewiring the brain. Prog Neurobiol. 2004;73(1):1–60. doi: 10.1016/j.pneurobio.2004.03.009. [DOI] [PubMed] [Google Scholar]
  31. Pfeiffer M, Pfeil T. Deep learning with spiking neurons: opportunities and challenges. Front Neurosci. 2018;12:774. doi: 10.3389/fnins.2018.00774. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Racine RJ. Modification of seizure activity by electrical stimulation: Ii motor seizure. Electroencephalography Clin Neurophysiol. 1972;32(3):281–294. doi: 10.1016/0013-4694(72)90177-0. [DOI] [PubMed] [Google Scholar]
  33. Richardson MP. Large scale brain models of epilepsy: dynamics meets connectomics. J Neurol Neurosurgery Psychiatry. 2012;83(12):1238–1248. doi: 10.1136/jnnp-2011-301944. [DOI] [PubMed] [Google Scholar]
  34. Rizvi S, Ladino LD, Hernandez-Ronquillo L, Téllez-Zenteno JF. rEpidemiology of early stages of epilepsy: risk of seizure recurrence after a first seizure. Seizure. 2017;49:46–53. doi: 10.1016/j.seizure.2017.02.006. [DOI] [PubMed] [Google Scholar]
  35. Samokhina E, Samokhin A. Neuropathological profile of the pentylenetetrazol (PTZ) kindling model. Int J Neurosci. 2018;128(11):1086–1096. doi: 10.1080/00207454.2018.1481064. [DOI] [PubMed] [Google Scholar]
  36. Schäfer AM, Zimmermann H-G. Recurrent neural networks are universal approximators. Int J Neural Syst. 2007;17(04):253–263. doi: 10.1142/S0129065707001111. [DOI] [PubMed] [Google Scholar]
  37. Schmidhuber J. Deep learning in neural networks: an overview. Neural Netw. 2015;61:85–117. doi: 10.1016/j.neunet.2014.09.003. [DOI] [PubMed] [Google Scholar]
  38. Schmoll H, Badan I, Grecksch G, Walker L, Kessler C, Popa-Wagner A. Kindling status in sprague-dawley rats induced by pentylenetetrazole: involvement of a critical development period. Am J Pathol. 2003;162(3):1027–1034. doi: 10.1016/S0002-9440(10)63897-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Schubert M, Siegmund H, Pape H-C, Albrecht D. Kindling-induced changes in plasticity of the rat amygdala and hippocampus. Learn Memory. 2005;12(5):520–526. doi: 10.1101/lm.4205. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Shahpari M, Mirnajafi-Zadeh J, Firoozabadi SMP, Yadollahpour A. Effect of low-frequency electrical stimulation parameters on its anticonvulsant action during rapid perforant path kindling in rat. Epilepsy Res. 2012;99(1–2):69–77. doi: 10.1016/j.eplepsyres.2011.10.023. [DOI] [PubMed] [Google Scholar]
  41. Shapshak P. Artificial intelligence and brain. Bioinformation. 2018;14(1):38. doi: 10.6026/97320630014038. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Skarda CA, Freeman WJ. Chaos and the new science of the brain. Concepts Neurosci. 1990;1(2):275–285. [Google Scholar]
  43. Sun C, Cui H, Zhou W, Nie W, Wang X, Yuan Q. Epileptic seizure detection with EEG textural features and imbalanced classification based on easyensemble learning. Int J Neural Syst. 2019;29(10):1950021. doi: 10.1142/S0129065719500217. [DOI] [PubMed] [Google Scholar]
  44. Svalheim S, Taubøll E, Bjørnenak T, Røste LS, Mørland T, Sætre ER, Gjerstad L. Onset of epilepsy and menarche—is there any relationship? Seizure. 2006;15(8):571–575. doi: 10.1016/j.seizure.2006.07.003. [DOI] [PubMed] [Google Scholar]
  45. Szyndler J, Maciejak P, Turzyńska D, Sobolewska A, Walkowiak J, Płaźnik A. The effects of electrical hippocampal kindling of seizures on amino acids and kynurenic acid concentrations in brain structures. J Neural Transmission. 2012;119(2):141–149. doi: 10.1007/s00702-011-0700-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Thal DR, Holzer M, Rüb U, Waldmann G, Günzel S, Zedlick D, Schober R. Alzheimer-related τ-pathology in the perforant path target zone and in the hippocampal stratum oriens and radiatum correlates with onset and degree of dementia. Exp Neurol. 2000;163(1):98–110. doi: 10.1006/exnr.2000.7380. [DOI] [PubMed] [Google Scholar]
  47. Wasterlain CG, Morin AM, Jonec V. Interactions between chemical and electrical kindling of the rat amygdala. Brain Res. 1982;247(2):341–346. doi: 10.1016/0006-8993(82)91259-8. [DOI] [PubMed] [Google Scholar]
  48. Wendling F. Computational models of epileptic activity: a bridge between observation and pathophysiological interpretation. Exp Rev Neurotherapeutics. 2008;8(6):889–896. doi: 10.1586/14737175.8.6.889. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Witter MP, Naber PA, van Haeften T, Machielsen WCM, Rombouts SARB, Barkhof F, Scheltens P, Lopes da Silva FH. Cortico-hippocampal communication by way of parallel parahippocampal-subicular pathways. Hippocampus. 2000;10(4):398–410. doi: 10.1002/1098-1063(2000)10:4<398::AID-HIPO6>3.0.CO;2-K. [DOI] [PubMed] [Google Scholar]
  50. Yadollahpour A, Firouzabadi SM, Shahpari M, Mirnajafi-Zadeh J. Repetitive transcranial magnetic stimulation decreases the kindling induced synaptic potentiation: effects of frequency and coil shape. Epilepsy Res. 2014;108(2):190–201. doi: 10.1016/j.eplepsyres.2013.11.023. [DOI] [PubMed] [Google Scholar]
  51. Yuan S, Zhou W, Chen L. Epileptic seizure prediction using diffusion distance and bayesian linear discriminate analysis on intracranial EEG. Int J Neural Syst. 2018;28(01):1750043. doi: 10.1142/S0129065717500435. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Data will be made available on reasonable request.

Articles from Cognitive Neurodynamics are provided here courtesy of Springer Science+Business Media B.V.

RESOURCES