Reducing Neuronal Networks to Discrete Dynamics

Abstract

We consider a general class of purely inhibitory and excitatory-inhibitory neuronal networks, with a general class of network architectures, and characterize the complex firing patterns that emerge. Our strategy for studying these networks is to first reduce them to a discrete model. In the discrete model, each neuron is represented as a finite number of states and there are rules for how a neuron transitions from one state to another. In this paper, we rigorously demonstrate that the continuous neuronal model can be reduced to the discrete model if the intrinsic and synaptic properties of the cells are chosen appropriately. In a companion paper [1], we analyze the discrete model.

1 Introduction

Oscillatory behavior arises throughout the nervous system. Examples include the thalamocortical system responsible for the generation of sleep rhythms [2–5], networks within the basal ganlgia that have been implicated in the generation of Parkinsonian tremor [6,7], and networks within the olfactory bulb of mammals, or antennal lobe of insects [8–10]. Each of these systems has been modeled as an excitatory-inhibitory neuronal network and each model displays complex firing patterns. Thalamocortical models, for example, may generate clustered activity in which the network breaks up into subpopulations of cells, or clusters; neurons within each cluster fire in near synchrony, while cells within different clusters fire out-of-phase with each other. A recently proposed model for neuronal activity within the antennal lobe of insects [9] displays dynamic clustering: cells fire during distinct episodes and during each episode some subpopulation, or cluster, of cells fire in near synchrony. However, the membership of clusters may change so that two cells may fire together during one episode but they do not fire together during some other subsequent episode.

Several papers have used dynamical systems methods to analyze synchronous and clustered activity in excitatory-inhibitory networks [11–18]. However, very few papers have studied mechanisms underlying dynamic clustering. Moreover, previous papers have typically considered small networks with very simple architectures. It remains poorly understood how population rhythms depend on the network architecture.

Here we consider a general class of excitatory-inhibitory networks, with a general class of architectures, and characterize the complex firing patterns that emerge. Our strategy for studying these networks is to first rigorously reduce them to a discrete model. In the discrete model, each neuron is represented by a finite number of states and there are rules for how a neuron transitions from one state to another. In particular, the rules determine when a neuron fires and how this affects the state of other neurons.

The goal of this paper is to demonstrate that certain types of neuronal models can, in fact, be rigorously reduced to the discrete model. In a companion paper [1], we analyze the discrete model. By studying the discrete model, we are able to characterize properties of the dynamics of the original neuronal system. We demonstrate in [1], for example, that these networks typically exhibit a large number of stable oscillatory patterns. We also determine how properties of the attractors depend on network parameters, including the underlying architecture.

This paper is organized as follows. In the next section, we describe the neuronal model and introduce the two types of networks we are going to study: purely inhibitory and excitatory-inhibitory networks. In Section 3, we introduce the basic fast/slow analysis that will be used throughout the paper. We also consider some simple networks that will motivate the discrete dynamics. The discrete model is formally defined in Section 4. In Section 5, we briefly describe why it may not be possible, in general, to reduce a purely inhibitory network to the discrete model. The main analysis is given in Section 6 where we find conditions on parameters for when an excitatory-inhibitory network can be rigorously reduced to the discrete model. The results of numerical simulations are presented in Section 7 and there is a discussion of our results in the last section.

2 The neuronal model

A neuronal network consists of three components. These are: (1) the individual cells within the network; (2) the synaptic connections between cells; and (3) the network architecture. We now describe how each of these components is modeled. The models are written in a rather general form since the analysis does not depend on the specific forms of the equations. A concrete example is given in Section 7.

Individual Cells

We consider a general two-variable model neuron of the form

$$\begin{array}{l}\frac{dv}{dt}=f(v,w)\\ \frac{dw}{dt}=\epsilon g(v,w).\end{array}$$ (1)

Here, v represents the membrane potential of the cell, w represents a channel gating variable and ε is a small, positive, singular perturbation parameter. We assume that the v-nullcline {f = 0} defines a cubic-shaped curve and the w-nullcline {g = 0} is a monotone increasing curve. Moreover, f > 0 (f < 0) below (above) the v-nullcline and g > 0 (< 0) below (above) the w nullcline.

We further assume that the v- and w-nullclines intersect at a unique fixed point p₀. If p₀ lies along the left branch of the v-nullcline, then p₀ is stable and the cell is said to be excitable. In the oscillatory case, p₀ lies along the middle branch and, if ε is sufficiently small, then the cell exhibits stable oscillations. In the limit ε → 0, the limit cycle approaches the singular trajectory shown in Figure 1. For most of this paper, we will assume that individual cells, without any coupling, are excitable.

Figure 1.

Singular trajectory for an oscillatory cell. Note that the fixed point p₀ lies along the middle branch of the cubic-shaped v-nullcline. During the silent and active phase, the singular trajectory lies on the left and right branch, respectively, of the v-nullcline. Transitions between the silent and active phase take place when the trajectory reaches a left or right knee of the v-nullcline. For most of the paper, we assume that the cell is excitable; that is, p₀ lies along the left branch of the v-nullcline.

Synaptic Connections

A pair of mutually coupled neurons is modeled as:

$$\begin{array}{l}\frac{d{v}_i}{dt}=f(v_i,w_i)-g_{\mathit{syn}}{s}_j(v_i-v_{\mathit{syn}})\\ \frac{d{w}_i}{dt}=\epsilon g(v_i,w_i)\\ \frac{d{s}_i}{dt}=\alpha (1-s_i)H(v_i-\theta )-\beta s_i\end{array}$$ (2)

where i and j are 1 or 2 with i ≠ j. Moreover, g_syn represents a constant maximal conductance, s_j corresponds to the fraction of open synaptic channels, and α and β represent rates at which the synapse turns on and turns off. Note that s_j depends on the presynaptic cell. To simplify the discussion, we will assume H(v) is the Heaviside step function.

Note that the coupling between cells is through the synaptic variables s_i. Suppose, for example, that cell 1 is the presynaptic cell. When cell 1 fires a spike, its membrane potential v₁ crosses the threshold θ and this results in activation of the synaptic variable s₁. More precisely, $s_1\to \frac{\alpha }{\alpha +\beta }$ at the rate α + β This then turns on the synaptic current to cell 2. When cell 1 is silent, so that v₁ < θ, then s₁ turns off at the rate β.

Synapses may be either excitatory or inhibitory. This depends primarily on the synaptic reversal potential v_syn. We say that the synapse is inhibitory if v_syn < v_i(t), i = 1, 2, along solutions of interest. In the excitatory case, v_syn > v_i(t).

Synapses may also be either direct or indirect. In a direct synapse, the postsynaptic receptor contains both the transmitter binding site and the ion channel opened by the transmitter as part of the same receptor. In an indirect synapse, the transmitter binds to receptors that are not themselves ion channels. Direct synapses are typically much faster than indirect synapses. The synapses we have considered so far are direct since they activate as soon as a membrane crosses the threshold. We model indirect synapses as described in [13]. We introduce a new independent variable x_i for each cell, and replace (2) with the following equations for each (x_i, s_i):

$$\begin{array}{l}\frac{d{x}_i}{dt}=\epsilon {\alpha }_x(1-x_i)H(v_i-\theta )-\epsilon {\beta }_x{x}_i\\ \frac{d{s}_i}{dt}=\alpha (1-s_i)H(x_i-{\theta }_x)-\beta s_i.\end{array}$$ (3)

The constants α_x and β_x are assumed to be independent of ε. The variable x corresponds to a secondary process that is activated when transmitters bind to the postsynaptic cell. The effect of the indirect synapses is to introduce a delay from the time one cell jumps up until the time the other cell feels the synaptic input. For example, if cell 1 fires, a secondary process is turned on when v₁ crosses the threshold θ. The synapse s₁ does not turn on until x₁ crosses some threshold θ_x; this takes a finite amount of time since x₁ evolves on the slow time scales, like the w_i.

Network Architecture

We will be interested in two types of networks; these are purely inhibitory (I-) networks and excitatory-inhibitory (E-I-) networks. First we consider I-networks. Then the network architecture can be viewed as a directed graph D = <V_D, A_D>. The vertices correspond to the neurons; for convenience of notation, we assume that V_D = [n] ≡ {1, …, n}. If there is an arc (or directed edge) e = <i, j>, then cell i sends inhibition to cell j. This network is modeled as:

$$\begin{array}{l}\frac{d{v}_i}{dt}=f(v_i,w_i)-g_{\mathit{syn}}(v_i-v_{\mathit{syn}})\sum s_j\\ \frac{d{w}_i}{dt}=\epsilon g(v_i,w_i)\\ \frac{d{s}_i}{dt}=\alpha (1-s_i)H(v_i-\theta )-\beta s_i\end{array}$$ (4)

We are assuming that the cells are homogeneous so that the nonlinear functions f and g do not depend on the cell i. The sum in (4) is over all presynaptic cells; that is, {j: <j, i> ∈ A_D}. Here we assume that v_syn is chosen so that the synapses are inhibitory.

Now consider E-I-networks. We assume that there are two populations of cells. These are the excitatory (E-) cells and the inhibitory (I-) cells. The E-cells send excitation to some subset of I-cells. The I-cells send inhibition to some subset of E-cells as well as to I-cells. In the analysis which follows, We will need to assume that inhibitory synapses, corresponding to I →E and I →I connections, are indirect, while E →I connections are realized by direct synapses. Figure 2 shows the network. In Section 6.1, we write the equations.

Figure 2.

An excitatory-inhibitory network. Each I-cell sends inhibition to some subset of E-cells as well as to I-cells. Each E-cell sends excitation to some subset of I-cells.

3 Some example networks

We now consider some rather simple examples of inhibitory networks. These examples will be used to describe the sorts of solutions that we are interested in. The examples will also allow us to explain how the dynamics corresponding to the system of differential equations will be reduced to a discrete model. It will first be necessary to introduce some notation.

3.1 Some notation

Let Φ(v, w, s) ≡ f(v, w) − g_syns(v − v_syn). Then the right hand side of the first equation in (4) is Φ (v_i, w_i, Σs_j) where the sum is over all cells presynaptic to cell i. If g_syn and s are not too large, then each ${\mathcal{C}}_s=\{\mathrm{\Phi }(v,w,s)=0\}$ defines a cubic-shaped curve. We will sometimes write $\mathcal{C}(K)$ to denote the cubic corresponding to synaptic input from K other active cells. We express the left branch of $\mathcal{C}(K)$ as {v = Φ_L(w, K)} and the right branch of $\mathcal{C}(K)$ as {v = Φ_R(w, K)}. Suppose that the left and right knees of $\mathcal{C}(K)$ are at w = w_L(K) and w = w_R(K), respectively.

Throughout this paper, we will use geometric singular perturbation methods to analyze solutions. That is, we construct singular trajectories for the limit when ε = 0. Recall, for example, the singular trajectory for the single oscillatory cell shown in Figure 1. While the cell is in either the silent or active phase, the singular trajectory lies on either the left or right branch of the cubic-shaped v-nullcline, respectively. The jump-up to the active phase or the jump-down to the silent phase occurs when the singular trajectory reaches the left or right knee of the v-nullcline.

In larger networks, the singular trajectory corresponding to each cell will also lie along either the left or right branch of a cubic-shaped v-nullcline during the silent or active phase. The jumps up and down between the silent and active phases occur when a singular trajectory reaches the left or right knee of some cubic. One can view the synaptic input as moving the cubic-shaped nullcline up or down, meaning that there is a family of cubic-shaped nullclines, depending on the synaptic inputs s_i. Which cubic a cell lies on depends on how many active cells it receives synaptic input from.

3.2 Post-inhibitory rebound

It is well known that two cells coupled through mutual inhibition can generate antiphase oscillations through the mechanism known as post-inhibitory rebound. This phenomenon arises in a variety of neuronal systems [19,20]. Along an antiphase solution, the cells take turns firing action potentials; when one cell jumps down, it releases the other cell from inhibition and that other cell then jumps up. This mechanism will play a central role in our analysis of larger networks. For this reason, we will briefly describe the geometric construction of a singular trajectory corresponding to the antiphase solution.

The singular trajectory is shown in Figure 3. There are two trajectories; these correspond to the projections of (v₁, w₁) and (v₂, w₂) onto the (v, w)- phase plane. Note that each cell, without any coupling, is excitable. The w-nullcline intersects both ${\mathcal{C}}_0$ and ${\mathcal{C}}_1$ along their left branches. Moreover, cells cannot fire unless they receive inhibitory input first.

Figure 3.

Post-inhibitory rebound. The cells take turns firing. When one cell jumps down, it releases the other cell from inhibition. If, at this time, the inhibited cell lies below the left knee of ${\mathcal{C}}_0$, then the inhibited cell will jump up to the active phase.

We now step through the construction of the singular trajectory corresponding to the antiphase solution. We begin with cell 1 at the right knee of ${\mathcal{C}}_0$ ready to jump down. We further assume that cell 2 is silent and lies along the left branch of ${\mathcal{C}}_1$ below the left knee of ${\mathcal{C}}_0$. When cell 1 jumps down, s₁ → 0. Note that s₁ → 0 instantaneously with respect to the slow time scale. Since (v₂, w₂) lies below the left knee of ${\mathcal{C}}_0$, cell 2 exhibits post-inhibitory rebound and jumps up to the right branch of ${\mathcal{C}}_0$.

Cell 2 then moves up the right branch of ${\mathcal{C}}_0$ and cell 1 moves down the left branch of ${\mathcal{C}}_1$ towards p₁. Eventually, cell 2 reaches the right knee of ${\mathcal{C}}_0$ and jumps down. If at this time, cell 1 lies below the left knee of ${\mathcal{C}}_0$, then it jumps up due to post-inhibitory rebound. The roles of cell 1 and cell 2 are now reversed. The cells continue to take turns firing when they are released from inhibition. Note that the time each cell spends in the active phase must be sufficiently long. This gives the silent cell enough time to evolve along the left branch of ${\mathcal{C}}_1$ to below the left knee of ${\mathcal{C}}_0$ so that it is ready to jump up when it is released from inhibition.

3.3 A larger inhibitory network

The next example network consists of seven cells and the network architecture is shown in Figure 4. All connections are assumed to be inhibitory with direct synapses. Two different responses, for the same parameter values but different initial conditions, are shown in the figure. Each response consists of discrete episodes in which some subset of the cells fire in near synchrony. These subsets change from one episode to the next; moreover, two different cells may belong to the same subset for one episode but belong to different subsets during other episodes. These solutions correspond to dynamic clustering. Note that cells fire due to post-inhibitory rebound.

Figure 4.

An example inhibitory network with seven cells. The left panel shows the graph of the network architecture. Cell 1, for example, sends inhibition to cells 4 and 5. Subsets or clusters of cells fire in distinct episodes. Each horizontal row in the middle panels represents the time course of a single cell. A black rectangle indicates when the cell is active. In the right panel, we keep track of which cells fire during each subsequent episode. The equations and parameters used are precisely those described in Section 7 except $g_{\mathit{syn}}^I=.4$, α_x = 1, β_x = 4, I = 16 for the E-cells and I = 10 for the I-cells.

Consider, for example, the solution labeled (A). During the first episode, cells 1 and 6 fire action potentials and the cells that fire during the second episode are 4 and 5. After this transient period, the response becomes periodic. Note that the cells which fire during the third episode are cells 2, 3 and 7. These are precisely the same cells that fire during the eighth episode. This subset of cells continues to fire together every fifth episode.

Now consider the solution labeled (B). This solution has the same periodic attractor as the solution shown in the first panel, although the initial response is different. These two solutions demonstrate that two responses may have different initial transients but approach the same periodic attractor. There are, in fact, many periodic attractors. In the next section, we will demonstrate that this example exhibits seven periodic attractors.

4 The discrete model

As the examples presented in the previous section illustrate, solutions of the neuronal model may consist of discrete episodes. During each episode, some subset of cells fire in near synchrony; moreover, this subset changes from one episode to another. This suggests that we can reduce the neuronal model to discrete dynamics that keeps track of which cells fire during each discrete episode. In this section, we shall formally define the discrete dynamics. We then define what it means that the neuronal system can be reduced to the discrete model. We will then state the main result of this paper. In later sections, we shall describe conditions for when the neuronal model can be rigorously reduced to the discrete model. In [1], we embark upon a rigorous analysis of the discrete model.

4.1 Definition of the discrete model

First we consider a purely inhibitory network. In order to derive the discrete dynamics, we need to make two assumptions on solutions of the neuronal model. Later we will describe when parameters in the neuronal model can be chosen so that these assumptions are satisfied. The first assumption is that there is a positive integer p such that every cell has refractory period p. That is, if a cell fires during an episode then it cannot fire during the next p subsequent episodes. The second assumption is that a cell must fire during an episode if it received inhibitory input from an active cell during the previous episode and if it has not fired during the previous p episodes.

Consider, for example, the example shown in the first panel of Figure 4. Here, p = 1. Cells 1 and 6 fire during the first episode. Both of these cells send inhibition to cells 4 and 5 so, by the second assumption, both of these cells must fire during the second episode. During the third episode, cells 2, 3 and 7 fire. Note that cell 2 sends inhibition to cell 7; however, cell 7 cannot fire during the fourth episode because it fired during the third. Continuing in this manner, we can determine which subset of cells fire during each subsequent episode.

We now formally define the discrete model. Assume that we are given a directed graph D = <V_D, A_D> and a positive integer p. The vertices of D signify neurons and an arc <v₁, v₂> ∈ A_D signifies a synaptic connection from neuron v₁ to neuron v₂. We associate a discrete-time, finite-state dynamical system S_N with a network N = <D, p>. A state s→ of the system at the discrete time κ is a vector s→ (κ) = [P₁(κ), …, P_n(κ)] where P_i(κ) ∈ {0, 1, …, p} for all i ∈ [n]. The state P_i(κ) = 0 of neuron i is interpreted as firing at time κ.

The dynamics on S_N is defined as follows:

• (D1) If P_i(κ) < p, then P_i(κ + 1) = P_i(κ) + 1.

• (D2) If P_i(κ) = p, and there exists a j ∈ [n] with P_j(κ) = 0 and <j, i> ∈ A_D, then P_i(κ + 1) = 0.

• (D3) If P_i(κ) = p and there is no j ∈ [n] with P_j(κ) = 0 and <j,i> ∈ A_D, then P_i(κ + 1) = p.

Recall that a cell ‘fires’ when P_i(κ) = 0. (D1) implies that after a cell fires, its state P_i increases by one unit each episode until P_i = p; at this time, the cell is ready to fire again. (D2) implies that if a cell is ready to fire at time κ, then it will do so at time κ + 1 if it receives input from some cell that has fired at time κ. Finally, (D3) states that even if a cell is ready to fire, it will not do so unless it receives input from some active cell.

Note that the dynamical system S_N can be viewed as a directed graph on its state space. This state transition digraph is different from D, the network connectivity graph. If there are n cells, then there are (p + 1)ⁿ states. For the example discussed in Section 3.3, there are seven cells and p = 1. Hence, there are 128 states. The entire directed graph corresponding to the discrete dynamics is shown in Figure 5. In the figure we have changed notation in order to simplify it. At each node, we list those cells which fire; these are the cells with P_i = 0.

Figure 5.

Discrete dynamics corresponding to the network shown in Figure 4

Now consider an E-I-network. We formally reduce this to an I-network by constructing a directed graph D whose nodes correspond to the E-cells. Suppose that there are n E-cells which we label as {E_i: i ∈ [n]}. For any i, j ∈ [n], we assume that there is an edge <i, j> ∈ D if there is an I-cell, say I_k, such that there exist both E_i → I_k and I_k → E_j connections. Now that we have the network connectivity graph D, we can consider the discrete-time dynamical system S_N = <D, p> defined above.

The definition of the discrete dynamics for the E-I-network is based on two assumptions. Suppose that E_i and E_j are any E-cells and I_k is any I-cell such that there exist both E_i → I_k and I_k → E_j connections. The first assumption is that if E_i fires during an episode, then so will I_k. The second assumption is that if I_k fires during an episode, then E_j will fire during the next episode if and only if it has not fired during the previous p episodes.

4.2 Reduction from the neuronal model to the discrete model

Our goal is to find conditions on parameters for when the neuronal system generates dynamics that is consistent with a discrete model. Here we give a more precise definition for what it means that the neuronal system can be reduced to the discrete model. We only consider purely inhibitory networks in this subsection; a similar definition holds for E-I-networks.

Consider any network with any fixed architecture and fix p, the refractory period. We can then define both the continuous neuronal and discrete models, as was done in the preceding sections. Let s→ be any state of the discrete model. We then wish to show that there exists a solution of the neuronal system in which different subsets of cells take turns jumping up to the active phase. The active cells during each subsequent episode are precisely those determined by the discrete orbit s→(κ), and this exact correspondence to the discrete dynamics remains valid throughout the trajectory of the initial state. We will say that such a solution realizes the orbit predicted by the discrete model. This solution will be stable in the sense that there is a neighborhood of the initial state such that every trajectory that starts in this neighborhood realizes the same discrete orbit.

4.3 The main result

We now formally state our main result.

Theorem

Suppose we fix n, corresponding to the size of the network, and p, the refractory period. Consider any excitatory-inhibitory network such that both the number of E-cells and the number of I-cells are bounded by n. Assume that the E-cells and the I-cells are modeled by the equations that satisfy the assumptions spelled out in Subsection 6.4. We further assume that there is all-to-all coupling among the I-cells, the excitatory synapses are direct and the inhibitory synapses are indirect. Finally, we consider any architecture of E → I and I → E connections. We can then define both the continuous and the discrete models, as was done in the preceding sections. Then there are intervals for the choice of the intrinsic parameters of the cells and the synaptic parameters so that:

1. Every orbit of the discrete model is realized by a stable solution of the differential equations model.

2. Every solution of the differential equations model eventually realizes a periodic orbit of the discrete model. That is, if X(t) is any solution of the differential equations model, then there exists T > 0 such that the solution {X(t):t > T} realizes a periodic orbit or a steady state of the discrete model.

We remark that the intrinsic and synaptic parameters will not depend on the network architecture. Hence, every orbit of the discrete model, for any network architecture, can be realized by a solution of the neuronal model. Moreover, every attractor of the differential equations model corresponds to a periodic orbit of the discrete model.

In Section 7 we give a concrete example of equations that satisfy the assumptions spelled out in Subsection 6.4. As we shall see in the proof of the theorem, what is important are the positions of the left and right knees of the cubic-shaped nullclines, the rates at which the slow variables evolve during the silent and active phases and the strengths of the synaptic connections.

The proof of the Theorem is constructive in the sense that we give precise bounds on the parameters. In particular, the proof leads to an estimate for the time-duration corresponding to each episode of the discrete model (see inequality (16)).

5 A problem with inhibitory networks

We now discuss an example that illustrates difficulties that arise in I-networks. This example suggests that it is not possible, in general, to reduce the dynamics of an I-network to a discrete model. In the next section, we demonstrate that these difficulties can be overcome in E-I-networks.

Consider the inhibitory network shown in Figure 6. The discrete dynamics predicts that there is a solution in which the network breaks up into two clusters; Cluster A consists of cells 1 and 2 and cluster B consists of cells 3 and 4. There is actually no problem in demonstrating that such a solution exists. This solution is very similar to the antiphase solution described in the preceding section. The clusters take turns jumping up to the active phase when they are released from inhibition by the other cluster.

Figure 6.

An example network used to illustrate difficulties that arise with inhibitory networks.

We next consider the stability of this solution. The following analysis will, in fact, suggest that this solution is not stable. In order to discuss stability, we need to define a notion of distance between two cells. Here, we define the distance between cells (v₁, w₁) and (v₂, w₂) to be simply |w₁ − w₂|.

Suppose we start so that cells within each cluster are very close to each other as shown in Figure 7. Note that Cluster A is active and lies along the right branch of ${\mathcal{C}}_1$. Cell 1 is at a right knee ready to jump down. Cluster B is silent and lies along the left branch of ${\mathcal{C}}_1$ very close to the point p₁ where ${\mathcal{C}}_1$ intersects the w-nullcline. We now step through what happens once cell 1 jumps down. We will demonstrate that instabilities can arise in both the jumping-down and jumping-up processes.

Figure 7.

A singular trajectory corresponding to the network shown in Figure 6. (Left Panel) Cells 1 and 2 and cells 3 and 4 are initially close to each other with cell 1 at a right knee ready to jump down. (Right Panel) The trajectory until cell 2 reaches a right knee and jumps down.

When cell 1 jumps down, it stops sending inhibition to cells 2 and 3. Cell 3 responds by jumping up to the active phase. Cell 2, on the other hand, moves to the right branch of ${\mathcal{C}}_0$. During the next step, cells 2 and 3 move up the right branch of ${\mathcal{C}}_0$, while cell 1 moves down the left branch of ${\mathcal{C}}_2$ and cell 4 move down the left branch of ${\mathcal{C}}_1$. During this time, both the distances between cells 1 and 2 and between cells 3 and 4 increase. This continues until cell 2 reaches the right knee of ${\mathcal{C}}_0$ and jumps down. At this time, cell 4 jumps up. Then both cells 1 and 2 lie in the silent phase and cells 3 and 4 are active. However, cells within each cluster are further apart from each other than they were initially. This expansion in the distance between cells within each cluster may continue and destabilize the clustered solution.

Note that we have not rigorously demonstrated that the network shown in Figure 6 cannot reproduce the dynamics predicted by the discrete model for some choice of parameter values. The analysis shows that instabilities can arise in the singular limit as ε →0. On the other hand, numerical simulations indicate that it may be possible to choose parameters (with ε bounded away from zero) so that this network does reproduce the discrete dynamics. However, the dynamics is quite sensitive to changes in parameter values. In contrast, for the excitatory-inhibitory networks described in the next section, we are able to rigorously prove that under certain conditions the network will always reproduce the dynamics of the discrete model.

6 E-I-networks

We now consider E-I-networks and find conditions when they generate dynamics corresponding to that of the discrete model.

6.1 The equations

The architecture of E-I-network is shown in Figure 2. Recall that the E-cells send excitation to some subpopulation of I-cells, while the I-cells inhibit some subpopulation of E-cells. We will need to assume that each I-cell sends inhibition to every I-cell, including itself. The excitatory synapses are direct, while the inhibitory synapses are indirect.

The equations for the E-cells can then be written as:

$$\begin{array}{l}\frac{d{v}_i}{dt}=f(v_i,w_i)-g_{IE}{S}_{IE}(v_i-v_{\mathit{syn}}^I)\\ \frac{d{w}_i}{dt}=\epsilon g(v_i,w_i)\\ \frac{d{s}_i}{dt}=\alpha (1-s_i)H(v_i-\theta )-\beta s_i\end{array}$$ (5)

and the equations for the I-cells can be written as:

$$\begin{array}{l}\frac{d{v}_i^I}{dt}=f_I(v_i^I,w_i^I)-g_{EI}{S}_{EI}(v_i^I-v_{\mathit{syn}}^E)-g_{II}{S}_{II}(v_i^I-v_{\mathit{syn}}^I)\\ \frac{d{w}_i^I}{dt}=\epsilon g_I(v_i^I,w_i^I)\\ \frac{d{x}_i^I}{dt}=\epsilon {\alpha }_x(1-x_i^I)H(v_i^I-{\theta }_I)-\epsilon {\beta }_x{x}_i^I\\ \frac{d{s}_i^I}{dt}={\alpha }_I(1-s_i^I)H(x_i^I-{\theta }_x)-{\beta }_I{s}_i^I\end{array}$$ (6)

Here,

$$S_{IE}=\sum s_j^I,\mathrm{\hspace{0.17em} }{S}_{EI}=\sum s_j,\text{ and }{S}_{II}=\sum s_j^I$$

where each sum is taken over the corresponding presynaptic cells. The reversal potentials $v_{\mathit{syn}}^E$ and $v_{\mathit{syn}}^I$ correspond to inhibitory and excitatory synapses, respectively. Note that we are assuming that all of the excitatory cells are the same, as well as all of the inhibitory cells; however, the excitatory cells need not be the same as the inhibitory cells.

6.2 Strategy

Suppose we are given an E-I-network as described in Section 6.1 and a positive integer p, corresponding to the refractory period. Let S_N be the discrete-time dynamical system as defined in Section 4. Here we describe our strategy for showing that the neuronal model reproduces the dynamics of the discrete model. The analysis will be for singular solutions. Along the singular solution, all of the cells lie on either the left or right branch of some cubic during the silent and active phase, respectively, except when a cell is jumping up or down. In order to construct the singular solutions, we first introduce a slow time variable η = εt and then set ε = 0. This leads to reduced equations for just the slow variables. The reduced equations determine the evolution of the slow variables as a cell evolves along the left or right branch of some cubic. Once we construct the singular solutions, it is necessary to demonstrated that these singular solutions perturb to actual solutions of (5) and (6) with ε > 0. This analysis is straightforward but rather technical and we will not present the details here. A similar analysis can be found in [14].

We will construct disjoint intervals $J_k=(W_k^1,W_k^2)$, k = 1, …, p, and J₀ = (0, W₀) with the following properties. Let s→(0) = (P₁, P₂, …, P_n) be any state of the discrete model. Assume that when η = 0,

• (A1) If P_i(0) = k, then w_i(0) ∈ J_k.

• (A2) If P_i(0) = 0, then E_i lies in the active phase.

• (A3) If P_i(0) > 0, then E_i lies in the silent phase.

• (A4) Every I-cell lies in the silent phase ready to jump up.

• (A5) Each $x_i^I(0)\le {\theta }_x$.

Then there exists T_* > 0 such that when η = T_*,

• (B1) If P_i(1) = k, then w_i(T_*) ∈ J_k.

• (B2) If P_i(1) = 0, then E_i lies in the active phase.

• (B3) If P_i(1) > 0, then E_i lies in the silent phase.

• (B4) Every I-cell lies in the silent phase ready to jump up.

• (B5) Each $x_i^I(T_{\ast })\le {\theta }_x$.

• (B6) The only E-cells that are both active and jump down in the interval [0, T_*] are those with P_i(0) = 0.

In (A4) and (B4), we state that the I-cells are “ready to jump up.” By this we mean that each I-cell will jump up to the active phase if it receives excitatory input from some active E-cell. A more precise condition that the I-cells must satisfy will be given shortly, once we introduce some more notation.

Note that we can then keep repeating this argument to show that the solution of the neuronal model realizes the orbit predicted by the discrete model.

6.3 Slow equations and some notation

The first step in the analysis is to reduce the dynamics of each cell to equations for the slow variables. We introduce the slow variable η = εt and then let ε = 0 to obtain the reduced equations:

$$\begin{array}{l}0=f(v_i,w_i)-g_{IE}{S}_{IE}(v_i-v_{\mathit{syn}}^I)\\ {\stackrel{.}{w}}_i=g(v_i,w_i)\\ 0=\alpha (1-s_i)H(v_i-{\theta }_E)-\beta s_i\\ 0=f_I(v_i^I,w_i^I)-g_{EI}{S}_{EI}(v_i^I,v_{\mathit{syn}}^E)-g_{II}{S}_{II}(v_i^I,v_{\mathit{syn}}^I)\\ {\stackrel{.}{w}}_i^I=g_I(v_i^I,w_i^I)\\ {\stackrel{.}{x}}_i^I={\alpha }_x(1-x_i^I)H(v_i^I-{\theta }_I)-{\beta }_x{x}_i^I\\ 0={\alpha }_I(1-s_i^I)H(x_i^I-{\theta }_x)-{\beta }_I{s}_i^I\end{array}$$ (7)

Differentiation is with respect to η.

We assume that α_x is sufficiently large; in particular, α_x ≫ β_x. It follows that if $v_i^I>{\theta }_I$, then $x_i^I\to {\alpha }_x({\alpha }_x+{\beta }_x)\approx 1$, while if $v_i^I>{\theta }_I$, then $x_i^I$ decays at the rate β_x. Note that once an I-cell stops firing, there is a delay, which we denote as Δ, until it releases other cells from inhibition. This delay is determined by the slow variables, $x_i^I$; if α_x is sufficiently large, then

$$\mathrm{\Delta }\approx -\frac{1}{{\beta }_x}ln{\theta }_x.$$ (8)

The first and fourth equations in (7) state that the E- and I-cells lie on some cubic. Which cubic a cell lies on depends on the number of inputs it receives from active presynaptic cells. Note that if E_i is active, so that v_i > θ_i, then s_i = α/(α + β) ≡ σ_E. If I_i is active, and $x_i^I>{\theta }_x$, then $s_i^I={\alpha }_I/({\alpha }_I+{\beta }_I)\equiv {\sigma }_I$. Suppose that E_i receives input from J active I-cells, and I_i receives input from K active E-cells and M active I-cells. Then

$$S_{IE}=J{\sigma }_I,\mathrm{\hspace{0.17em} }\mathrm{\hspace{0.17em} }{S}_{IE}=K{\sigma }_E\text{ and }{S}_{II}=M{\sigma }_I.$$

Let

$$\begin{array}{l}\mathrm{\Phi }(v,w,J)\equiv f(v,w)-g_{IE}(J{\sigma }_I)(v-v_{\mathit{syn}}^I),\\ {\mathrm{\Phi }}_I(v,w,K,M)\equiv f_I(v,w)-g_{EI}(K{\sigma }_E)(v-v_{\mathit{syn}}^E)-g_{II}(M{\sigma }_I)(v-v_{\mathit{syn}}^I).\end{array}$$

Then the first and fourth equations in (7) can be written as

$$\mathrm{\Phi }(v_i,w_i,J)=0\text{ and }{\mathrm{\Phi }}_I(v_i^I,w_i^I,K,M)=0$$

If g_IE, g_EI and g_II are not too large, then these define cubic-shaped curves denoted by $\mathcal{C}(J)$ and $\mathcal{C}(K,M)$, respectively. We express the left and right branches of $\mathcal{C}(J)$ as {v = Φ_L(w, J)} and {v = Φ_R(w, J)} and the left and right knees of $\mathcal{C}(J)$ as w = w_L(J) and w = w_R(J), respectively. The left and right branches, along with the left and right knees, of $\mathcal{C}(K,M)$ can be expressed as w_L(K,M) and w_R(K,M), respectively.

We can now write the second and fifth equations of (7) as

$$\begin{array}{l}{\stackrel{.}{w}}_i=g({\mathrm{\Phi }}_{\xi }(w_i,J),w_i)\\ {\stackrel{.}{w}}_i^I=g_I({\mathrm{\Phi }}_{\xi }^I(w_i^I,K,M),w_i^I).\end{array}$$ (9)

Here, ξ = L or R depending on whether the cell is silent or active. These are scalar equations for the evolution of the slow variables w_i and $w_i^I$.

6.4 Assumptions

In many neuronal models, the nonlinear function g(v, w) is of the form

$$g(v,w)=(w_{\infty }(v)-w)/\tau (v)$$ (10)

where w_∞(v) is nearly a step function. We assume that g(v, w) is of this form. Moreover, there exist V₁ < V₂ and constants τ₁ and τ₂ such that if v < V₁, then w_∞(v) = 0 and τ(v) = τ₁; if v > V₂, then w_∞(v) = 1 and τ(v) = τ₂.

We need to assume that the nullclines are such that E-cells can fire due to post-inhibitory rebound. We assume that $\mathcal{C}(0)$ intersects the w-nullcline at some point, p_A = (v_A, w_A), where V₁ < v_A < V₂. Moreover, the left knee of $\mathcal{C}(0)$ is at some point (v_L(0), w_L(0)) where w_L(0) > 0.

We also assume that the left branches of the inhibited cubics lie in the region where v < V₁. Finally, we assume that the right branches of each cubic $\mathcal{C}(J)$ for J ≥ 0 lie in the region v > V₂.

These assumptions imply that the slow dynamics (9) do not depend on which cubic the cell lies on, unless the cell is on the uninhibited cubic $\mathcal{C}(0)$. That is, if E_i lies in the silent phase, and this cell receives some inhibitory input, then w_i satisfies the simple equation

$$\stackrel{.}{w}=-w/{\tau }_1.$$ (11)

We will assume that V₂ − V₁ is small. Hence, even if E_i does not receive any inhibitory input, then, except in some small neighborhood of the fixed point p_A, w_i also satisfies (11). If E_i is active, then w_i satisfies

$$\stackrel{.}{w}=(1-w)/{\tau }_2.$$ (12)

Similarly, we assume that g_I(v, w) can be written as

$$g_I(v,w)=(w_{\infty }^I(v)-w)/{\tau }^I(v).$$

Moreover, there exist constants $V_1^I<V_2^I,{\tau }_1^I$ and ${\tau }_2^I$ such that if $v<V_1^I$, then $w_{\infty }^I(v)=0$ and ${\tau }^I(v)={\tau }_1^I$; if $v>V_2^I$, then $w_{\infty }^I(v)=1$ and ${\tau }^I(v)={\tau }_2^I$. For each cubic $\mathcal{C}(K,M)$ that is inhibited (M > 0) or does not receive excitation (K = 0), the part of the left branch above the v-axis lies in the region $v>V_1^I$. The right branch of each cubic lies in the region $v>V_2^I$.

Finally, we need to assume that an I-cell will fire only when it receives excitatory input from an E-cell and it does not receive inhibitory input from some other I-cell. This will be the case if the nullclines corresponding to the I-cells are as shown in Figure 9. Suppose that the left knee of ${\mathcal{C}}^I(K,M)$ is at $w_L^I(K,M)$; recall that K and M are the number of excitatory and inhibitory inputs that the I-cell receives. We assume that $w_L^I(K,M)<0$ if either K = 0 or M ≥ 1. Moreover, $w_L^I(K,0)>0$ if K ≥ 1.

Figure 9.

Assumptions on the nullclines for the I-cells.

We can now give a more precise statement of (A4). We assume that when η = 0, every I-cell lies in the silent phase with $w_i^I(0)<w_L^I(1,0)$. For the assumptions just given, this implies that each I-cell will jump up to the active phase if it receives input from an active E-cell. A more precise statement of (B4) is that when η = T_*, every I-cell lies in the silent phase with $w_i^I(T_{\ast })<w_L^I(1,0)$.

6.5 Analysis

We now step through the solution keeping track of where cells are in phase space. We must keep track of where and when cells jump up and down and where they lie along the left branches of cubics in the silent phase. The sets J_k and the constant T_* will be defined as they are needed in the analysis.

When do I-cells jump up?

We note that when one I-cell jumps up, it sends inhibition to every I-cell, including itself. However, because the inhibitory synapses are indirect, there is a delay from when one I-cell jumps up or down and other I-cells receive or are released from the resulting inhibition. These delays are modeled by the auxiliary variables x_i. At time η = 0, the last of these variables crosses the threshold for releasing all I-cells from inhibition. For this reason, all of the I-cells are able to jump up when η = 0, as long as they receive excitatory input. Let I⁰ be those I-cells that receive input from at least one E_i with P_i(0) = 0. We have assumed in (A4) that each of these cells lies below the left knee of $\mathcal{C}(1,0)$. It follows that every cell in I⁰ is induced to jump up precisely when η = 0.

We note that one potential problem is that when an I-cell, say $(v_i^I,w_i^I)$, jumps up and $v_i^I$ crosses the threshold θ_I, then inhibition to all of the I-cells is turned back on. Hence, some of the I-cells that receive excitatory input may be induced to “turn around” and return to the silent phase. Note that it is the middle branch of the corresponding cubic that separates those I-cells that will return to the silent phase from those that will continue to jump up to the active phase: I-cells that lie to the left of the middle branch will return to the silent phase, while I-cells to the right of the middle branch will continue to jump up. For this reason, we need to choose θ_I so that it is to the right of the middle branch of the corresponding cubics. This will guarantee that inhibition is not turned back on until the I-cells are committed to fire.

When do I-cells jump down?

Note that active I-cells may lie on different cubics and jump down at different right knees. Choose constants $w_R^I(\mathit{min})<w_R^I(\mathit{max})$ so that when an I-cell jumps down, it does so with $w_R^I(\mathit{min})<w<w_R^I(\mathit{max})$. Each I-cell in I⁰ jumps up at η = 0 with $0<w<w_L^I(1,0)$. Moreover, while in the active phase, w^I(η) satisfies

$$\stackrel{.}{w}=(1-w^I)/{\tau }_2^I.$$

It follows that every I-cell in I⁰ jumps down at some time T_ijd that depends on the cell and always satisfies

$$T_I\le T_{\mathit{ijd}}\le T_I+{\delta }_I.$$ (13)

where

$$T_I={\tau }_2^{I}ln\frac{1-w_L^I(1,0)}{1-w_R^I(\mathit{min})}\text{ and }{\delta }_I={\tau }_2^{I}ln\frac{1-w_R^I(\mathit{min})}{(1-w_R^I(\mathit{max}))(1-w_L^I(1,0))}$$ (14)

We will need to assume that δ_I < Δ. Later we will find conditions on parameters for when this is the case. Let $T_{\ast }^I$ be the time when the last I-cell in I⁰ jumps down.

The set J₀ and the constant T_*

Let W₀ = 1 + (w_L(0) − 1)e^{−δ_I/τ₂}

$$J_0=(0,W_0)\text{ and }{T}_{\ast }=T_{\ast }^I+\mathrm{\Delta }.$$ (15)

It follows from (13) that

$$T_I+\mathrm{\Delta }\le T_{\ast }\le T_I+\mathrm{\Delta }+{\delta }_I.$$ (16)

Note that T_* corresponds to time-duration of an episode in the discrete model. Hence, (16), together with (14) and (8), gives a precise estimate on the lengths of these episodes.

When do E-cells jump down?

Consider those E-cells, E_i, with P_i(0) = 0. Recall that these E-cells are initially active. Here we estimate when these E-cells jump down. Suppose that the maximum number of I-cells that an E-cell receives input from is M. Then when an E-cell jumps down, it does so with w_R(M) ≤ w ≤ w_R(0). Moreover, when η = 0, each w_i ∈ [0,W₀] and, while in the active phase, the slow variables w_i satisfy (12). It follows that the E-cells jump down at some time T_ejd that satisfies

$$T_E\le T_{\mathit{ejd}}\le T_E+{\delta }_E.$$ (17)

where

$$T_E={\tau }_{2}ln\frac{1-W_0}{1-w_R(M)}\text{ and }{\delta }_E={\tau }_{2}ln\frac{1-w_R(M)}{(1-w_R(0))(1-w_L(0))}$$ (18)

We will need to assume that E-cells that jump up remain active until time T_* when all I-cells are released from inhibition. This will be the case if the following inequality holds.

$${\delta }_I<T_E.$$ (19)

We also need to assume that all E-cells that are active in a given episode jump down before the end of the episode, which translates into

$$T_E+{\delta }_E<T_I+\mathrm{\Delta }-{\delta }_I.$$ (20)

Later, we find conditions on parameters for when this is the case.

The sets J_k

We now define the sets J_kfork > 0. To do this, we need to estimate the positions of the E-cells at time T_*. Let ${\mathcal{E}}_k=\{E_i:P_i(0)=k\}$.

First consider a cell E_i = (v_i, w_i) that initially lies in ${\mathcal{E}}_0$. Note that E_i jumps down at some T_ejd and then lies in the silent phase for all times η with T_ejd < η < T_*. While in the silent phase, w_i satisfies (11). Hence,

$$w_i(T_{\ast })=w_i(T_{\mathit{ejd}})e^{-(T_{\ast }-T_{\mathit{ejd}})/{\tau }_1}.$$

The position at which E_i jumps down satisfies w_R(M) ≤ w_i(T_ejd) ≤ w_R(0). Moreover, T_ejd satisfies (17) and T_* satisfies (16). Therefore,

$$W_1^1\le w_i(T_{\ast })\le W_1^2$$

where $W_1^1=w_R(M)e^{-K_1/{\tau }_1},W_1^2=w_R(0)e^{-K_2/{\tau }_2}$,

$$K_1=T_I-T_E+\mathrm{\Delta }+{\delta }_I\hspace{0.17em}\text{and}\hspace{0.17em}{K}_2=T_I-T_E+\mathrm{\Delta }-{\delta }_E.$$

Let $J_1\equiv [W_1^1,W_1^2]$. We have shown that if $E_i\in {\mathcal{E}}_0$, then w_i(T_*) ∈ J₁.

We now consider the other E-cells. For 1 ≤ k < p, let

$$W_{k+1}^1=W_k^1{e}^{-(T_I+\mathrm{\Delta }+{\delta }_I)/{\tau }_1}\hspace{0.17em}\text{and}\hspace{0.17em}{W}_{k+1}^2=W_k^2{e}^{-(T_I+\mathrm{\Delta })/{\tau }_1}.$$

Similarly, let

$$W_0^1=W_p^1{e}^{-(T_I+\mathrm{\Delta }+{\delta }_I)/{\tau }_1}\hspace{0.17em}\text{and}\hspace{0.17em}{W}_0^2=W_p^2{e}^{-(T_I+\mathrm{\Delta }-{\delta }_I)/{\tau }_1}.$$

If k ≠ p, let $J_k=(W_k^1,W_k^2)$. If k = p, let $J_p=(w_L(0),W_p^2)$. Here, we assume that

$$W_0^2<w_L(0<W_p^1<w_A<W_p^2.$$ (21)

(Recall that the fixed point along ${\mathcal{C}}_0$ lies at p_A = (v_A, w_A).) This assumption will be verified later (see inequality (23) or Step 4 of Section 6.7). We also assume that as long as $w_i>W_p^2$, (v_i,w_i) lies in the region where w_i satisfies (11); that is, v_i < V₁. The latter will be the case if V₁ is sufficiently close to V₂, as we assumed in Section 6.4.

Suppose that $E_i\in {\mathcal{E}}_k$, 1 ≤ k < p, and w_i(0) ∈ J_k. It follows from (21) that E_i lies in the silent phase for 0 ≤ η ≤ T_*. Using (16) and the definition of J_k+1, we conclude that w_i(T_*) ∈ J_k+1.

It remains to consider those E-cells in ${\mathcal{E}}_p$. Suppose that w_i(0) ∈ J_p. There are two cases to consider. First suppose that E_i does not receive inhibitory input from an I-cell in I⁰. Then, using (21), (v_i, w_i) approaches the fixed point p_A and w_i(η) remains in J_p.

Now suppose that E_i does receive inhibitory input from an I-cell in I⁰. We need to show that E_i jumps up at some time η < T_* and then lies in J₀ when η = T_* To do this, we estimate the time at which E_i is released from inhibition. Recall that the I-cells in I⁰ jump down at some time T_ijd that satisfies (13). Moreover the last I-cell in I⁰ jumps down at $T_{\ast }^I$. Hence, E-cells are released from inhibition at some time T_eju that satisfies

$$T_I+\mathrm{\Delta }\le T_{\mathit{eju}}\le T_I^{\ast }+\mathrm{\Delta }\equiv T_{\ast }$$ (22)

From the definitions, $w_i(T_I+\mathrm{\Delta })\le W_0^2$. It then follows from (21) that E_i jumps up at some η ∈ [T_I + Δ, T_*].

Finally, we need to show that w_i(T_*) ∈ J₀. Note that when E_i jumps up, w_i(T_eju) < w_L(0). Moreover, w_i(η) satisfies (12) for T_eju ≤ η T_*. It follows from (22) and (16) that T_* − T_eju < δ_I. Hence,

$$\begin{array}{l}{w}_i(T_{\ast })=1+(w_i(T_{\mathit{eju}})-1)e^{-(T_{\ast }-T_{\mathit{eju}})/{\tau }_2}\\ <1+(w_L(0)-1)e^{-{\delta }_I/{\tau }_2}\equiv W_0.\end{array}$$

Clearly, w_i(T_*) > 0. It now follows that w_i(T_*) ∈ J₀.

Property (B2)

As remarked above, T_* − T_eju < δ_I, thus all E-cells that are induced to jump up between times 0 and T_* do so in the interval (T_* − δ_I, T_*]. By (19), these cells will still be in the active phase at time T_*, and Property (B2) follows.

Where are the I-cells when η = T_*?

For the proof of (B4), recall that the last I-cell jumps down when $\eta =T_I^{\ast }$. Hence, all of the I-cells are released from inhibition when $\eta =T_I^{\ast }+\mathrm{\Delta }$. From the assumption of Section 6.4 it follows that since all I-cells still receive inhibitory input between times $T_I^{\ast }$ and $T_I^{\ast }+\mathrm{\Delta }$, the I-cells cannot fire in this interval, and must remain in the silent phase, for $\eta \le T_I^{\ast }+\mathrm{\Delta }=T_{\ast }$. In order to guarantee that when η = T_*, all the I-cells lie below the left knee of ${\mathcal{C}}^I(1,0)$, we assume that the I-cells have fast refractory periods. In particular, the time it takes for the I-cells to evolve in the silent phase from their jump-down positions to below the left knee of ${\mathcal{C}}^I(1,0)$ is less than Δ. This will be the case if ${\tau }_1^I$ is sufficiently small.

Properties (B5) and (B6)

Property (B5) follows from our choice of $T_I^{\ast }$ as the time when the last I-cell jumps down and the assumption that α_x ≫ β_x. The latter implies that the x_i for the last I-cell to jump down can be assumed to cross the threshold at time $T_{\ast }=T_I^{\ast }+\mathrm{\Delta }$.

Property (B6) follows from the fact that the next E-cell to jump up after time η = 0 can do so only after the first I-cell that jumped up at time 0 has released its inhibition.

6.6 Choosing parameters

Here we demonstrate how to estimate the various constants needed in the analysis in terms of parameters in the model. These estimates will demonstrate how one needs to choose the positions of the left and right knees of the cubic-shaped nullclines, the time-constants τ₁, τ₂, ${\tau }_1^I$ and ${\tau }_2^I$, and the delay Δ defined in (8).

First consider T_E and δ_E, which are given in (18). We must choose parameters so that δ_I < T_E < T_E + δ_E < T_I + Δ − δ_I and δ_I < Δ. We note that in many neuronal models, the right knees depend weakly on synaptic coupling. An example is given in the next section. If this is the case, then

$$w_R(0)\approx w_R(M)\text{ and }{w}_R^I(\mathit{min})\approx w_R^I(\mathit{max}).$$

If we further assume that w_L(0) and $w_L^I(1,0)$ are both close to 0, it follows that both δ_E and δ_I can be made to be as small as we please. In particular, δ_I < Δ. We can guarantee that T_E + δ_E < T_I by choosing the time constants τ₂ and ${\tau }_2^I$ appropriately.

We next compute the $W_k^1$ and $W_k^2$. After some calculation, we find that

$$\begin{array}{l}{W}_p^1=w_R(0)e^{-[p(T_I+\mathrm{\Delta }+{\delta }_I)-T_E]/{\tau }_1}\\ W_p^2=w_R(M)e^{-[p(T_I+\mathrm{\Delta })-{\delta }_E-T_E]/{\tau }_1}\\ W_0^2=w_R(M)e^{-[(p+1)(T_I+\mathrm{\Delta })-{\delta }_I-{\delta }_E-T_E]/{\tau }_1}.\end{array}$$

We need to choose parameters so that

$$W_0^2<w_L(0)W_p^1.$$ (23)

To get some idea when this is the case, we will consider, as above, the limiting case in which δ_E = δ_I = 0. In particular, all of the E-cells jump down at the same position, which we denote as w_R, and all of the I-cells jump down at the same position, which we denote as $w_R^I$. We further let w_L(0) = w_L. Then (23) becomes

$$w_R{e}^{-[(p+1)(T_I+\mathrm{\Delta })-T_E]/{\tau }_1}<w_L<w_R{e}^{-[p(T_I+\mathrm{\Delta })-T_E]/{\tau }_1}.$$

This is satisfied if

$$\frac{w_L}{w_R}{e}^{-T_E/{\tau }_1}<e^{-p(T_I+\mathrm{\Delta })}<\frac{w_L}{w_R}{e}^{(T_I+\mathrm{\Delta }-T_E)/{\tau }_1}$$

or

$$\frac{{\tau }_1}{T_I+\mathrm{\Delta }}ln\frac{w_R}{w_L}-\frac{T_I+\mathrm{\Delta }-T_E}{T_I+\mathrm{\Delta }}<p<\frac{{\tau }_1}{T_I+\mathrm{\Delta }}ln\frac{w_R}{w_L}+\frac{T_E}{T_I+\mathrm{\Delta }}.$$ (24)

Note that τ₁ln(w_R/w_L) is the time it takes for a solution of (11) starting at w_R to reach w_L. Hence, one can interpret (24) as saying that p is roughly the ratio of the times that an E-cell spends in the silent phase and an I-cell spends in the active phase.

We have shown that in the limiting case when all right knees coincide and thus δ_E = δ_I = 0, we can have δ_I < T_E < T_E + δ_E < T_I < T_I + Δ − δ_I and δ_I < Δ. Since these numbers depend continuously on the positions of the knees, the inequalities will continue to hold if w_R(0) ≈ w_R(M) and $w_R^I(\mathit{min})\approx w_R^I(\mathit{max})$.

Positions of the right knees

We have assumed that the positions of the right knees depend weakly on the synaptic inputs. This is often the case in neuronal models and here we demonstrate why. Consider an E-cell, (v,w). Recall that along a cubic-nullcline,

$$f(v,w)-g_{\mathit{syn}}S(v-v_{\mathit{syn}})=0$$ (25)

where S now represents the total synaptic input. If we let w = W(v, S) denote the cubic-nullcline, plug this into (25) and differentiate with respect to v, then we find that at (v, w) = (v, w(v, S)),

$$f_v+f_w\frac{\partial W}{\partial v}-g_{\mathit{syn}}S=0$$

At the right knee, $\frac{\partial W}{\partial v}=0$and, therefore,

$$f_v=g_{\mathit{syn}}S.$$ (26)

We now write the position of the right knee as (v_R(S), w_R(S)). Plugging this into (25) and differentiating with respect to S, we find that

$$f_v{v}_R^{\prime }(S)+f_w{w}_R^{\prime }(S)-g_{\mathit{syn}}(v_R(S)-v_{\mathit{syn}})-g_{\mathit{syn}}S{v}_R^{\prime }(S)=0.$$

Together with (26), this implies that

$$w_R^{\prime }(S)=\frac{g_{\mathit{syn}}(v_R(S)-v_{\mathit{syn}})}{f_w(v_R,w_R))}.$$

We need to choose parameters so that $w_R^{\prime }(S)$ is small. To do this, we need to estimate f_w(v_R, w_R). To get a sense of how large f_w is, we now consider the concrete example presented in the next section. For that example,

$$f_w=g_{Na}{w}_{\infty }^3(v-v_{Na})-4g_K{w}^3(v-v_K).$$

It follows that we can make $w_R^{\prime }(S)$ as small as we please by choosing g_syn sufficiently small and choosing either g_Na or g_K, or both, sufficiently large. For the example presented in the next section, we find that at the right knees, w ≈ .5, v ≈ 0, and w_∞(v) ≈ 1. It then follows that $w_R^{\prime }(S)\approx .005$.

6.7 An alternative way of choosing parameters

Here we describe an alternative way of choosing suitable parameters. Rather than having nullclines with all right knees close together, we will only require that their positions are bounded away from 0 and 1. Specifically, we assume that there are positive numbers 0 < a < c and 0 < b so that for every ρ > 0 we can choose the nullclines for the E-cells in such a way that a < w_R(0) < c, $\frac{w_R(0)}{w_R(M)}\le b$ and 0 < w_L(0) < ρ. Let $d=\frac{a}{b}$. Note that our assumptions imply that d < w_R(M) < c.

On the other hand, we may need very small values for the parameter β_x, which is consistent with the rest of our argument, but places a restriction of a very slow release from inhibition on E-I-networks.

The construction focuses on the choice of five parameters: τ₁, τ₂, ${\tau }_1^I$, ${\tau }_2^I$, Δ. The parameter Δ does not depend on any of the other four parameters; according to equation (8), we can make Δ any positive number we want if we choose a suitably small β_x for a given threshold θ_x. Moreover, the parameters τ₁, τ₂, ${\tau }_1^I$, ${\tau }_2^I$ do not influence the shapes or relative positions of the nullclines and their knees.

We choose parameters in the following sequence of steps:

Step 1

Choose parameters so that the v- and w-nullclines for the I-cells have the desired shapes and are in the correct relative positions; in particular, so that the knees are in the required relative positions. This does not place constraints on τ₁,τ₂, ${\tau }_1^I$, ${\tau }_2^I$ Δ.

Step 2

We will initially require that the nullclines for the E-cells satisfy our assumptions regarding the bounds a, b, c and are such that $w_L(0)<\frac{d}{4}$. Let τ₂ = 1, and choose ${\tau }_2^I$ small enough so that $\frac{{\delta }_I}{{\tau }_2}={\delta }_I<ln\frac{1-d/4}{1-d/2}.$. In particular, δ_I < −ln(1 − d/2). In view of the definition of W₀, the latter implies that 1−W₀ > 1−d/2. In view of (18) and the bounds on the knees this in turn implies that

$$ln(1-d/2)-ln(1-d)<T_E<-ln(1-c)$$ (27)

and

$${\delta }_E<-ln(1-c)-ln(1-d/4).$$ (28)

Note that $\frac{1-d/4}{1-d/2}<\frac{1-d/2}{1-d}$. Thus ${\delta }_I<ln\frac{1-d/2}{1-d}$, and in view of the first inequality in (27), this will ensure that δ_I < T_E. After this step, T_I and δ_I are fixed.

Step 3

The refractory period p was fixed at the outset. Now fix τ₁ sufficiently small so that $\frac{{\delta }_I}{{\tau }_1}>-lnb$. Eventually we will have −ln b > lnw_R(0) − lnw_R(M). This will ensure that $W_{k+1}^2<W_k^1$ for all k. Now choose a preliminary value of Δ that is sufficiently large so as to ensure that $W_{k+1}^2<W_k^1$ for all k < p and $W_0^2<W_p^1$. Here Δ > 4p(T_I + T_E + δ_I + δ_E) will do for a quick and dirty estimate, where T_E and δ_E can be replaced by the upper bounds in (27) and (28). Finally, let ρ be sufficiently small so that $\rho \le W_p^1=w_R(0)e^{-[p(T_I+\mathrm{\Delta }+{\delta }_I)-T_E]/{\tau }_1}$ and $\rho \le \frac{d}{4}$ after this initial choice of Δ. In order to assure the inequality $\rho \le W_p^1$, we can replace w_R(0) by its lower bound a and T_E by its upper bound −ln(1 − c).

Step 4

Choose the v-nullclines for the E-cells in such a way that $\frac{w_R(0)}{w_R(M)}<b$, a < w_R(0) < c, and 0 < w_L(0) < Δ. This assures that $w_L(0)<W_p^1$ and the endpoints $W_k^j$ of our intervals are in the correct relative position. If $W_0^2<w_L(0)$ after this initial choice of Δ we are done; if not, increase the Δ a little bit more (which will move both $W_0^2$ and $W_p^1$ down without changing the strict order relation between the interval endpoints) until we have $W_0^2<w_I(0)<W_p^1$. Choose the w-nullclines for the E-cells in such a way that $W_p^1<w_A<W_p^2$ and V₂ − V₁ is sufficiently small.

Step 5

Fix ${\tau }_1^I$ “sufficiently small” as required in the last line of the proof of (B4) in Section 6.5.

6.8 Completion of the proof of the Theorem

We have so far shown that every orbit of the discrete model is realized by a stable solution of the differential equations model. In particular, the continuous dynamics realizes the discrete dynamics on trajectories that start in some open subset of the state space of the ODE model. This open subset is characterized by the conditions (A1)–(A6) given in Section 6.2. It is still not clear how solutions that start outside of this subset behave. We now prove that every solution of the differential equations model eventually realizes a periodic orbit of the discrete model. This then will complete the proof of the Theorem.

We start from any initial state at time η = 0 and define time η₁ as follows: If no I-cell is active at time 0, let η₁ = 0. If some I-cells are active at time 0, then all I-cells receive inhibition, and no additional I-cells can jump up until the last currently active I-cell jumps down. Let η₁ be the time when the last I-cell that was active at time 0 jumps down. Now let η₂ ≥ η₁ be the earliest time when the last I-cell releases its inhibition. If at that time no E-cells are active or jump up exactly at η₂, then no I-cells will jump up at time η₂, and no E-cells will jump up subsequently to η₂. In this case, there will be no subsequent firing, and the system will reach a state with all E-cells in the interval J_p, which corresponds to the steady state of the discrete system.

If at time η₂ some E-cells are active or jump up, then those I-cells that are ready to fire and receive excitation will jump up at time η₂. Let η₂ < η₃ <… < η_2+p be the subsequent times when all I-cells are released from inhibition. If at any of these times no E-cells are active, then we are back to the situation described in the previous paragraph. Notice that for every i = 1, …, p we must have

$$T_I+\mathrm{\Delta }<{\eta }_{2+i}-{\eta }_{2+i-1}<T_I+\mathrm{\Delta }+{\delta }_I.$$

This implies that at time η_2+p all I-cells will have moved to the region where they are ready to fire if they are released from inhibition and receive excitation.

Now consider the E-cells at time η_2+p. We distinguish two cases. If an E-cell has never been active between times [η₂, η_2+p] it will have had plenty of time to move to the interval J_p. The same is true for the E-cells that were active at time η₂ and did not fire again before η_2+p. If an E-cell did jump up at any time in the interval (η₂, η_2+p], it must have received inhibition from an I-cell that was active in this interval and whould have jumped up at some time in an interval (η_2+i − δ_I, η_2+i] for some i = 1, … p. Now our previous analysis applies to such cells and shows that at time η_2+p this E-cell must be in the interval J_p−i. We have now shown that (A1) – (A6) given in Section 6.2 are satisfied at time η_2+p. We can now use the analysis given in the previous sections to conclude that the continuous model realizes the discrete model for η > η_2+p.

7 A concrete example

Here we give a concrete example of a neuronal model. Even though the connectivity of this network is much weaker than the connectivity assumed in proving the results of Section 6, numerical experiments indicate that this network still reliably generates firing patterns consistent with the predictionsof the discrete dynamics. The model consists of populations of both excitatory and inhibitory cells. The equations for each can be written as:

$$\begin{array}{l}\frac{dv}{dt}=-g_L(v-v_L)-g_{Na}{w}_{\infty }^3(v)(.5-w)(v-v_{Na})-g_K{w}^4(v-v_K)+I\\ \frac{dw}{dt}=\epsilon (w_{\infty }(v)-w)/\tau (v)\end{array}$$ (29)

where g_L = 2.25, g_Na = 37.5, g_K = 45, v_L = −60, v_Na = 55 and v_K = −80 represent the maximal conductances and reversal potentials of a leak, sodium and potassium current, respectively. Moreover, ε = .04, I = 0, m_∞(v) = 1/(1 + exp(− (v + 30)/15)) and w_∞(v) = 1/(1 + exp(− (v + 45)/3)). The function τ(v) can be written as τ(v) = τ₁ + τ₂/(1 + exp(v/.1)) where τ₁ = 4 and τ₂ = 3 for each E-cell and τ₁ = 4.5 and τ₂ = 3.5 for each I-cell. The synaptic connections are modeled as in (3); however, in this model, we do not have the I-cells send inhibition to each other. We also assume that both the excitatory and inhibitory synapses are indirect. Here we take g_syn = .15, $g_{\mathit{syn}}^I=.2$, v_syn = 0 and $v_{\mathit{syn}}^I=-100$. Moreover, α_x = 1.2 and β_x = 4.8 for both the inhibitory and excitatory indirect synapses. To simplify the equations, we assume that the synaptic variables s_i and $s_i^I$ turn on and off instantaneously; that is, each s_i satisfies $s_i^{\prime }=H(v_i-\theta )$ where H is the Heaviside step-function and θ = .1 is some threshold. A similar equation holds for each $s_i^I$.

Figure 10 shows an example in which there are 100 excitatory and 100 inhibitory cells. Each E-cell has a refractory period of one and sends excitation to one I-cell and each I-cell sends inhibition to nine E-cells, chosen at random. We show solutions of both the neuronal model and the corresponding discrete model. Note that the cells that fire during each episode are exactly the same. After a transient of 2 cycles, there appears to be a stable attractor of 19 episodes.

Figure 10.

Solutions of the discrete model (left) and neuronal model (right) for a network of 100 excitatory and 100 inhibitory cells. Each E-cell effectively inhibits nine other E-cells chosen at random. In this example, the refractory period is one and the length of the attractor is nineteen. The top panels show the full network and the bottom panel shows a blow-up of the rectangular regions highlighted in the top panel. In the right panels, the bright areas indicate when a neuron fires; the dark areas indicate when a neuron receives inhibition.

8 Discussion

In this paper, we have demonstrated that it is possible to reduce a large class of excitatory-inhibitory networks to a discrete model. In a companion paper [1], we analyze the discrete dynamics and characterize when these networks exhibit a large number of stable firing patterns. Analysis of the discrete model also allows us to determine how the structure of firing patterns depend on the refractory period, the firing threshold, and the underlying network architecture.

The class of E-I-networks considered in this paper arises in many important applications. These include models for thalamocortical sleep rhythms [2–5], models for synchronous activity in the basal ganglia [6,7] and models for oscillations seen in a mammal’s olfactory bulb or an insect’s antennal lobe (AL) [8,9]. Each of these systems may exhibit rhythmic activity in which some subpopulation of cells fires in synchrony. Experimental recordings from neurons within the AL demonstrate that the network may exhibit dynamic clustering in which different subpopulations of cells take turn firing; moreover, the membership of these subpopulations may change from one episode to another. As the examples in Section 3.3 and Section 7 demonstrate, this sort of firing pattern arises naturally in the neuronal networks considered here. The analysis of the discrete model given in [1] examines how the tendency of the network to exhibit dynamic clustering depends on network architecture.

In order to reduce the neuronal model to the discrete model, we needed to make a number of assumptions on the network. In particular, we assumed that the inhibitory connections are indirect and the inhibitory cells have a very short refractory period. These assumptions are motivated by previous models for excitatory-inhibitory networks in the thalamus responsible for the generation of sleep rhythms [3]. In those models, the indirect synapses correspond to slow GABA_B inhibition and the inhibitory cells do indeed have a very short refractory period. In fact, the concrete model presented in Section 7 is based on models for sleep rhythms generated by excitatory-inhibitory networks within the thalamus [3,4].

The analysis in Section 6 gives precise estimates on the parameter values so that the differential equations model reproduces the discrete dynamics. One may expect that it becomes increasingly more difficult to satisfy these estimates as the refractory period, p, or the size of the network, n, increases. This is true for p; however, the results are robust for a large class of arbitrarily large networks. For larger values of p, we must keep an increasing number of clusters separated along the left branch of the cubic-nullclines. Numerical simulations suggest that the largest value of p for which we can robustly reproduce the discrete dynamics is p = 5. Of course, the Theorem states that this is possible for any value of p. However, this result requires that the singular parameter ε be very small. If p is too large, then we may have to choose ε so small that it is impossible to generate the desired behavior.

The estimates given in Section 6 do not depend as crucially on the size of the network. Note that p is an intrinsic parameter of individual cells and does not depend on n. The only potential problem is if two cells within the same active cluster receive a significantly different amount of synaptic input from other active cells. The amount of synaptic input determines the right knee at which a cell jumps down; the estimates given in Section 6 depend on the maximum possible separation between these right knees. Therefore, as long as active cells receive approximately the same amount of synaptic input, our results do not depend on the size of the network.

Our analysis demonstrates that it may not be possible to rigorously reduce the continuous neuronal model to a discrete model in purely inhibitory networks, but this is possible in excitatory-inhibitory networks. A critical role of the additional layer of inhibitory cells in the E-I-network is to stabilize the timings of the firing of cells within each cluster. We note that the example discussed in Section 5 demonstrates that instabilities may arise during the jumping-up and jumping-down processes. That is, because cells within the same cluster may jump down at different times and at different right knees, the distances between these cells may increase. Moreover, these cells may release other cells from inhibition at different times and this may result in expansions in the distances between cells that jump up to the active phase. These types of expansions are in general also possible in E-I-networks because active cells within both the excitatory and inhibitory populations may also jump down at different right knees. It is, therefore, not obvious what advantages E-I-networks have over purely inhibitory networks. Note, however, that in our analysis of E-I-networks, all of the I-cells jump up at the same time. This follows from the assumption that the inhibitory synapses are indirect and there is all-to-all coupling between the inhibitory cells. Because all of the I-cells jump up at the same time, we were able to derive apriori bounds on when different I-cells jump down and then release E-cells from inhibition. (See inequalities (13)). This leads to an a priori bound on the possible expansion of the distances between cells during the jumping-down process and this, in turn, leads to an apriori bound on possible expansion during the jumping-up process. We remark that it is possible to weaken the assumption of all-to-all coupling among the I-cells in cases in which the underlying network architecture has some special structure; in particular, all-to-all coupling among simultaneously active I-cells is sufficient. It is likely that even this assumption can be further weakened. In the numerical simulation shown in Figure 10, for example, there is no coupling among the I-cells.

There have been numerous studies of clustering, dynamic clustering and transient synchrony in neuronal networks [12–18]. Previous work has typically considered smaller networks or larger networks with symmetries imposed on the network architecture; this includes all-to-all coupling. In these networks, dynamic clustering often emerges due to the presence of structurally stable heteroclinic cycles [10,15]. This is in contrast to the mechanism described in this paper which involves rebound properties of the excitatory cells. Our results hold for a more realistic neuronal model, we consider a completely general class of network architectures and the reduction to the discrete model leads to a complete characterization of the network behavior. We note that the clustering that may appear in the I-network considered in Section 4 appears to be unstable to perturbations of some clusters but not to others. This is similar to results presented in [15] where phase oscillator models were considered. Fast-slow analysis has been used in many studies of neuronal systems. This approach was used in [21], for example, where the analysis of the firing of a single excitable neuron, subject to stochastic input trains, was reduced to a discrete time Markov chain analysis. Finally, reproducible sequence generation in a class of excitatory-inhibitory network with random connections was studied in [16]. In that network, it was found that the highest likelihood for the existence of a stable limit cycle was close the regime of balanced excitatory-inhibitory input to each cluster; transient behavior was more likely far from the region of balanced input. The results presented in [1] demonstrate that for the network considered in the present paper, the existence of stable limit cycles or the existence of long or short transients, does not depend on a balance of excitatory and inhibitory input in the underlying E-I-network. Instead, long transients are more likely in networks with sparse connectivity, while short transients and a large number of stable limit cycles arise in networks with a dense connectivity. We note that the networks considered here and in [16] are quite different so that the different conclusions are not surprising.

Discrete models such as the one presented here have been proposed in numerous other studies of neuronal dynamics and other biological systems. If the refractory period of each neuron is one, then our model is an example of a Boolean dynamical system similar to the ones proposed as models for gene regulatory networks [22]. The relationship between network connectivity and “typical” network dynamics can be studied by investigating Random Boolean Networks (RBNs) [22], and a similar approach has been taken in [1]. We note, however, that the discrete model considered here is not random in the sense of RBNs. This is because once the architecture of the network is fixed, the dynamics of the discrete model is completely determined. The results in [1] demonstrate that the discrete model considered in this paper has properties that are in sharp contrast with those of RBNs. For example, in the study of Random Boolean Networks, a distinction is made between two types of behavior, called the ordered regime and the chaotic regime. Results on RBNs show that the network tends to become more chaotic, and hence less ordered, as the average number of inputs to the Boolean regulatory functions increases. Moreover, the attractors tend to be very few and very long in the chaotic regime. The results in [1] demonstrate that the dynamics of our discrete model becomes more ordered, as measured by the length and number of attractors, as the average degree of connectivity either decreases below or increases above a certain threshold.

Figure 8.

The sets J_k used in the analysis. Here, p = 2. Cells in J₀ evolve in the active phase until they jump down to lie in J₁. During this time, cells in J₁ move to J₂. Note that only those cells in J₂ that receive inhibition move to J₀. The rest remain in J₂.