A Mechanism for Graded, Dynamically Routable Current Propagation in Pulse-Gated Synfire Chains and Implications for Information Coding

Abstract

Neural oscillations can enhance feature recognition [1], modulate interactions between neurons [2], and improve learning and memory [3]. Numerical studies have shown that coherent spiking can give rise to windows in time during which information transfer can be enhanced in neuronal networks [4–6]. Unanswered questions are: 1) What is the transfer mechanism? And 2) how well can a transfer be executed? Here, we present a pulse-based mechanism by which a graded current amplitude may be exactly propagated from one neuronal population to another. The mechanism relies on the downstream gating of mean synaptic current amplitude from one population of neurons to another via a pulse. Because transfer is pulse-based, information may be dynamically routed through a neural circuit with fixed connectivity. We demonstrate the transfer mechanism in a realistic network of spiking neurons and show that it is robust to noise in the form of pulse timing inaccuracies, random synaptic strengths and finite size effects. We also show that the mechanism is structurally robust in that it may be implemented using biologically realistic pulses. The transfer mechanism may be used as a building block for fast, complex information processing in neural circuits. We show that the mechanism naturally leads to a framework wherein neural information coding and processing can be considered as a product of linear maps under the active control of a pulse generator. Distinct control and processing components combine to form the basis for the binding, propagation, and processing of dynamically routed information within neural pathways. Using our framework, we construct example neural circuits to 1) maintain a short-term memory, 2) compute time-windowed Fourier transforms, and 3) perform spatial rotations. We postulate that such circuits, with automatic and stereotyped control and processing of information, are the neural correlates of Crick and Koch’s zombie modes.

INTRODUCTION

Accumulating experimental evidence implicates coherent activity as an important element of cognition. Since their discovery [7], gamma band oscillations have been demonstrated to exist in hippocampus [8–10], visual cortex [2, 7, 11], auditory cortex [12], somatosensory cortex [13], parietal cortex [14–16], various nodes of the frontal cortex [15, 17, 18], amygdala and striatum [19]. Gamma oscillations sharpen orientation [1] and contrast [20] tuning in V1, and speed and direction tuning in MT [21]. Attention has been shown to enhance gamma oscillation synchronization in V4, while decreasing low-frequency synchronization [22, 23] and to increase synchronization between V4 and FEF [17], LIP and FEF [15], V1 and V4 [24], and MT and LIP [25]; Interactions between sender and receiver neurons are improved when consistent gamma-phase relationships exist between two communicating sites [2].

Theta-band oscillations have been shown to be associated with visual spatial memory [26, 27], where neurons encoding the locations of visual stimuli and an animal’s own position have been identified [26, 28]. Additionally, loss of theta gives rise to spatial memory deficits [29] and pharmacologically enhanced theta improves learning and memory [3].

These experimental investigations of coherence in and between distinct brain regions have informed the modern understanding of information coding in neural systems [30, 31]. Understanding information coding is crucial to understanding how neural circuits and systems bind sensory signals into internal mental representations of the environment, process internal representations to make decisions, and translate decisions into motor activity.

Classically, coding mechanisms have been shown to be related to neural firing rate [32], population activity [33–35], and spike timing [36]. Firing rate [32] and population codes [37–41] are two different ways for a neural system to average spike number to represent graded stimulus information, with population codes capable of faster and more accurate processing since averaging is performed across many fast responding neurons. Thus population and temporal codes are capable of making use of the sometimes millisecond accuracy [36, 42, 43] of spike timing to represent signal dynamics.

Although classical mechanisms serve as their underpinnings, new mechanisms have been proposed for short-term memory [5, 44, 45], information transfer via spike coincidence [4, 46, 47] and information gating [6, 47–50] that rely on gamma- and theta-band oscillations. For example, the Lisman-Idiart interleaved-memory (IM) model [5], and Fries’s communication-through-coherence (CTC) model [47] both make use of the fact that synchronous input can provide windows in time during which spikes may be more easily transferred through a neural circuit. Thus, neurons firing coherently can transfer their activity quickly downstream. Additionally, synchronous firing has been used in Abeles’s synfire network [4, 46, 51–54] giving rise to volleys of propagating spikes.

The precise mechanism and the extent to which the brain can make use of coherent activity to transfer information have remained unclear. Previous theoretical and experimental studies have largely focused on feedforward, synfire chains [51, 53, 55–58]. These studies have shown that it is possible to transfer volleys of action potentials stably from layer to layer, but that the waveform tends to an attractor with fixed amplitude. Therefore, in these models, although a volley can propagate, graded information, in the form of a rate amplitude cannot. Other numerical work has shown that it is possible to transfer firing rates through layered networks when background current noise is sufficient to effectively keep the network at threshold [59]. The disadvantage of this method is that there is no mechanism to control the flow of firing rate information other than increasing or decreasing background noise. Recently, it has been shown that external gating, similar, in principle, to that used in the IM model and to the gating that we introduce below, can stabilize the propagation of fixed amplitude pulses and act as an external factor to control pulse propagation [49, 50].

In the Methods section, we show that information contained in the amplitude of a synaptic current may be exactly transferred from one neuronal population to another, as long as well-timed current pulses are injected into the populations. This mechanism is distinct from the synfire chains mentioned above that can only transfer action potential volleys of fixed amplitude, in contrast to [59], by using current pulses to gate information through a circuit, it provides a neuronal-population-based means of propagating graded information through a neural circuit.

We derive our pulse-based transfer mechanism using mean-field equations for a current-based neural circuit (see circuit diagram in Fig. 1a) and demonstrate it in an integrate-and-fire neuronal network. Graded current amplitudes are transferred between upstream and downstream populations: A gating pulse excites the upstream population into the firing regime thereby generating a synaptic current in the downstream population. For didactic purposes, we first present results that rely on a square gating pulse with an ongoing inhibition keeping the downstream population silent until the feedforward synaptic current is integrated. We then show how more biologically realistic pulses with shapes filtered on synaptic time-scales may be used for transfer. We argue that our mechanism represents crucial principles underlying what it means to transfer information. We then generalize the mechanism to the case of transfer from one vector of populations to a second vector of populations and show that this naturally leads to a framework for generating linear maps under the active control of a pulse generator.

FIG. 1.

Exact current transfer - Mean-field and I&F: a) Circuit diagram for the current transfer mechanism. Excitatory pulse gating the upstream (u) population and ongoing inhibition acting on the downstream (d) population. The upstream population excites the downstream population and transfers its current. b, e) Dynamics of a single current amplitude transfer with T/τ = 1 and T/τ = 2 with τ = 4 ms. Dashed red traces represent the growth and exponential decay of three different current amplitudes, I_u(t), in the upstream population. Dashed blue traces represent excitatory firing rates of the upstream population. Solid red traces represent the integration and subsequent decay of the downstream current, I_d(t). Dashed magenta traces represent the excitatory gating pulse current to the upstream population. Magenta traces are displaced from zero for clarity. c, f) Dynamics of two N = 100 neuron populations of current-based, I&F neurons showing current amplitude transfer averaged over 20 trials. d, g) Mean current amplitudes of twelve current-based, I&F neuronal populations (N = 100 neurons/population, averaged over 20 trials) successively transferring their currents. Below each of these panels is a plot of one realization of the spike times for all neurons in each respective population. h) Dynamics of a single current amplitude transfer between two gating periods, with T/τ = 0.8 and T/τ = 1.2. i) Dynamics of two N = 100 neuron populations showing current amplitude transfer with the same gating periods as in i). j) Five graded current amplitude transfers between six populations with T/τ = 0.8, followed by a transfer to T/τ = 1.2, followed by five equally graded transfers with T/τ = 1.2.

In the Results section, we demonstrate pulse-gated transfer solutions in both mean-field and integrate-and-fire neuronal networks. We demonstrate the robustness of the mechanism to noise in pulse timing, synaptic strength and finite-size effects and show how biologically realistic pulses may be used for gating. We then go on to present three examples of circuits that make use of the framework for generating actively controlled linear maps.

In the Discussion section, we consider some of the implications of our mechanism and information coding framework, and future work.

METHODS

What are the crucial principles underlying information transfer between populations of neurons? First, a carrier of information must be identified, such as synaptic current, firing rate, spike timing, etc. Once the carrier has been identified, we must determine the type of information, i.e. is the information analog or digital? Finally, we must identify what properties the information must exhibit for us to say that information has been transferred. In the mechanism that we present below, we use synaptic current as the information carrier. Information is graded and represented in a current amplitude and thus is best considered analog. The property that identifies information transfer is that the information exhibit a discrete, time-translational symmetry. That is, the waveform representing a graded current or firing rate amplitude in a downstream neuronal population must be the same as that in an upstream population, but shifted in time.

As noted in the Introduction, mechanisms exist for propagating constant activity that have demonstrated time-translational symmetries in both strong [51] and sparsely coupled [57] regimes. Here, we address a mechanism for propagation of graded activity.

An additional consideration for biologically realistic information transfer is that it be dynamically routable. That is, that neural pathways may be switched on the millisecond time scale. This is achieved in our mechanism via pulse gating.

Circuit Model

Our neuronal network model consists of a set of j = 1, …, M populations, each with i = 1, …, N, of current-based, integrate-and-fire (I&F) point neurons. Individual neurons have membrane potentials, υ_i,j, described by

$$\frac{d{\upsilon }_{i,j}}{dt}=-g_{\mathit{\text{leak}}}({\upsilon }_{i,j}-V_{\mathit{\text{Leak}}})+I_{i,j}^{\mathit{\text{Total}}}$$ (1a)

and feedforward synaptic current

$$\tau \frac{d{I}_{i,j}^s}{dt}=-I_{i,j}^s+S\sum _i\sum _k\delta (t-t_{i,j-1}^k),$$ (1b)

with total currents

$$I_{i,j}^{\mathit{\text{Total}}}=I_{i,j}^s+I_j^{\mathit{\text{Exc}}}-I_j^{\mathit{\text{Inh}}}$$ (1c)

and V_Leak is the leakage potential. The excitatory gating pulse on neurons in population j is

$$I_j^{\mathit{\text{Exc}}}(t)=(I_0^{\mathit{\text{Exc}}}+\epsilon )(\theta (t-jT)-\theta (t-(j+1)T)),$$ (2)

where θ(t) is the Heaviside step function: θ(t) = 0, t < 0 and θ(t) = 1, t > 0. The ongoing inhibitory current is $I_j^{\mathit{\text{Inh}}}(t)=I_0^{\mathit{\text{Inh}}}$.

Here, τ is a current relaxation timescale depending on the type of neuromodulator (typical time constants are τ_AMPA ~ 3 – 11 ms or τ_NMDA ~ 60 – 150 ms). Individual spike times, $\{t_{i,j}^k\}$, with k denoting spike number, are determined by the time when the voltage υ_i,j reaches the threshold voltage, V_Thres, at which time the voltage is reset to V_Reset. We use units in which only time retains dimension (in seconds) [60]: the leakage conductance is g_Leak = 50/sec. We set V_Reset = V_Leak = 0 and normalize the membrane potential by the difference between the threshold and reset potentials, V_Thres − V_Reset = 1. For the simulations reported here, we use $I_0^{\mathit{\text{Exc}}}=180/\text{sec}$ and $I_0^{\mathit{\text{Inh}}}=150/\text{sec}$. Synaptic background activity is modeled by introducing noise in the excitatory pulse amplitude via ε, where ε ~ N(0, σ²), with σ = 1/sec. The probability that neuron i in population j synapses on neuron k in population j + 1 is P_ik = p. In our simulations, pN = 80.

This network is effectively a synfire chain with prescribed pulse input [4, 51, 53, 61, 62].

Mean-field Equations

Averaging (coarse-graining) spikes over time and over neurons in population j (see, e.g. Shelley and McLaughlin [60]) produces a mean firing rate equation given by

$$m_j=\frac{-g_{\mathit{\text{Total}}}}{\text{ln}\left(\frac{{[I_j^{\mathit{\text{Total}}}-g_{\mathit{\text{Total}}}{V}_{\mathit{\text{Thres}}}]}^{+}}{g_{\mathit{\text{Total}}}(V_{\mathit{\text{Thres}}}-V_{\mathit{\text{Reset}}})+{[I_j^{\mathit{\text{Total}}}-g_{\mathit{\text{Total}}}{V}_{\mathit{\text{Thres}}}]}^{+}}\right)},$$ (3)

where g_Total = g_Leak, and

$$I_j^{\mathit{\text{Total}}}=I_j+I_j^{\mathit{\text{Exc}}}-I_j^{\mathit{\text{Inh}}}.$$

The feedforward synaptic current, I_j+1, is described by

$$\tau \frac{d}{dt}{I}_{j+1}=-I_{j+1}+S{m}_j.$$ (4a)

The downstream population receives excitatory input, m_j, with synaptic coupling, S, from the upstream population. As in the I&F simulation, we set, V_Reset = 0, and non-dimensionalize the voltage using V_Thres − V_Reset = 1, so that

$$m_j=\frac{-g_{\mathit{\text{Leak}}}}{\text{ln}\left(\frac{{[I_j^{\mathit{\text{Total}}}-g_{\mathit{\text{Leak}}}]}^{+}}{g_{\mathit{\text{Leak}}}+{[I_j^{\mathit{\text{Total}}}-g_{\mathit{\text{Leak}}}]}^{+}}\right)}.$$ (4b)

This relation, the so-called f-I curve, can be approximated by

$$m(I)\approx {[m(I_0)+m\prime (I_0)(I-I_0)]}^{+}={[m\prime (I_0)I-(m\prime (I_0)I_0-m(I_0))]}^{+}\approx {[I-g_0]}^{+}$$ (5)

near I ≈ I₀, where m′ (I₀) ≈ 1 (here the prime denotes differentiation), and letting g₀ = m′ (I₀) I₀ − m(I₀) be the effective threshold in the linearized f-I curve.

Exact Transfer

We consider transfer between an upstream population and a downstream population, denoted by j = u and j + 1 = d.

For the downstream population, for t < 0, I_d = 0. This may be arranged as an initial condition or by picking a sufficiently large $I_0^{\mathit{\text{Inh}}}$, with

$$\tau \frac{d}{dt}{I}_d=-I_d+S{[I_u(t)-I_0^{\mathit{\text{Inh}}}-g_0]}^{+}.$$ (6)

At t = 0, the excitatory gating pulse is turned on for the upstream population for a period T, so that for 0 < t < T, the synaptic current of the downstream population obeys

$$\tau \frac{d}{dt}{I}_d=-I_d+S{[I_u(t)+I_0^{\mathit{\text{Exc}}}-I_0^{\mathit{\text{Inh}}}-g_0]}^{+}.$$ (7)

Therefore, we set the amplitude of the excitatory gating pulse to be $I_0^{\mathit{\text{Exc}}}=I_0^{\mathit{\text{Inh}}}+g_0$ to cancel the threshold. Making the ansatz I_u (t) = Ae^−t/τ, we integrate

$$\tau \frac{d}{dt}{I}_d=-I_d+S{I}_u$$

to obtain the expression

$$I_d(t)=SA\frac{t}{\tau }{e}^{-t/\tau },0<t<T.$$ (8a)

During this time, ongoing inhibition is acting on the downstream population to keep it from spiking, i.e., we have

$$m_d(t)={[I_d(t)-I_0^{\mathit{\text{Inh}}}-g_0]}^{+}=0.$$ (8b)

For T < t < 2T, the downstream population is gated by an excitatory pulse, while the upstream population is silenced by ongoing inhibition. The downstream synaptic current obeys

$$\tau \frac{d}{dt}{I}_d=-I_d$$ (9a)

with

$$I_d(T)=SA\frac{T}{\tau }{e}^{-T/\tau }.$$ (9b)

so that we have

$$I_d(t)=SA\frac{T}{\tau }{e}^{-T/\tau }{e}^{-(t-T)/\tau },T<t<2T$$ (9c)

and

$$m_d(t)={[I_d(t)+I_0^{\mathit{\text{Exc}}}-g_0]}^{+}=I_d(t).$$ (9d)

For exact transfer, we need I_d (t − T) = I_u (t), therefore we write

$$SA\frac{T}{\tau }{e}^{-T/\tau }=A.$$ (10)

So we have exact transfer with

$$S_{\mathit{\text{exact}}}=\frac{\tau }{T}{e}^{T/\tau }.$$ (11)

To recap, we have the solution, with S_exact,

$$I_d(t)=\{\begin{array}{c}A\frac{t}{\tau }{e}^{-t/\tau },0<t<T\hfill \\ A\frac{T}{\tau }{e}^{-t/\tau },T<t<\infty \hfill \end{array}$$ (12a)

and

$$m_d(t)=\{\begin{array}{cc}0,\hfill & 0<t<T\hfill \\ A\frac{T}{\tau }{e}^{-t/\tau },\hfill & T<t<2T\hfill \\ 0,\hfill & 2T<t<\infty \hfill \end{array}.$$ (12b)

A Synfire-Based Gating Mechanism

In our exact solution, gating pulses have biologically unrealistic instantaneous onset and offset. Therefore, it becomes important to understand how robust graded propagation can be for gating pulses of realistic shape, and is there a natural mechanism for their generation? To test the structural robustness of graded propagation with a known pulse-generating mechanism, we implemented an I&F neuronal network model with two sets of populations, one set had synaptic strengths such that it formed stereotypical pulses with fixed mean spiking profile and mean current waveform [51, 53]. The second set used these pulses, instead of square gating pulses, for current propagation. We call this neural circuit a Synfire-Gated Synfire Chain (SGSC).

Individual I&F neurons in the SGSC have membrane potentials described by

$$\frac{d}{dt}{\upsilon }_{i,j}^{\sigma }=-g_{\mathit{\text{leak}}}({\upsilon }_{i,j}^{\sigma }-V_{\mathit{\text{leak}}})+\sum _{\sigma \prime =1}^2{I}_{i,j}^{\sigma \sigma \prime }+I_{i,j}^{\sigma }$$ (13a)

$$\tau \frac{d}{dt}{I}_{i,j}^{\sigma \sigma \prime }=-I_{i,j}^{\sigma \sigma \prime }+\frac{S^{\sigma \sigma \prime }}{p_{\sigma \sigma \prime }{N}_{\sigma \prime }}\sum _{i\prime }\sum _k\delta (t-t_{i\prime ,j-1}^{\sigma \prime ,k})$$ (13b)

$$\tau \frac{d}{dt}{I}_{i,j}^{\sigma }=-I_{i,j}^{\sigma }+f^{\sigma }\sum _k\delta (t-s_{i,j}^k)$$ (13c)

where i = 1, …, N_σ, j = 1, …, M and σ, σ′ = 1, 2 with 1 for the graded chain and 2 for the gating chain; individual spike times, $\{t_{i,j}^{\sigma ,k}\}$, with k denoting spike number, are determined by the time when ${\upsilon }_{i,j}^{\sigma }$ reaches V_Thres. The gating chain receives a noise current, $I_{ij}^2$, generated from Poisson spike times, $\{s_{i,j}^k\}$, with strength f² = 0.05 and rate ν₂ = 400 Hz, i.e. a noise current averaging 20/sec that is subthreshold (given by g_leak = 50/sec). The current $I_{i,j}^{\sigma \sigma \prime }$ is the synaptic current of the σ population produced by spikes of the σ′ population. In the simulations reported in Results, τ = 5 msec and the synaptic coupling strengths are {S¹¹, S¹², S²¹, S²²} = {2.28, 0.37, 0, 2.72}. The probabilities that a neuron in population σ′ synapses on a neuron in population σ are given by {p₁₁, p₁₂, p₂₁, p₂₂} = {0.02, 0.01, 0, 0.8}. The two chains have population size {N₁, N₂} = {1000, 100}. There was a synaptic delay of 4 ms between each successive layer in the gating chain.

Information Processing Using Graded Transfer Mechanisms

Because for our mechanism current amplitude transfer is in the linear regime, downstream computations may be considered as linear maps (matrix operations) on a vector of neuronal population amplitudes. For instance, consider an upstream vector of neuronal populations with currents, I^u, connected via a connectivity matrix K to a downstream vector of neuronal populations, I^d:

$${\mathbf{I}}^u(t)\stackrel{K}{\to }{\mathbf{I}}^d(t).$$ (14)

With feedforward connectivity, given by the matrix K, the current amplitude, I^d, from the mean-field model obeys

$$\tau \frac{d}{dt}{\mathbf{I}}^d=-{\mathbf{I}}^d+S{[\sum _{k}K{\mathbf{I}}^u+{\mathbf{p}}^j(t)]}^{+},$$ (15)

where p^u(t) denotes a vector gating pulse on layer j. This results in the solution I^d(t − T) = PKI^u(t), where P is a diagonal matrix with the gating pulse vector, p, of 0s and 1s on the diagonal indicating which neurons were pulsed during the transfer.

For instance, if the matrix of synaptic weights, K, were square and orthogonal, the transformation would represent an orthogonal change of basis in the vector space ℝⁿ, where n is the number of populations in the vector. Convergent and divergent connectivities would be represented by non-square matrices.

This type of information processing is distinct from concatenated linear maps in the sense that information may be dynamically routed via suitable gating. Thus, we can envision information manipulation by sets of non-abelian operators, i.e., with non-commuting matrix generators, that may be flexibly coupled. We can also envision re-entrant circuits or introducing pulse-gated nonlinearities into our circuit to implement regulated feedback.

Information Coding Framework

Our discussion has identified three components of a unified framework for information coding:

1. information content - graded current, I

2. information processing - synaptic weights, K

3. information control - pulses, p

Note that the pulsing control, p, serves as a gating mechanism for routing neural information into (or out of) a processing circuit. We, therefore, refer to amplitude packets, I, that are guided through a neural circuit by a set of stereotyped pulses as “bound” information.

Consider one population coupled to multiple downstream populations. Separate downstream processing circuits may be multiplexed by pulsing one of the set of downstream circuits. Similarly, copying circuit output to two (or more) distinct downstream populations may be performed by pulsing two populations that are identically coupled to one upstream population.

In order to make decisions, non-linear logic circuits would be required. Many of these are available in the literature [61, 63]. Simple logic gates should be straight-forward to construct within our framework by allowing interaction between information control and content circuits. For instance, to construct an AND gate, use gating pulses to feed two sub-threshold outputs into a third population, if the inputs are (0, 0), (0, 1) or (1, 0), none of the combined pulses exceed threshold and no output is produced. However, the input (1, 1) would give rise to an output pulse. Other logic gates, including the NOT may be constructed, giving a Turing complete set of logic gates. Thus, these logic elements could be used for plastic control of functional connectivity, i.e. the potential for rapidly turning circuit elements on or off, enabling information to be dynamically processed.

RESULTS

Exact Transfer

In Fig. 1, we demonstrate our current amplitude transfer mechanism in both mean-field and spiking models. The neural circuit for one upstream and one downstream layer is shown in Fig. 1a. Fig. 1b and e show the exact, mean-field transfer solution for T = τ = 4 ms and T = 2τ = 8 ms. Fig. 1c and f show corresponding transfer between populations of N = 100 current-based, I&F neurons. Fig. 1d and g show mean currents computed from simulations of I&F networks with N = 100. Mean amplitude transfer for these populations is very nearly identical to the exact solution and, as may be seen, graded amplitudes are transferred across many synapses and are still very accurately preserved. Fig. 1h shows the exact mean-field transfer solution between populations gated for T/τ = 0.8 and T/τ = 1.2 with τ = 5 ms. Fig. 1i shows the corresponding transfer between populations of N = 100 I&F neurons. Fig. 1j shows how integration period, T, may be changed within a sequence of successive transfers within an I&F network with a value of S_exact that supports two different timescales.

This mechanism has a number of features that are represented in the analytic solution, Eqns. (11)–(12), and in Fig. 1: 1) Exact transfer is possible for any T and τ. This means that transfer may be enacted on a wide range of time scales. This range is set roughly by the value of S_exact. Roughly, 0.1 < T/τ < 4 gives S small enough that firing rates are not excessive in the corresponding I&F simulations. 2) τ sets the “reoccupation time” of the upstream population. After one population has transferred its amplitude to another, the current amplitude must fall sufficiently close to zero for a subsequent exact transfer. Therefore, synapses mediated by AMPA (NMDA) may allow repeated exact transfers. 3) Pulse-gating controls information flow, not information content. As an example, one upstream population may be synaptically connected to two (or more) downstream populations. A graded input current amplitude may then be selectively transferred downstream depending on whether one, the other, or both downstream populations are pulsed appropriately. This allows the functional connectivity of neural circuits to be plastic and rapidly controllable by pulse generators. 4) S_exact has an absolute minimum at T/τ = 1, and, except at the minimum, there are always two values of T/τ that give the same value of S. This means, for instance, that an amplitude transferred via a short pulse may subsequently be transferred by a long pulse and vice versa (see Fig. 1h,i,j). Thus, not only may downstream information be multiplexed using pulse-based control, but the time scale of the mechanism may also be varied from transfer to transfer.

The means by which the mechanism can fail are also readily apparent: 1) The gating pulses might not be accurately timed. 2) Synaptic strengths might not be correct for exact transfer. 3) The amplitude of the excitatory pulse $I_0^{\mathit{\text{Exc}}}$ might not precisely cancel the effective threshold $I_0^{\mathit{\text{Inh}}}+g_0$. 4) The mean-field approximation might break down due to too few neurons in the neuronal populations.

Robustness to Variability in Pulse Timing, Synaptic Strength and Finite Size Effects

In Fig. 2, we investigate mean current variability for the transfer mechanism in the spiking model due to the modes of failure discussed above for T/τ = 1 with τ = 4 ms. Fig. 2a shows the distribution of mean current amplitudes averaged over populations of N = 1000 neurons, calculated from 1000 realizations. Fig. 2b shows the distribution with just N = 100. Clearly, more neurons per population gives less variability in the distribution. The signal-to-noise ratio (SNR) decreases as the square-root of the number of neurons per population, as would be expected. Thus, for circuits needing high accuracy, neuronal recruitment would increase the SNR. Fig. 2c shows the distribution for N = 100 with 10% jitter in pulse start and end times. Fig. 2d shows the distribution for N = 100 with 2% jitter in synaptic coupling, S. Note that near T/τ = 1, S_exact varies slowly, thus the effect of both timing and synaptic coupling jitter on the stability of the transfer is minimal. Pulse timing, synaptic strengths, synaptic recruitment, and pulse amplitudes are regulated by neural systems. So mechanisms are already known that could allow networks to be optimized for graded current amplitude transfer.

FIG. 2.

Sources of variability in the current transfer mechanism: Distributions of mean current amplitudes from current-based, I&F neuronal simulations. a) N = 1000. b) N = 100. c) N = 100 with the start and end times of each pulse jittered uniformly by 10% of the pulse width. d) N = 100 with synaptic coupling, S, jittered uniformly by 2%. Gray scale: White denotes 0 probability, Black denotes probability maximum. All distributions sum to unity along the vertical axis. Red curves in all panels denote the mean.

A Synfire-Gated Synfire Chain

We examine our pulse-gating mechanism in a biologically realistic circuit. Instead of using square gating pulses with unrealistic on- and offset times, we show that we can use the synaptic current generated by a well-studied synfire chain model [51] to gate the synaptic current transfer. In the graded transfer chain, the downstream (d) population receives excitatory synaptic currents from both the upstream (u) population current and from the corresponding synfire (s) population (see Fig. 3a). Note that the synaptic currents generated by the synfire chain play the role of the gating pulse, allowing the upstream population to transfer its current to the downstream population in a graded fashion. In Fig. 3b, we show that our circuit can indeed transfer a stereotypical synaptic current pulse in a graded fashion (3 current amplitudes shown) in an I&F simulation with M = 12 layers. By itself, the gating synfire population is dynamically an attractor, with firing rates of fixed waveform in each layer [51], producing a stereotypical gating current that is repeated across all layers (Fig. 3d). Fig. 3c shows spike times in the populations transferring graded currents and Fig. 3e shows spike times in the gating populations.

FIG. 3.

A Synfire-Gate Synfire Chain (SGSC). a) Neural circuit for an SGSC. b) Mean synaptic current amplitudes (N = 1000 neurons) in a graded synfire chain receiving gating pulses from a synfire chain with coupling set such that it generates constant amplitude pulses (gating synfire chain). c) Action potentials from one realization of the graded synfire chain. d) Mean synaptic current amplitude (N = 100 neurons) from spikes of a gating chain. e) Action potentials from one realization of the gating chain. The mean currents are averaged over 50 realizations.

These results (exact transfer in an analytically tractable mean-field model, the corresponding I&F neuronal network simulations, and the more biologically realistic SGSC model) demonstrate the structural robustness of our graded transfer mechanism. In these cases, a key essential theoretical mechanism was the time window of integration provided by a gating synaptic current (either put in by hand or generated intrinsically by a subpopulation of the neuronal circuit in the SGSC case). We note that, in neural circuits in vivo, time windows provided by gating pulses can be set and controlled by many mechanisms, for instance, time-scales of excitatory and inhibitory postsynaptic currents, absolute and relative refractoriness of individual neurons, time-scales in a high-conductance network state [64, 65] and coherence of the network dynamics. Indeed, different parts of the brain may use different combinations of neuronal and network mechanisms to implement graded current transfer.

A High-Fidelity Memory Circuit

As a first complete example of how graded information may be processed in circuits using pulse-gating, we demonstrate a memory circuit using the mean-field model. Our circuit generalizes the IM model by allowing for graded memory and arbitrary multiplexing of memory to other neural circuits. Because it is a population model, it is more robust to perturbations than the IM model, which transfers spikes between individual neurons. It is different from the IM model in that our circuit retains only one graded amplitude, not many (although this could be arranged) [5]. However, our model retains the multiple timescales that generate theta and gamma oscillations from pulse gating inherent to the IM model [5]. Additionally, other graded memory models based on input integration [45, 66] make use of relatively large time constants that are larger even than NMDA timescales, whereas ours makes use of an arbitrary synaptic timescale, τ, which may be modified to make use of any natural timescale in the underlying neuronal populations, including AMPA or NMDA. Our model is based on exact, analytical expressions, and because of this, the memory is infinitely long-lived at the mean-field level (until finite-size effects and other sources of variability are taken into account).

The circuit has four components, a population for binding a graded amplitude into the circuit (‘read in’), a cyclical memory, a ‘read out’ population meant to emulate the transfer of the graded amplitude to another circuit, and an input population. The memory is a set of n populations coupled one to the other in a circular chain with one of the populations (population 1) receiving gated input from the read in population. Memory populations receive coherent, phase shifted (by phase T) pulses that transfer the amplitude around the chain. In this circuit, n must be large enough that when population n transfers its amplitude back to population 1, population 1’s amplitude has relaxed back to (approximately) zero. The read out is a single population identically coupled to every other population in the circular chain. This population is repeatedly pulsed allowing the graded amplitude in the circular chain to be repeatedly read out.

In Fig. 4, we show an example of the memory circuit described here with n = 6. The gating pulses sequentially propagate the graded current amplitude around the circuit. The read out population is coupled to every other population in the memory. Thus, in this example, the oscillation frequency of the read out population is three times that of the memory populations, i.e. theta-band frequencies in the memory populations would give rise to gamma-band frequencies in the read out.

FIG. 4.

Memory circuit maintaining a single graded current amplitude. (Left) Connectivity Matrix. White denotes 0 entries and black denotes 1s. The connectivity matrix is subdivided into four sets of rows. “Input” designates filtered input from an outside source. The first row connects the “Read In” population to the input. The Read In population transduces the filtered input into a graded current packet that then propagates through the memory circuit. The “Cyclic Memory” contains cyclically connected, feedforward populations around which the graded packet is propagated. The “Read Out” population is postsynaptic to every other population in the Cyclic Memory and may be used to transfer graded packets at high frequencies to another circuit. (Middle) Gating Pulses. White denotes 0, black denotes g₀, the firing threshold. T/τ = 8. This sequence of gating pulses is used to bind and propagate the graded memory. Time runs from top to bottom. We show three complete cycles of propagation. The initial pulse on the Read In population binds the filtered input. The subsequent pulses within the cyclic memory rotate the packet through the memory populations. The pulses in the Read Out population copy the memory to a distinct population, which could be in another circuit. (Right) Current Amplitudes. White denotes 0, black denotes the maximum for this particular current packet. The input is transduced into the Read In population after time 0 (upper left of panel). The memory is subsequently propagated through the circuit and copied from every other population to the Read Out.

This memory circuit, and other circuits that we present below, has the property that the binding of information is instantiated by the pulse sequence and is independent of the information carried in graded amplitudes and also independent of synaptic processing. Because of the independence of the control apparatus from information content and processing, this neural circuit is an automatic processing pathway whose functional connectivity (both internal and input/output) may be rapidly switched on or off and coupled to or decoupled from other circuits. We propose that such dynamically routable circuits, including both processing and control components, are the neural correlates of automatic cognitive processes that have been termed zombie modes [67].

A Moving Window Fourier Transform

The memory circuit above used one-to-one coupling. It was simple in that information was copied, but not processed. Our second example demonstrates how more complex information processing may be accomplished within a zombie mode. With a simple circuit that performs a Hadamard transform (a Fourier transform using square-wave-shaped Walsh functions as a basis), we show how streaming information may be bound into a memory, then processed via synaptic couplings between populations in the circuit.

A set of read in populations are synaptically coupled to the input. A set of memory chains are coupled to the read in. The final population in each memory chain is coupled via a connectivity matrix that implements a Hadamard transform. Gating pulses cause successive read in and storing in memory of the input until the Hadamard transform is performed once the memory contains all successive inputs in a given time window simultaneously. Because the output of the Hadamard transform may be negative, two populations of Hadamard outputs are implemented, one containing positive coefficients, and another containing absolute values of negative coefficients.

In Fig. 5, we show a zombie mode where four samples are bound into the circuit from an input, which changes continuously in time. Memory populations hold the first sample over four transfers, the second sample over three transfers, etc. Once all samples have been bound within the circuit, the Hadamard transform is performed with a pulse on the entire set of Hadamard read out populations. While this process is occurring, a second sweep of the algorithm begins and a second Hadamard transform is computed.

FIG. 5.

4 × 4 Hadamard transform on a window of input values moving in time. (Left) Connectivity Matrix. White denotes −1/2, light gray denotes 0, dark gray denotes 1/2, and black denotes 1. The connectivity matrix is subdivided into three sets of rows. “Memory” designates Read In and (non-cyclic) Memory populations. “Hadamard” designates populations for the calculation of Hadamard coefficients. Because the packet amplitudes can only be positive, the Hadamard transform is divided into two parallel operations, one that results in positive coefficients and one that results in absolute values of negative coefficients. “Input” designates filtered input from an outside source. (Middle) Gating Pulses. White denotes 0, black denotes g₀. T/τ = 2. Time runs from top to bottom. We show the computation for two successive windows, each of length 4T. The pulses transduce the input into four memory chains of length 4T, 3T, 2T and T. Thus, four temporally sequential inputs are bound in four of the memory populations beginning at times t = 4, 8T. Hadamard transforms are performed beginning at t = 5, 9T. Note that the second read in starts one packet length before the Hadamard transform so that the temporal windows are adjacent. (Right) Current Amplitudes. White denotes 0, black denotes the maximum current amplitude. Purple outlines denote Read In, red denote Memory and blue denote Hadamard transform populations. The left four Hadamard outputs are positive coefficients. The right four are absolute values of negative coefficients. The sinusoidal input waveform is shown to the right.

The connectivity matrix for the positive coefficients of the Hadamard transform was given by

$$H=\frac{1}{2}\left[\begin{array}{cccc}\hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill -1\hfill & \hfill 1\hfill & \hfill -1\hfill \\ \hfill 1\hfill & \hfill 1\hfill & \hfill -1\hfill & \hfill -1\hfill \\ \hfill 1\hfill & \hfill -1\hfill & \hfill -1\hfill & \hfill 1\hfill \end{array}\right],$$

and the absolute values of the negative coefficients used the transform −H.

A Re-entrant Spatial Rotation Circuit

Our final example makes use of plastic internal connectivity to perform an arbitrary set of rotations of a vector on the sphere. Three fixed angle rotations about the x, y and z axes are arranged such that the output from each rotation may be copied to the input of any of the rotations. Because the destination is determined by the pattern of gating pulses, this circuit is more general than a zombie mode with a fixed gating pattern because it is not automatic: manipulation of the rotations would be expected to occur from a separate routing control circuit (here, implemented by hand).

In Fig. 6, initial spatial coordinates of (1, 1, 1) were input to the circuit. The pulse sequence rotated the input first about the x-axis, then sequentially about y, z, x, z, y, y, z, x axes. Views from two angles illustrate the rotations that were performed by the circuit.

FIG. 6.

Spatial rotation of three-dimensional coordinates. (Left, Top) A diagram of the circuit. Diamonds represent spatial rotation about the given axis, x, y, or z, by angle 2π/10. R_x takes coordinates 1, 2 and 3 as input and outputs 4, 5 and 6. R_y takes 7, 8 and 9 as inputs and outputs 10, 11 and 12. And R_z takes 13, 14 and 15 as input and outputs 16, 17 and 18. Outputs may be routed to any of R_x, R_y or R_z giving a reentrant circuit. (Left, Bottom) Connectivity Matrix. Light gray denotes −sin (2π/10), gray denotes 0, dark gray denotes sin (2π/10), black denotes 1, with some black squares representing cos (2π/10) indistinguishable from 1. Input may be read into 1, 2 and 3 only. Outputs of the rotations may be read into any of (1, 2, 3), (7, 8, 9) or (13, 14, 15) and subsequent rotations performed on these amplitudes. (Middle) Gating Pulses. White denotes 0, black denotes g₀. T/τ = 3. Time runs from top to bottom. We show read in, then routing of the coordinates through R_x, R_y, R_z, R_x, R_z, R_y, R_y, R_z, and R_x, successively. (Right) Current Amplitudes. Initially uniform coordinates (x, y, z) = (1, 1, 1), are successively rotated about the various axes. (Bottom) Two views of the coordinates connected by geodesics as they are rotated.

This circuit demonstrates the flexibility of the information coding network that we have introduced. It shows a complex circuit capable of rapid computation with dynamic routing, but with a fixed connectivity matrix. Additionally, it is an example of how a set of non-commuting generators may be used to form elements of a non-abelian group within our framework.

DISCUSSION

The existence of graded transfer mechanisms, such as the one that we have found, points toward a natural modular organization wherein each neural circuit would be expected to have 1) sparsely coupled populations of neurons that encode information content, 2) pattern generators that provide accurately timed pulses to control information flow, and 3) regulatory mechanisms for maintaining optimal transfer.

A huge literature now exists implicating oscillations as an important mechanism for information coding. Our mechanism provides a fundamental building block with which graded information content may be encoded and transferred in current amplitudes, dynamically routed with coordinated pulses, and transformed and processed via synaptic weights. From this perspective, coherent oscillations may be an indication that a neural circuit is performing complex computations pulse by pulse.

Our mechanism for graded current transfer has allowed us to construct a conceptual framework for the active manipulation of information in neural circuits. The most important aspect of this type of information coding is that it separates control of the flow of information from information processing and the information itself. This type of segregation has been made use of previously [5, 44, 49, 50] in mechanisms for gating the propagation of fixed amplitude waveforms. Here, by generalizing the mechanism to the propagation of graded information, active linear maps take prominence as a key processing structure in information coding.

The four functions that must be served by a neural code are [31, 68]: stimulus representation, interpretation, transformation and transmission. Our framework serves three of these functions, and we believe is capable of being extended to the fourth. From last to first: the exact transfer mechanisms that we have identified serve the transmission function; synaptic couplings provide the capability of transforming information; the pulse dependent, selective read out of information in part serves the interpretation function. In the examples that we showed above changes in pulse sequences were introduced by hand, but we argued in the Methods section that interaction of pulse chains should be able to achieve fully general decision making. Finally, read in populations, as we demonstrated in our examples, may be used to convert stimulus information into a bound information representation.

The current transfer mechanism is sufficiently flexible that the pulses used for gating may be of different durations depending on the pulse length, T, and the time constant, τ, of the neuronal population involved.

The separation of control populations from those representing information content distinguishes our framework from mechanisms such as CTC, where communication between neuronal populations depends on the co-incidence of integration windows in phase-coherent oscillations. In the CTC mechanism, information containing spikes must coincide to be propagated. In our framework, information containing spikes must coincide with gating pulses that enable communication. In this sense, it is ‘communication through coherence with a control mechanism’.

The separation of control and processing has further implications, one of which is that, as noted above, while a given zombie mode is processing incoming information, one does not expect the pulse sequence to change dependent on the information content. This has been seen in experiment, and presented as an argument against CTC in the visual cortex [69], but is consistent with our framework.

The basic unit of computation in our framework is a pulse gated transfer. Given this, we suggest that each individual pulse within an oscillatory set of pulses represents the transfer and processing of a discrete information packet. For example, in a sensory circuit that needs to quickly and repeatedly process a streaming external stimulus, short pulses could be repeated in a stereotyped, oscillatory manner using high-frequency gamma oscillations to rapidly move bound sensory information through the processing pathway. Circuits that are used occasionally or asynchronously might not involve oscillations at all, just a precise sequence of pulses that gate a specific information set through a circuit. A possible example of such an asynchronous circuit is bat echo-location, an active sensing mechanism, where coherent oscillations have not been seen [70].

An important point to note is that, given a zombie mode that implements an algorithm for processing streaming input, one can straightforwardly predict the rhythms that the algorithm should produce (for instance in our examples, calculate power spectra of the current amplitudes.). This feature of zombie modes can provide falsifiable hypotheses for putative computations that the brain uses to process information.

Since, with our transfer mechanism, information routing is enacted via precisely timed pulses, neural pattern generators would be expected to be information control centers. Cortical pattern generators, such as those proposed by Yuste [71] or hubs proposed by Jahnke, et al. [49], because of their proximity to cortical circuits, could logically be the substrate for zombie mode control pulses. They would be expected to generate sequential, stereotyped pulses to dynamically route information flow through a neural circuit, as has been found in rat somatosensory cortex [72]. Global routing of information via attentional processes, on the other hand, would be expected to be performed from brain regions with broad access to the cortex, such as regions in the basal ganglia or thalamus.

Recent evidence shows that parvalbumin-positive basket cells (PVBCs) can gate the conversion of current to spikes in the amygdala [73]. Also, PVBCs and oriens lacunosum moleculare (OLM) cells have been implicated in precision spiking related to gamma- and theta-oscillations [43] and shown to be involved in memory-related structural-plasticity [74]. Therefore, zombie mode pattern generators would likely be based on a substrate of these neuron types.