Pooling and Correlated Neural Activity

Abstract

Correlations between spike trains can strongly modulate neuronal activity and affect the ability of neurons to encode information. Neurons integrate inputs from thousands of afferents. Similarly, a number of experimental techniques are designed to record pooled cell activity. We review and generalize a number of previous results that show how correlations between cells in a population can be amplified and distorted in signals that reflect their collective activity. The structure of the underlying neuronal response can significantly impact correlations between such pooled signals. Therefore care needs to be taken when interpreting pooled recordings, or modeling networks of cells that receive inputs from large presynaptic populations. We also show that the frequently observed runaway synchrony in feedforward chains is primarily due to the pooling of correlated inputs.

Introduction

Cortical neurons integrate inputs from thousands of afferents. Similarly, a variety of experimental techniques record the pooled activity of large populations of cells. It is therefore important to understand how the structured response of a neuronal network is reflected in the pooled activity of cell groups.

It is known that weak dependencies between the response of cell pairs in a population can have a significant impact on the variability and signal-to-noise ratio of the pooled signal (Shadlen and Newsome, 1998; Salinas and Sejnowski, 2000; Moreno-Bote et al., 2008). It has also been observed that weak correlations between cells in two populations can cause much stronger correlations between the pooled activity of the populations (Bedenbaugh and Gerstein, 1997; Chen et al., 2006; Gutnisky and Josić, 2010; Renart et al., 2010). We give a simple example of this effect in Figure 1C: Weak correlations were introduced between the spiking activity of cells in two non-overlapping presynaptic pools each providing input to a postsynaptic cell (see diagram in Figure 1B). The activity between pairs of excitatory, and pairs of inhibitory cells was correlated, but excitatory–inhibitory pairs were uncorrelated. Even without shared inputs and with background noise, pooling resulted in strong correlations in postsynaptic membrane voltages. The connectivity in the presynaptic network was irrelevant – it only mattered that the inputs to the downstream neurons reflected the pooled activity of the afferent populations. A similar effect can cause large correlations between recordings of multiunit activity (MUA) or recordings of voltage sensitive dyes (VSD), even when correlations between cells in the recorded populations are small (Bedenbaugh and Gerstein, 1997; Chen et al., 2006; Stark et al., 2008). The effect is the same, but in this case pooling occurs at the level of a recording device rather than a downstream neuron (compare Figures 1A,B).

Figure 1.

Models of pooled recordings and the effects of pooling on correlations. (A) Pooling in experimental recordings. Cells from different populations are correlated with average correlation coefficient ${\overline{\text{ρ}}}_{12}$ and cells from the same population have average correlation coefficient ${\overline{\text{ρ}}}_{11}$ or ${\overline{\text{ρ}}}_{22}.$ We examine correlations between the pooled signals, X₁ and X₂. (B) Pooled inputs to cell pairs. Individual excitatory (e) and inhibitory (i) inputs are correlated with coefficient ρ_ee, ρ_ii, and ρ_ei, respectively. The total input to a cell is the summed activity of its excitatory (E_K) and inhibitory (I_K) presynaptic population. The membrane potentials, V₁ and V₂, are obtained by filtering these inputs. (C) A simulation of the setup in (B) with background noise. Correlations between excitatory and between inhibitory cells were uniform (ρ_ee = ρ_ii = 0.05), but excitatory–inhibitory correlations were absent (ρ_ei = 0). The raster plot shows the activity in a subset of the input population. The correlation coefficient between the sub-threshold activity of the postsynpatic cells was ρ_vv = 0.768 ± 0.001 s.e. Each cell receives 250 correlated and 250 uncorrelated excitatory Poisson inputs as well as 84 correlated and 84 uncorrelated inhibitory Poisson inputs (n_e = 250 and n_i = 84, q_e = q_i = 1, ν_e = 5 Hz, ν_i = 7.5 Hz, ℐ = 4ℰ, and ℰ≈2.3 nS·ms – see Materials and Methods and Figure 3A for notation and precise model description). (D) Same as (B), but with ρ_ei = 0.05, ν_e = ν_i=5 Hz, ℐ = 6ℰ to maintain balance. In this case ρ_VV = 0.0085 ± 0.0024 s.e.. The simulations were run 8000 times at 10 s each.

We present a systematic overview, as well as extensions and applications of a number of previous observations related to this phenomenon. Using a linear model, we start by examining the potential effects of pooling on recordings from large populations obtained using VSD or MUA recording techniques. These techniques are believed to reflect the pooled postsynpatic activity of groups of cells. We extend earlier models introduced to examine the impact of pooling on correlations (Bedenbaugh and Gerstein, 1997; Chen et al., 2006; Nunez and Srinivasan, 2006), and show that heterogeneities in the presynaptic pools can have subtle effects on correlations between pooled signals.

Since neurons respond to input from large presynaptic populations, pooling also impacts the activity of single cells and cell pairs. As observed in Figure 1C, pooling can inflate weak correlations between afferents. However, excitatory–inhibitory correlations (Okun and Lampl, 2008) can counteract this amplification, as shown in Figure 1D (Hertz, 2010; Renart et al., 2010). We examine these effects analytically by modeling the subthreshold activity of postsynaptic cells as a filtered version of the inputs received (Tetzlaff et al., 2008). The impact of correlated subthreshold activity on the output spiking statistics is a nontrivial question which we address only briefly (Moreno-Bote and Parga, 2006; de la Rocha et al., 2007; Ostojić et al., 2009).

The effects of pooling provide a simple explanation for certain aspects of the dynamics of feedforward chains. Simulations and in vitro experiments show that layered feedforward architectures give rise to a robust increase in synchronous spiking from layer to layer (Diesmann et al., 1999; Litvak et al., 2003; Reyes, 2003; Doiron et al., 2006; Kumar et al., 2008). We describe how output correlations in one layer impact correlations between the pooled inputs to the next layer. This approach is used to derive a mapping that describes how correlations develop across layers (Tetzlaff et al., 2003; Renart et al., 2010), and to illustrate that the pooling of correlated inputs is the primary mechanism responsible for the development of synchrony in feedforward chains. Examining how correlations are mapped between layers also helps explain why asynchronous states are rarely observed in feedforward networks in the absence of strong background noise (van Rossum et al., 2002; Vogels and Abbott, 2005). This is in contrast to recurrent networks which can display stable asynchronous states (Hertz, 2010; Renart et al., 2010) similar to those observed in vivo (Ecker et al., 2010).

Materials and Methods

Correlations between stochastic processes

The cross-covariance of a pair of stationary stochastic processes, x(t) and y(t), is C_xy(t) = cov(x(s), y(s + t)). The auto-covariance function, C_xx(t), is the cross-covariance between a process and itself. The cross- and auto-covariance functions measure second order dependencies at time lag t between two processes, or a process and itself. We quantify the total magnitude of interactions over all time using the asymptotic statistics,

$${\gamma }_{xy}={\displaystyle \underset{-\infty }{\overset{\infty }{\int }}{C}_{xy}(t)dt},{\sigma }_x^2={\gamma }_{xx},{\rho }_{xy}=\frac{{\gamma }_{xy}}{{\sigma }_x{\sigma }_y}.$$ (1)

While the asymptotic correlation, ρ_xy,, measures correlations between x(t) and y(t) over large timescales, the auto- and cross- covariance functions determine the timescale of these dependencies.

Correlations between sums of random variables

Given two collections of correlated random variables ${\left\{x_i\right\}}_{i=1}^{n_x}$ and ${\left\{y_j\right\}}_{j=1}^{n_y},$ define the pooled variables, $X={\Sigma }_i{x}_i$ and $Y={\Sigma }_i{y}_i.$ Since covariance is bilinear $(cov({\Sigma }_i{x}_i,{\Sigma }_j{y}_j)={\Sigma }_{ij}cov(x_i,y_j))$ the variance and covariance of the pooled variables are

$${\sigma }_X^2={\displaystyle \sum _{i=1}^{n_x}{\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne i\end{array}}^{n_x}{\sigma }_{x_i}{\sigma }_{x_j}{\rho }_{x_i{x}_j}}+{\displaystyle \sum _{i=1}^{n_x}{\sigma }_{x_i}^2},\text{\hspace{1em}and\hspace{1em}}}{\gamma }_{XY}={\displaystyle \sum _{i=1}^{nx}{\displaystyle \sum _{j=1}^{ny}{\sigma }_{x_i}{\sigma }_{y_j}{\rho }_{x_i{y}_j}}},$$

and similarly for ${\sigma }_Y^2.$

Using these expressions along with some algebraic manipulation, the correlation coefficient, ${\rho }_{XY}={\gamma }_{XY}/\sqrt{{\sigma }_X{\sigma }_Y},$ between the pooled variables can be written as

$${\rho }_{XY}=\frac{\overline{\rho }xy}{\sqrt{\left[w_x{\overline{\rho }}_{xx}+\frac{1}{n_x}\left(\frac{{\overline{v}}_x}{\overline{{\sigma }_x{\sigma }_y}}-w_x{\overline{\rho }}_{xx}\right)\right]\left[w_y{\overline{\rho }}_{yy}+\frac{1}{n_y}\left(\frac{{\overline{v}}_y}{\overline{{\sigma }_x{\sigma }_y}}-w_y{\overline{\rho }}_{yy}\right)\right]}}$$ (2)

$$=\frac{{\overline{\rho }}_{xy}}{\sqrt{{\overline{\rho }}_{xx}{\overline{\rho }}_{yy}}}+\mathcal{O}\left(\frac{1}{\sqrt{n_x{n}_y}}\right),$$ (3)

where

$$\begin{array}{l}{w}_x=\frac{\overline{{\sigma }_x{\sigma }_x}}{\overline{{\sigma }_x{\sigma }_y}},\overline{v_x}=\frac{1}{n_x}{\displaystyle \sum _{i=1}^{n_x}{\sigma }_{x_i}^2},\\ \overline{{\sigma }_x{\sigma }_y}=\frac{1}{n_x{n}_y}{\displaystyle \sum _{i=1}^{n_x}{\displaystyle \sum _{j=1}^{n_y}{\sigma }_{x_i}{\sigma }_{y_i}}},\overline{{\sigma }_x{\sigma }_x}=\frac{1}{n_x\left(n_x-1\right)}{\displaystyle \sum _{i=1}^{n_x}{\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne 1\end{array}}^{n_x}{\sigma }_{x_i}{\sigma }_{x_j}}},\\ {\overline{\rho }}_{xy}=\frac{1}{n_x{n}_y\overline{{\sigma }_x{\sigma }_y}}{\displaystyle \sum _{i=1}^{n_x}{\displaystyle \sum _{j=1}^{n_y}{\sigma }_{x_i}{\sigma }_{y_j}{\rho }_{x_i{y}_j}}},\\ {\overline{\rho }}_{xx}=\frac{1}{n_x\left(n_x-1\right)\overline{{\sigma }_x{\sigma }_x}}{\displaystyle \sum _{i=1}^{n_x}{\displaystyle \sum _{\begin{array}{l}j=1\\ j\ne 1\end{array}}^{n_x}{\sigma }_{x_i}{\sigma }_{y_j}{\rho }_{x_i{x}_j}}}\end{array}$$

and similarly for w_y, v_y, $\overline{{\sigma }_y{\sigma }_y},$ and ${\overline{\rho }}_{yy}.$ In deriving Eq. (3) we assumed that all pairwise statistics are uniformly bounded away from zero in the asymptotic limit.

Each overlined term above is a population average. Notably, ${\overline{\rho }}_{xy}$ represents the average correlation between x_i and y_j pairs, weighted by the product of their standard deviations, and similarly for ${\overline{\rho }}_{xx}$ and ${\overline{\rho }}_{yy}.$ Correlation between weighted sums can be obtained by substituting $x_i\to w_{x_i}{x}_i$ and $y_j\to w_{y_j}{y}_j$ for weights $w_{x_i}$ and $w_{y_j}$ and making the appropriate changes to the terms in the equation above (e.g., ${\sigma }_{x_i}\to \left|w_{x_i}\right|{\sigma }_{x_i},{\rho }_{x_i{y}_j}\to \text{sign\hspace{0.17em}}(w_i{w}_j){\rho }_{x_i{y}_j}$). Overlap between the two populations can be modeled by taking ${\rho }_{x_i{y}_j}=1$ for some pairs.

Assuming that variances are homogeneous within each population, that is ${\sigma }_{x_i}={\sigma }_x$ and ${\sigma }_{y_j}={\sigma }_y$ for i = 1,…,n_x and j = 1,…,n_y, simplifies these expressions. In particular, $\overline{v_x}=\overline{{\sigma }_x{\sigma }_x}={\sigma }_x^2,$ $\overline{{\sigma }_x{\sigma }_y}={\sigma }_x{\sigma }_y,$ and

$${\rho }_{XY}=\frac{{\overline{\rho }}_{xy}}{\sqrt{\left[{\overline{\rho }}_{xx}+\frac{1}{n_x}\left(1-{\overline{\rho }}_{xx}\right)\right]\hspace{0.17em}\left[{\overline{\rho }}_{yy}+\frac{1}{n_y}\left(1-{\overline{\rho }}_{yy}\right)\right]}}.$$ (4)

Assuming further that the populations are symmetric, σ_x = σ_y = σ, n_x = n_y = n, and ${\overline{\rho }}_{xx}={\overline{\rho }}_{yy},$ the expression above simplifies to

$${\rho }_{XY}=\frac{{\rho }^b}{{\rho }^w+\frac{1}{n}\left(1-{\rho }^w\right)}=\frac{{\rho }^b}{{\rho }^w}+\mathcal{O}\left(\frac{1}{n}\right),$$ (5)

where ${\rho }^b={\overline{\rho }}_{xy}$ is the average pairwise correlation between the two populations and ${\rho }^w={\overline{\rho }}_{xx}={\overline{\rho }}_{yy}$ is the average pairwise correlation within each population. Eq. (5) was derived in Bedenbaugh and Gerstein (1997) in an examination of correlations between multiunit recordings. In Chen et al. (2006), a version of Eq. (5) with ρ^w = ρ^b is derived in the context of correlations between two VSD signals. The asymptotic, ρ_xy → 0, limit when ρ^w = ρ^b is discussed in Renart et al. (2010).

Note that the results above hold for correlations computed over arbitrary time windows. We concentrate on infinite windows, and discuss extensions in the Appendix.

Neuron model

In the second part of the presentation we consider two excitatory and two inhibitory input populations projecting to two postsynaptic cells. The j^th excitatory input to cell k is labeled e_j,k(t) (k = 1 or 2). Similarly, i_j,k(t) denotes the j^th inhibitory input to cell k. Each cell receives n_e excitatory and n_i inhibitory inputs with individual rates ν_e and ν_i respectively.

Each of the excitatory and inhibitory inputs to cell k, are stationary spike trains modeled by point processes, ${\text{e}}_{j,k}(t)={\Sigma }_i\delta (t-t_{j,k}^i)$ and ${\text{i}}_{j,k}(t)={\Sigma }_i\delta (t-s_{j,k}^i)$ where $\left\{t_{j,k}^i\right\}$ and $\left\{s_{j,k}^i\right\}$ are input spike times. We assume that the spike trains are stationary in a multivariate sense (Stratonovich, 1963). The pooled excitatory and inhibitory inputs to neuron k are $E_k(t)={\Sigma }_{j=1}^{n_{e_k}}{\text{e}}_{j,k}(t),$ and $I_k(t)={\Sigma }_{j=1}^{n_{{\text{i}}_k}}{\text{i}}_{j,k}(t).$

To generate correlated inputs to cells, we used the multiple interaction process (MIP) method (Kuhn et al., 2003), then jittered each spike time independently by a random value drawn from an exponential distribution with mean 5ms. The resulting processes are Poisson with cross-covariance functions proportional to a double exponential, C_xy(t) ∼ e^{− |t| /5}. Note that since each input is Poisson, ${\sigma }_{\text{e}}^2={\nu }_{\text{e}}$ and ${\sigma }_{\text{i}}^2={\nu }_{\text{i}}.$

While the dynamics of the afferent population were not modeled explicitly, the response of the two downstream neurons was obtained using a conductance-based IF model. The membrane potentials of the neurons were described by

$$C_m\frac{d{V}_k}{dt}=-g_L(V_k-V_L)-g_{E_k}(t)(V_k-V_E)-g_{I_k}(t)(V_k-V_I),$$ (6)

with excitatory and inhibitory conductances determined by $g_{E_k}(t)=({\text{E}}_k\ast {\alpha }_{\text{e}})(t)$ and $g_{{\text{I}}_k}(t)=({\text{I}}_k\ast {\alpha }_{\text{i}})(t)$ where * denotes convolution. We used synaptic responses of the form ${\alpha }_{\text{e}}(t)=ℰ{\tau }_{\text{e}}^{-2}t{\text{e}}^{-t/{\tau }_{\text{e}}}\Theta (t)$ and ${\alpha }_{\text{i}}(t)=ℐ{\tau }_{\text{i}}^{-2}t{e}^{-t/{\tau }_{\text{i}}}\Theta (t),$ where Θ(t) is the Heaviside function. The area of a single excitatory or inhibitory postsynaptic conductance (EPSC or IPSC) is therefore equal to the synaptic weight, ℰ or ℐ, with units nS·ms. This analysis can easily be extended to situations where each input, e_j,k or i_j,k, has a distinct synaptic weight.

When examining spiking activity, we assume that when V_k crosses a threshold voltage, V^th, an output spike is produced and V_k is reset to V_L. When examining sub-threshold dynamics, we considered the free membrane potential without threshold.

As a measure of balance between excitation and inhibition we used (Troyer and Miller, 1997; Salinas and Sejnowski, 2000)

$$\beta =\frac{\left|V_L-V_{\text{E}}\right|ℰ{\nu }_{\text{e}}{n}_{\text{e}}}{\left|V_L-V_{\text{I}}\right|ℐ{\nu }_i{n}_i}.$$

When β = 1, the net excitation and inhibition are balanced and the mean free membrane potential equals V_L. In simulations, we set V_L = −60 mV, V_E = 0 mV, V_I = −90 mV, τ_e = 10 ms, τ_i = 20 ms, C_m = 114 pF, and g_L = 4.086 nS, giving a membrane time constant, τ_m = C_m/g_L = 27.9 ms. In all simulations except those in Figure 7, the cells are balanced (β = 1).

Figure 7.

Development of synchrony in feedforward networks. (A) A feedforward network with no overlap and independent, Poisson input. For excitatory cells, we set ℰ ≈ 1.55nS·ms, and ℐ ≈ 4.67nS·ms. For inhibitory cells, ℰ ≈ 3.61nS·ms, and ℐ ≈ 10.82nS·ms. (B) Same as A, except inputs to layer 1 are correlated with coefficient ρ₀ = 0.05. The network is highly synchronized in the second layer, even though inputs do not overlap. (C) Same as A, except for the presence of overlapping inputs (p_e = p_i = 0.05). Correlations due to overlap in the input to layer 2 result in average correlations of 0.05 between input currents. Layer 3 cells in C synchronize (Compare with layer 2 in B). In all three figures, each cell in the first layer was driven by excitatory Poisson inputs with rate ν₀ = 100 Hz.

The conductance-based IF neuron behaves as a nonlinear filter in the sense that membrane potentials cannot be written as a linear transformation of the inputs. However, following Kuhn et al. (2004) and Coombes et al. (2007), we derive a linear approximation to the conductance based model. Let U = V_k − V_L so that Eq. (6) becomes

$$C_m\frac{dU}{dt}=\left(-g_L-g_{\text{E}}(t)-g_{\text{I}}(t)\right)U-g_{\text{E}}(t)\left(V_L-V_{\text{E}}\right)-g_{\text{I}}(t)\left(V_L-V_{\text{I}}\right).$$

Define the effective membrane time constant,τ_effC_m/(E[g_L+g_E(t) +g_L(t)]) = C_m/(g_L +n_ev_eℰ +n_iν_iℐ Substituting this average value in the previous equation yields the linear approximation to the conductance based model,

$$\frac{dU}{dt}=-\frac{1}{{\tau }_{\text{eff}}}U+J_k(t),$$ (7)

where $J_k(t)=(-g_{{\text{E}}_k}(t)(V_L-V_{\text{E}})-g_{{\text{I}}_k}(t)(V_L-V_{\text{I}}))/C_m$ is the total input current to cell k. Solving and reverting to the original variables gives the linear approximation V_k(t) = (J_k*K)(t) + V_L, where $K(t)=\Theta (t){\text{e}}^{-t/{\tau }_{\text{eff}}}$ is the kernel of the linear filter induced by Eq. (7).

Results

The pooling of signals from groups of neurons can impact both recordings of population activity and the structure of inputs to postsynaptic cells. We start by discussing correlations in pooled recordings using a simple linear model. A similar model is then used to examine the impact of pooling on the statistics of inputs to cells. For simplicity we assume that all spike trains are stationary. However, non-stationary results can be obtained using similar methods as outlined in the Section “Discussion.” Though all parameters are defined in the Meterials and Methods, Tables 1 and 2 in the Appendix contain brief descriptions of parameters for quick reference. Also, Tables 3 and 4 summarize the values of parameters used for simulations throughout the article.

Table 1.

Definitions of variables pertaining to recordings.

X1(t), X2(t)	Signals from two populations.
${\text{ρ}}_{X_1{X}_2}$	Correlation between the signals.
${\overline{\text{ρ}}}_{jk}$	Average pairwise correlation between a cell in population j and a cell in population k.
r(d)	Firing rate of a cell at distance d from the center of a stimulus.

Table 2.

Definitions of variables pertaining to downstream cells. Subscripts e and E (i and I) denote excitation (inhibition).

ne, ni	Number of correlated inputs to a cell.
νe, νi	Input rates.
ℰ, ℐ	Synaptic weights.
qe, qi	Neurons received neqe(niqi) independent excitatory (inhibitory) inputs.
ρee, ρii, ρei	Correlations between pairs of afferents.
${\text{ρ}}_{xy}^{\text{w}},{\text{ρ}}_{xy}^b$	Correlations within or between two non-overlapping populations (x,y = e,i).
pe, pi	Proportion of shared input to the post-synaptic pair.
Ne, Ni	Number of cells per layer in feed-forward network model.
Ej, Ij	Pooled input spike trains to cell j.
σE, σI	Standard deviation of pooled excitatory or inhibitory spike trains.
${\text{γ}}_{{\text{E}}_j{\text{E}}_k},{\text{γ}}_{{\text{E}}_j{\text{I}}_k},{\text{γ}}_{{\text{I}}_j{\text{I}}_k}$	Covariance between pooled spike trains.
${\text{ρ}}_{{\text{E}}_j{\text{E}}_k},{\text{ρ}}_{{\text{E}}_j{\text{I}}_k},{\text{ρ}}_{{\text{I}}_j{\text{I}}_k}$	Correlations between pooled spike trains.
β	Measure of balance between excitation and inhibition in the inputs.
${\text{ρ}}_k^{\text{in}},{\text{ρ}}_k^{\text{out}}$	Correlations between inputs to or outputs from cell pairs in a feedforward network.
CXY(t)	Cross-covariance function between processes X and Y.
P(ρ)	Correlations between the pooled inputs to cells in the feedforward model.
S(ρ)	Correlation between output spike trains in terms of input current correlations between cell pairs in the feedforward model.

Table 3.

Parameter values for simulations of two downstream cells. For fields with “var,” various values of the indicated parameters were used and are described in the captions. For all simulations, V_L = −60 mV, V_E = 0 mV, V_I = −90 mV, C_m = 114 pF, g_L = 4.086 nS, τ_e = 10 ms, τ_i = 20 ms.

	ρee, ρii	ρei	ne, ni	pe, pi	qe, qi	νe, νi (Hz)	ℰ,ℐ (nS·ms)
Figure 1C	0.05	0	250, 84	0	1	5, 7.5	2.3, 9.2
Figure 1D	0.05	0.05	250, 84	0	1	5	2.3, 13.8
Figure 5	var	var	var, ne/3	var	var	5, 7.5	590/ne, 4ℰ
Figure 8	0.05	0.05	250, 84	0	1	5	2.3, 13.8

Table 4.

Parameter values for simulations of feedforward networks. The parameter ν₀ is the input rate to the first layer, (ℰ, ℐ)_e indicates synaptic weights for excitatory cells, and (ℰ, ℐ)_i for inhibitory cells. For all simulations, V_L = −60 mV, V_E = 0 mV, V_I = −90 mV, C_m = 114 pF, g_L = 4.086 nS, τ_e = 10 ms, τ_i = 20 ms. For Figure 6, theoretical values were obtained under the assumption that ν_e = ν_i and ℰ|V_E − V_L| = ℐ|V_I − V_L.

	ne, ni	Ne, Ni	pe, pi	ν0 (Hz)	ρ0	(ℰ, ℐ)e, (ℰ, ℐ)i (nS·ms)
Figure 6C	600, 400	12000, 8000	0.05	NA	0	NA
Figure 6D	600, 525	12000, 10500	0.05	NA	0	NA
Figure 7A	225, 75	NA	0	100	0	(1.55,5.67),(3.61,10.82)
Figure 7B	225, 75	NA	0	100	0.05	(1.55,5.67),(3.61,10.82)
Figure 7C	225, 75	4500, 1500	0.05	100	0	(1.55,5.67),(3.61,10.82)

Correlations between pooled recordings

Pooling can impact correlations between recordings of population activity obtained from voltage sensitive dyes (VSDs), multi-unit recordings and other techniques. Such signals might each represent the summed activity of hundreds or thousands of neurons. Let two recorded signals, X₁(t) and X₂(t), represent the weighted activity of cells in two populations (see diagram in Figure 1A). If we assume homogeneity in the input variances and equal size of the recorded populations, Eq. (4) gives the correlation between the recorded signals

$${\rho }_{X_1{X}_2}=\frac{{\overline{\rho }}_{12}}{\sqrt{\left[{\overline{\rho }}_{11}+\frac{1}{n}\left(1-{\overline{\rho }}_{11}\right)\right]\hspace{0.17em}\left[{\overline{\rho }}_{22}+\frac{1}{n}\left(1-{\overline{\rho }}_{22}\right)\right]}}=\frac{{\overline{\rho }}_{12}}{\sqrt{{\overline{\rho }}_{11}{\overline{\rho }}_{22}}}+\mathcal{O}\left(\frac{1}{n}\right).$$ (8)

Here n represents the number of neurons recorded, ${\overline{\rho }}_{kk},$ k = 1,2 represents the average correlation between cells contributing to signal X_k(t), and ${\overline{\rho }}_{12}$ represents the average correlation between cells contributing to different signals. The averages are weighted so that cells that contribute more strongly to the recording, such as those closer to the recording site, contribute more to the average correlations (see Materials and Methods). Cells common to both recorded populations can be modeled by setting the corresponding correlation coefficients to unity. A form of Eq. (8) with ${\overline{\rho }}_{11}={\overline{\rho }}_{22}$ was derived by Bedenbaugh and Gerstein (1997).

When the two recording sites are nearby, so that ${\overline{\rho }}_{12}\approx {\overline{\rho }}_{11}\approx {\overline{\rho }}_{22},$ even small correlations between individual cells are amplified by pooling so that the correlations between the recorded signals can be close to 1. This effect was observed in experiments and explained in similar terms in Stark et al. (2008).

A significant stimulus-dependent change in correlations between individual cells might be reflected only weakly in the correlation between the pooled signals. This can occur, for instance, in recordings of large populations when ${\overline{\text{ρ}}}_{12},$${\overline{\text{ρ}}}_{11},$ and ${\overline{\text{ρ}}}_{22}$ are increased by the same factor when a stimulus is presented. Similarly, an increase in correlations between cells can actually lead to a decrease in correlations between recorded signals when ${\overline{\text{ρ}}}_{11}$ and ${\overline{\text{ρ}}}_{22}$ increase by a larger factor than${\overline{\text{ρ}}}_{12}$.

To illustrate these effects, we construct a simple model of stimulus dependent correlations motivated by the experiments in Chen et al. (2006), in which VSDs were used to record the population response in visual area V1 during an attention task. In their experiments, the imaged area is divided into 64 pixels, each 0.25 mm × .25 mm in size. The signal recorded from each pixel represents the pooled activity of n ≈ 1.25 × 10⁴ neurons.

We model correlations between the signals, X₁(t) and X₂(t), recorded from two pixels in the presence or absence of a stimulus (see Figure 2B), using a simplified model of stimulus dependent rates and correlations. The firing rate of a cell located at distance d from the center of the retinotopic image of a stimulus is

Figure 2.

The effect of pooling on recordings of stimulus dependent correlations. (A) The response amplitude of a model neuron as a function of its distance from the retinotopic image of a stimulus [Eq. (9)] with B = 0.05 and λ = 10. (B) A diagram of our model. Signals X₁(t) and X₂(t) are recorded from two pixels (red and blue squares). The activity in response to a stimulus is shown as a gradient centered at some pixel (the center of the retinotopic image of the stimulus). (C) The prediction of the correlation between two pixels obtained using the stimulus-dependent model considered in the text with stimulus present (red) and absent (green). We assumed that one pixel is located at the stimulus center (d₁ = 0). Parameters are as in (A) with α = 1, S = 0.1, and n = 1.25×10⁴. A stimulus dependent change in correlations is undetectable. (D) Same as in (C), except that baseline activity, B, was scaled by 0.5 in the presence of a stimulus. Compare to Figure 2f in Chen et al., 2006.

$$r(d)=\{\begin{array}{cc}B+\frac{(1-B){(1+\cos (d\pi ))}^{\lambda }}{2}& \text{stimulus\hspace{0.17em}present}\\ B\text{\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{0.17em}\hspace{0.05em}}& \text{stimulus\hspace{0.17em}absent.}\end{array}$$ (9)

Here, B ∈ [0,1] represents baseline activity and λ ≥ 1 controls the rate at which activity decays with d. Both d and r were scaled so that their maximum value is 1 (see Figure 2A).

We assume that the correlation between the responses of two neurons is proportional to the geometric mean of their firing rates (de la Rocha et al., 2007; Shea-Brown et al., 2008), and that correlations decay exponentially with cell distance (Smith and Kohn, 2008; see however Poort and Roelfsema, 2009; Ecker et al., 2010). We therefore model the correlation between two cells as ${\text{ρ}}_{\mathrm{jk}}=S\sqrt{r\left(d_j\right)r\left(d_k\right)}{\text{e}}^{-\alpha D_{j,k}}$ where d_j and d_k are the distances from each cell to the center of the retinotopic image of the stimulus, D_j,k is the distance between cells j and k, α is the rate at which correlations decay with distance, and S ≤ 1 is a constant of proportionality.

If pixels are small compared to the scales at which correlations are assumed to decay, then the average correlation between cells within the same pixel are ${\overline{\text{ρ}}}_{11}=Sr(d_1)$ and ${\overline{\text{ρ}}}_{22}=Sr(d_2)$. The average correlation between cells in different pixels is ${\overline{\text{ρ}}}_{12}=S\sqrt{r\left(d_1\right)r\left(d_2\right)}{\text{e}}^{-\alpha D_{1,2}}.$

In this case, whether a stimulus is present or not, the correlation between the pooled signals is of the form ${\text{ρ}}_{X_1{X}_2}={\text{e}}^{-\alpha D_{1,2}}+\mathcal{O}(1/n).$ Thus, even significant stimulus dependent changes in correlations would be invisible in the recorded signals. This overall trend is consistent with the results in Chen et al. (2006) (compare Figure 2C to their Figure 2f). In such settings, it is difficult to conclude whether pairwise correlations are stimulus dependent or not from the pooled data.

However, in Supplementary Figure 3 of Chen et al. (2006) the presence of a stimulus apparently results in a slight decrease in correlations between more distant pixels. In Figure 2D this effect was reproduced using the alternative model described above, with the additional assumption that baseline activity, B, decreases in the presence of a stimulus (Mitchell et al., 2009). The effect can also be reproduced by assuming that spatial correlation decay, α, increases when a stimulus is present.

As this example shows, care needs to be taken when inferring underlying correlation structures from pooled activity. The statistical structure of the recordings can depend on pairwise correlations between individual cells in a subtle way, and different underlying correlation structures may be difficult to distinguish from the pooled signals. However, downstream neurons may also be insensitive to the precise structure of pairwise correlations, as they are driven by the pooled input from many afferents.

Correlations between the pooled inputs to cells

We next examine the effects of pooling by relating the correlations between the activity of downstream cells to the pairwise correlations between cells in the input populations (see Figure 1B). The idea that pooling amplifies correlations carries over from the previous section. However, the presence of inhibition and non- instantaneous synaptic responses introduces new issues.

A homogeneous population with overlapping and independent inputs

For simplicity, we first consider a homogeneous population model (see Figure 3A). Each cell receives n_e inputs from a homogeneous pool of inputs with pairwise correlation coefficients ρ_ee and an additional q_en_e inputs from an outside pool of independent inputs. The two cells share p_en_e of the inputs drawn from the correlated pool. Processes in the independent pool are uncorrelated with all other processes. All excitatory inputs have variance ${\sigma }_{\text{e}}^2.$

Figure 3.

Two population models considered in the text. (A) Homogeneous population with overlap and independent inputs: A homogeneous pool of correlated inputs (large black circle) with correlation coefficient between any pair of processes equal to ρ_ee. Each cell draws n_e inputs (larger red and blue circles) from this homogeneous input pool. Of these n_e correlated inputs, p_en_e are shared between the two neurons (purple dots). In addition, each cell receives q_en_e independent inputs (smaller red and blue circles), for a total of n_e + q_en_e inputs. All inputs have variance ${\sigma }_{\text{e}}^2.$ (B) A population model with distinct “within” and “between” correlations: Each cell receives n_e inputs. The average correlation between two inputs to the same cell is ${\text{ρ}}_{\text{ee}}^{\text{w}},$ and between inputs to different cells is ${\text{ρ}}_{\text{ee}}^{\text{b}}.$

The correlation between the pooled excitatory inputs is given by (see Appendix)

$${\rho }_{{\text{E}}_1{\text{E}}_2}=\frac{{\rho }_{\text{ee}}+\frac{p_{\text{e}}}{n_{\text{e}}}\left(1-{\rho }_{\text{ee}}\right)}{{\rho }_{\text{ee}}+\frac{1}{n_{\text{e}}}\left(1-{\rho }_{\text{ee}}+q_{\text{e}}\right)}.$$ (10)

A form of this equation, with p_e = 0 and q_e = 0, is derived in Chen et al. (2006). In the absence of correlations between processes in the input pools, ρ_ee = 0, the correlation between the pooled signals is just the proportion of shared inputs, ${\text{ρ}}_{{\text{E}}_1{\text{E}}_2}=p_{\text{e}}.$ When ρ_ee > 0 and n_e is large, pooled excitatory inputs are highly correlated, even when pairwise correlations in the presynaptic pool, ρ_ee, are small, and the neurons do not share inputs (p_e = 0). Even when most inputs to the downstream cells are independent (q_e > 1), correlations between the pooled signals will be nearly 1 for sufficiently large input pools (see Figure 4A).

Figure 4.

The effect of pooling on correlations between summed input spike trains. (A) The correlation coefficient between the pooled excitatory spike trains (ρ_EE) is shown as a function of the size of the correlated excitatory input pool (n_e) for various parameter settings. The solid blue line was obtained by setting ρ_ee = 0.05 for the population model in Figure 3A in the absence of shared or independent inputs (p_e = q_e = 0). The dashed line illustrates the decorrelating effects of the addition of n_e independent inputs (q_e = 1, q_e = 0, ρ_ee = 0.05). The dotted blue line shows that shared inputs increase correlations, but have a diminishing effect on ρ_EE with increasing input population size (p_e = 0.2, q_e = 0, ρ_ee = 0.05). The solid pink line shows the effect of reducing the pairwise input correlations (ρ_ee = 0.005, p_e = q_e = 0). The dashed tan line was obtained with uncorrelated inputs so that correlations reflected shared inputs alone (p_e = 0.2, ρ_ee = q_e = 0). The green line was obtained with disparity in the “within” and “between” correlations (${\rho }_{\text{e}}^b=0.05$ and ${\rho }_{\text{e}}^w=0.1$) using the model in Figure 3B. (B) The correlations coefficient, ρ_EE, between the pooled inputs as a function of the within and between correlations (${\rho }_{\text{e}}^b$ and ${\rho }_{\text{e}}^w$) for n_e = 50. Note that the pooled correlation is relatively constant along lines through the origin. Thus, changing ${\rho }_{\text{e}}^b$ and ${\rho }_{\text{e}}^w$ by the same proportion does not affect the pooled correlation.

Under analogous homogeneity assumptions for the inhibitory pools, the correlation, ${\text{ρ}}_{{\text{I}}_1{\text{I}}_2},$ between the pooled inhibitory inputs is given by an equation identical to Eq. (10), and the correlation between the pooled excitatory and inhibitory inputs is given by

$${\rho }_{{\text{E}}_1{\text{I}}_2}={\rho }_{{\text{I}}_1{\text{E}}_2}=\frac{{\rho }_{\text{ei}}}{\sqrt{\left({\rho }_{\text{ee}}+\frac{1}{n_e}\left(1-{\rho }_{ee}+q_e\right)\right)\left({\rho }_{\text{ii}}+\frac{1}{n_{\text{i}}}\left(1-{\rho }_{\text{ii}}+q_{\text{i}}\right)\right)}}.$$ (11)

Interestingly, since $\left|{\text{ρ}}_{{\text{E}}_1{\text{I}}_2}\right|\le 1,$ pairwise excitatory– inhibitory correlations obey the bound $\left|{\text{ρ}}_{\text{ei}}\right|\le \sqrt{{\text{ρ}}_{\text{ee}}{\text{ρ}}_{\text{ii}}}+\mathcal{O}(1/\sqrt{n_{\text{e}}{n}_{\text{i}}}).$ Combining this inequality with Eq. (10) and the analogous equation for ${\text{ρ}}_{{\text{I}}_1{\text{I}}_2},$ it follows that $\left|{\text{ρ}}_{{\text{E}}_1{\text{I}}_2}\right|\le \sqrt{{\text{ρ}}_{{\text{E}}_1{\text{E}}_2}{\text{ρ}}_{{\text{I}}_1{\text{I}}_2}}+\mathcal{O}\left(1/\sqrt{n_{\text{e}}{n}_{\text{i}}}\right)$ for homogeneous populations. These are a result of the non-negative definiteness of covariance matrices.

Heterogeneity and the effects of spatially dependent correlations

We next discuss how heterogeneity can dampen the amplification of correlations due to pooling. In the absence of any homogeneity assumptions on the excitatory input population (see the population model in the Materials and Methods), Eq. (3) gives the pooled excitatory signals, ${\text{ρ}}_{{\text{E}}_1{\text{E}}_2}={\overline{\text{ρ}}}_{{\text{e}}_1{\text{e}}_2}/\sqrt{{\overline{\text{ρ}}}_{{\text{e}}_1{\text{e}}_2}}+\mathcal{O}\left(1/\sqrt{n_{{\text{e}}_1}{n}_{{\text{e}}_2}}\right).$ The term ${\overline{\text{ρ}}}_{{\text{e}}_1{\text{e}}_2}$ is a weighted average of the correlation coefficients between the two excitatory populations, and ${\overline{\text{ρ}}}_{{\text{e}}_1{\text{e}}_1}$ and ${\overline{\text{ρ}}}_{{\text{e}}_2{\text{e}}_2}$ are weighted averages of the correlations within each excitatory input population.

To illuminate this result, we assume symmetry between the populations: Let $n_{{\text{e}}_k}=n_{\text{e}}$ and ${\text{σ}}_{{\text{e}}_k^j}={\sigma }_{\text{e}}$ for k = 1,2 and j = 1,2,…,n_e, and assume ${\overline{\text{ρ}}}_{{\text{e}}_1{\text{e}}_1}={\overline{\text{ρ}}}_{{\text{e}}_2{\text{e}}_2}.$ The average “within” and “between” correlations, are ${\text{ρ}}_{\text{ee}}^{\text{w}}={\overline{\text{ρ}}}_{{\text{e}}_1{\text{e}}_1}={\overline{\text{ρ}}}_{{\text{e}}_2{\text{e}}_2}$ and ${\text{ρ}}_{\text{ee}}^{\text{b}}={\overline{\text{ρ}}}_{{\text{e}}_1{\text{e}}_2}$ respectively (see Figure 3B). Under these assumptions, Eq. (5) can be applied to obtain (See also Bedenbaugh and Gerstein, 1997)

$${\rho }_{{\text{E}}_1{\text{E}}_2}=\frac{{\rho }_{\text{ee}}^{\text{b}}}{{\rho }_{\text{ee}}^{\text{w}}+\frac{1}{n_{\text{e}}}\left(1-{\rho }_{\text{ee}}^{\text{w}}\right)}=\frac{{\rho }_{\text{ee}}^{\text{b}}}{{\rho }_{\text{ee}}^{\text{w}}}+\mathcal{O}\left(\frac{1}{n_{\text{e}}}\right)$$ (12)

which is plotted in Figure 4A (green line) and Figure 4B. For large n_e, the correlation between the pooled signals is the ratio of “between” and “within” correlations.

This observation has implications for a situation ubiquitous in the cortex. A neuron is likely to receive afferents from cells that are physically close. The activity of nearby cells may be more strongly correlated than the activity of more distant cells (Chen et al., 2006; Smith and Kohn, 2008). We therefore expect that pairwise correlations within each input pool are on average larger than correlations between two input pools, that is, ${\text{ρ}}_{\text{ee}}^{\text{w}}>{\text{ρ}}_{\text{ee}}^{\text{b}}.$ This reduces the correlation between the inputs, regardless of the input population size.

An increase in correlations in the presynaptic pool can also decorrelate the pooled signals. If correlations within each input pool increase by a greater amount than correlations between the two pools, then the variance in the input to each cell will increased by a larger amount than the covariance between the inputs. As a consequence the correlations between the pooled inputs will be reduced. Modulations in correlation have been observed as a consequence of attention in V4 (Cohen and Maunsell, 2009; Mitchell et al., 2009; but apparently not in V1, Roelfsema et al., 2004). Such changes may be, in part, a consequence of small changes in “within” correlations between neurons in V1.

Equation 12 implies that correlations between large populations cannot be significantly larger than the correlations within each population. Since $\left|{\text{ρ}}_{{\text{E}}_1{\text{E}}_2}\right|\le 1,$ it follows that $\left|{\text{ρ}}_{\text{ee}}^{\text{b}}\right|\le \left|{\text{ρ}}_{\text{ee}}^{\text{w}}\right|+\mathcal{O}(1/n_{\text{e}}).$

The correlation, ${\text{ρ}}_{{\text{I}}_1{\text{I}}_2},$ between the pooled inhibitory inputs is given by an identical equation to Eq. (12) and the correlation between the pooled excitatory and inhibitory inputs is given by

$$\begin{array}{c}{\rho }_{{\text{E}}_1{\text{I}}_2}={\rho }_{{\text{I}}_1{\text{E}}_2}=\frac{{\rho }_{\text{ei}}^{\text{b}}}{\sqrt{\left({\rho }_{\text{ee}}^{\text{w}}+\frac{1}{n_e}\left(1-{\rho }_{\text{ee}}^{\text{w}}\right)\right)\left({\rho }_{\text{ii}}^{\text{w}}+\frac{1}{n_{\text{i}}}\left(1-{\rho }_{\text{ii}}^{\text{w}}\right)\right)}}\\ =\frac{{\rho }_{\text{ei}}^{\text{b}}}{\sqrt{{\rho }_{\text{ee}}^{\text{w}}{\rho }_{\text{ii}}^{\text{w}}}}+\mathcal{O}\left(\frac{\text{1}}{\sqrt{n_{\text{e}}{n}_{\text{i}}}}\right).\end{array}$$

Correlations between the free membrane potentials

We now look at the correlation between the free membrane potentials of two downstream neurons. The free membrane potentials are obtained by assuming an absence of threshold or spiking activity. For simplicity we assume symmetry in the statistics of the inputs to the postsynaptic cells: ${\text{σ}}_{{\text{E}}_k}={\sigma }_{\text{E}},$${\text{σ}}_{{\text{I}}_k}={\sigma }_{\text{I}},$${\text{ρ}}_{{\text{E}}_1{\text{I}}_2}={\text{ρ}}_{{\text{E}}_2{\text{I}}_1},$${\text{ρ}}_{{\text{E}}_1{\text{I}}_1}={\text{ρ}}_{{\text{E}}_2{\text{I}}_2},$${\text{ρ}}_{{\text{E}}_1{\text{E}}_1}={\text{ρ}}_{{\text{E}}_2{\text{E}}_2}$ and ${\text{ρ}}_{{\text{I}}_1{\text{I}}_1}={\text{ρ}}_{{\text{I}}_2{\text{I}}_2}.$ The analysis is similar in the asymmetric case.

In the Section “Materials and Methods”, we derive a linear approximation of the free membrane potentials,

$$V_k(t)=\left(J_k\ast K\right)(t)+V_L,$$

where $J_k(t)=-(g_{{\text{E}}_k}(t)(V_L-V_{\text{E}})+g_{{\text{I}}_k}(t)(V_L-V_{\text{I}}))/C_m$ are the total input currents and $K(t)=\Theta (t){\text{e}}^{-t/{\tau }_{\text{eff}}}$ for k = 1,2. Under this approximation, the correlation, ${\text{ρ}}_{V_1{V}_2},$ between the membrane potentials is equal to the correlation, ${\text{ρ}}_{\text{in}}={\text{ρ}}_{J_1{J}_2},$ between the total input currents and can be written as a weighted average of the pooled excitatory and inhibitory spike train correlations (see Appendix),

$${\rho }_{V_1{V}_2}\approx {\rho }_{\text{in}}=\frac{W_{\text{E}}^2{\rho }_{{\text{E}}_1{\text{E}}_2}+W_{\text{I}}^2{\rho }_{{\text{I}}_1{\text{I}}_2}-2W_{\text{E}}{W}_{\text{I}}{\rho }_{{\text{E}}_1{\text{I}}_2}}{W_{\text{E}}^2+W_{\text{I}}^2-2W_{\text{E}}{W}_{\text{I}}{\rho }_{{\text{E}}_1{\text{I}}_1}}$$ (13)

where ${\text{ρ}}_{{\text{E}}_1{\text{E}}_2},{\text{ρ}}_{{\text{E}}_1{\text{I}}_2},{\text{ρ}}_{{\text{I}}_1{\text{I}}_2}$ and ${\text{ρ}}_{{\text{E}}_1{\text{I}}_1}$ are derived above, and W_E = ℰ|V_E − V_L|σ_E and W_I = ℐ|V_I − V_L|σ_I are weights for the excitatory and inhibitory contributions to the correlation. In Figure 5, we compare this approximation with simulations.

Figure 5.

The effects of pooling on correlations between postsynaptic membrane potentials. Results of the linear approximation (solid, dotted, and dashed lines) match simulations (points). For the solid blue line, ρ_ee = ρ_ii = 0.05, and ρ_ei = p_e = p_i = q_e = q_i = 0. The total number of excitatory and inhibitory inputs to each cell was n = n_e + q_en_e, and n_i + q_in_i respectively. Here n_i = n_e/3, with other parameters given in the Section “Materials and Methods.” The dotted blue line was obtained by including independent inputs, q_e = q_i = 1. The pink line was obtained by decreasing input correlations to ρ_ee = ρ_ii = 0.005. The solid green line was obtained by including excitatory–inhibitory correlations, ρ_ei = 0.05, so that total input correlations canceled. The dashed tan line was obtained by setting ρ_ee = ρ_ii = ρ_ei = q_e = q_i = 0 and p_e = p_i = 0.2 so that correlations are due to input overlap alone. In all cases, $ℰ=\frac{590}{n}\text{nS}·\text{\hspace{0.17em}ms,}$ and ℐ = 4ℰ. Standard errors are smaller than twice the radii of the points.

The correlation between the membrane potentials has positive contributions from the correlation between the excitatory inputs $({\text{ρ}}_{{\text{E}}_1{\text{E}}_2}),$ and between the inhibitory inputs $({\text{ρ}}_{{\text{I}}_1{\text{I}}_2}).$ Contributions coming from excitatory–inhibitory correlations (${\text{ρ}}_{{\text{E}}_1{\text{I}}_2}$ and ${\text{ρ}}_{{\text{E}}_2{\text{I}}_1}$) are negative, and can thus decorrelate the activity of downstream cells. This “cancellation” of correlations is observed in Figures 1D and 5, and can lead to asynchrony in recurrent networks (Hertz, 2010; Renart et al., 2010).

Implications for synchronization in feedforward chains

Feedforward chains, like that depicted in Figure 6A, have been studied extensively (Diesmann et al., 1999; van Rossum et al., 2002; Litvak et al., 2003; Reyes, 2003; Tetzlaff et al., 2003; Câteau and Reyes, 2006; Doiron et al., 2006; Kumar et al., 2008). In such networks, cells in a layer necessarily share some of their inputs, leading to correlations in their spiking activity (Shadlen and Newsome, 1998). Frequently, spiking in deeper layers is highly synchronous (Reyes, 2003; Tetzlaff et al., 2003). However, in the presence of background noise, correlations can remain negligible (van Rossum et al., 2002; Vogels and Abbott, 2005).

Figure 6.

The development of synchrony in a feedforward chain can be understood using a model dynamical system (Tetzlaff et al., 2003). (A) Schematic diagram of the network. Each layer consists of N_e excitatory and N_i inhibitory cells. Each cell in layer k receives precisely n_e excitatory and n_i inhibitory, randomly selected inputs from layer k − 1. (B) Stages of processing in the feedforward network. Inputs from layer k − 1 are pooled with overlap, and drive the cells in layer k. (C) The correlation transfer map described by the pooling function, P(ρ) (blue dotted line), is composed with the decorrelating transfer function, S(ρ) = ρ² (red dotted line), to obtain the mapping, T = S◦P (solid blue line). Cobwebs show the development of correlations in the discrete dynamical system defined by ${\text{ρ}}_{k+1}^{\text{out}}=T({\text{ρ}}_k^{\text{out}})$ with ρ₀ = 0. Nearly perfect correlations develop by the fifth layer. The identity is shown as a dashed line. (D) Closer to balance (β≈1), the correlating effects of pooling are weakened, and the model develops a stable fixed point close to ρ = 0. However, cells may no longer decorrelate their inputs in the balanced regime, and fluctuations in the input statistics due to random connectivity can destabilize the fixed point and lead to synchrony. The shaded region in the inset represents the region two standard deviations away from the mean (blue line) when randomness in the overlap is taken into account (see Appendix). The standard deviations were calculated using Monte Carlo simulations. In C and D, N_e = 12000 and n_e = 600. In C, N_i = 8000 and n_i = 400. In D, N_i = 10500 and n_i = 525 to obtain approximate balance (β = 600/525). Filled black circles represent stable fixed points and open black circles represent unstable fixed points.

Feedforward chains amplify correlations as follows: When inputs to the network are independent, small correlations are introduced in the second layer by overlapping inputs. The inputs to each subsequent layer are pooled from the previous layer. The amplification of correlations by pooling is the primary mechanism for the development of synchrony (Compare solid and dotted blue lines in Figure 4A). Overlapping inputs serve primarily to “seed” synchrony in early layers. The internal dynamics of the neurons and background noise can decorrelate the output of a layer, and compete with the correlation amplification due to pooling.

We develop this explanation by considering a feedforward network with each layer containing N_e excitatory and N_i inhibitory cells. Each cell in layer k + 1 receives n_e excitatory and n_i inhibitory inputs selected randomly from layer k. For simplicity we assume that all excitatory and inhibitory cells are dynamically identical and ℰ|V_E − V_L| = ℐ|V_E − V_L|. Spike trains driving the first layer are statistically homogeneous with pairwise correlations ρ₀.

To explain the development of correlations, we consider a simplified model of correlation propagation (See also Renart et al., 2010 for a recurrent version). In the model, any two cells in a layer share the expected proportion p_e = n_e/N_e of their excitatory inputs and p_i = n_i/N_i of their inhibitory inputs (the expected proportions are taken with respect to random connectivity). We also assume that inputs are statistically identical across a layer.

For a pair of cells in layer k ≥ 1, let ${\text{ρ}}_k^{\text{in}}$ and ${\text{ρ}}_k^{\text{out}}$ represent the correlation coefficient between the total input currents and output spike trains respectively. The outputs from layer k are pooled (with overlap) to obtain the inputs to layer k + 1. Using the results developed above, ${\text{ρ}}_{\text{in}}^1=P({\text{ρ}}_0)$ and ${\text{ρ}}_{k+1}^{\text{in}}=P({\text{ρ}}_k^{\text{out}}),$ for k ≥ 1, where (see Appendix and Tetzlaff et al., 2003 for a similar derivation)

$$P(\rho )=\frac{\rho {(\beta -1)}^2+\frac{1}{n_{\text{i}}}(1-\rho )\left(\beta p_{\text{e}}+p_{\text{i}}\right)}{\rho {(\beta -1)}^2+\frac{1}{n_{\text{i}}}(1-\rho )(1+\beta )}.$$ (14)

Here β measures the balance between excitation and inhibition (see Materials and Methods). From our assumptions, β = n_e/n_i. With imbalance (β ≠ 1) and a large number of cells in a layer, pooling amplifies small correlations, P(ρ) > ρ, as discussed earlier.

To complete the description of correlation transfer from layer to layer, we relate the correlations between inputs to a pair of cells, ${\text{ρ}}_k^{\text{in}},$ to correlations in their output spike trains, ${\text{ρ}}_k^{\text{out}}.$ We assume that there is a transfer function, S, so that ${\text{ρ}}_k^{\text{out}}=S({\text{ρ}}_k^{\text{in}})$ at each layer k. We additionally assume that S(0) = 0 and S(1) = 1, that is uncorrelated (perfectly correlated) inputs result in uncorrelated (perfectly correlated) outputs. We also assume that the cells are decorrelating, |ρ| > |S(ρ)| > 0 for ρ ≠ 0,1 (Shea-Brown et al., 2008). This is an idealized model of correlation transfer, as output correlations depend on cell dynamics and higher order statistics of the inputs (Moreno-Bote and Parga, 2006; de la Rocha et al., 2007; Barreiro et al., 2009; Ostojić et al., 2009).

Correlations between the spiking activity of cells in layers k + 1 are related to correlations in layer k by the layer-to-layer transfer function, T = S ○P. The development of correlations across layers is modeled by the dynamical system, ${\text{ρ}}_{k+1}^{\text{out}}=T({\text{ρ}}_k^{\text{out}}),$ with ${\text{ρ}}_1^{\text{out}}=S({\text{ρ}}_0).$

When the network is not balanced (β ≠ 1), pooling amplifies correlations at each layer and the activity between cells in deeper layers can become highly correlated (see Figure 6C). The output of the first layer is uncorrelated if the individual inputs are independent (ρ₀ = 0). In this case all of the correlations between the total inputs to the second layer come from shared inputs,

$${\rho }_2^{\text{in}}=P(0)=\frac{n_{\text{e}}{p}_{\text{e}}+n_{\text{i}}{p}_{\text{i}}}{n_{\text{e}}+n_{\text{i}}}.$$

These correlations are then reduced by the second layer of cells, ${\text{ρ}}_2^{\text{out}}=S({\text{ρ}}_2^{\text{in}})=T(0)>0,$ and subsequently amplified by pooling and input sharing before being received by layer 3, ${\rho }_{\text{3}}^{\text{in}}=P({\rho }_{\text{2}}^{\text{out}}).$ This process continues in subsequent layers. If the correlating effects of pooling and input sharing dominate the decorrelating effects of internal cell dynamics, correlations will increase from layer to layer (see Figure 6C).

When ρ₀ = 0, overlapping inputs increase the input correlation to layer 2, but have a negligible effect on the mapping once correlations have developed since the effects of pooling dominate [see Eq. (14) and the dashed blue line in Figure 4A which shows that the effects of input overlaps are small when n_e is large, ρ > 0 and β ≠ 1]. Therefore, shared inputs seed correlated activity at the first layer, and pooling drives the development of larger correlations. When ρ₀ = 0, we cannot expect large correlations before layer 3, but when ρ₀ > 0 large correlations can develop by layer 2.

To verify this conclusion, we constructed a two-layer feedforward network with no overlap between inputs (P_e = P_i = 0). In Figure 7A, the inputs to layer 1 were independent (ρ₀ = 0), and the firing of cells in layer 2 was uncorrelated. In Figure 7B, we introduced small correlations (ρ₀ = 0.05) between inputs to layer 1. These correlations were amplified by pooling so that strong synchrony is observed between cells in layer 2. We compared these results with a standard feedforward network with overlap in cell inputs (Figure 7C, where P_e = P_i = 0.05). Inputs to layer 1 were independent (ρ₀ = 0), and hence outputs from layer 1 uncorrelated. Dependencies between inputs to layer 2 were weak and due to overlap alone, ${\rho }_{\text{2}}^{\text{in}}=P(0)=\mathrm{0.05.}$ Cells in layer 3 received pooled inputs from layer 2, and their output was highly correlated.

These results predict that correlations between spike trains develop in deeper layers, but they do not not directly address the timescale of the correlated behavior. In simulations, spiking becomes tightly synchronized in deeper layers (see for instance Litvak et al., 2003; Reyes, 2003; and Figure 7). This can be understood using results in Maršálek et al. (1997) and Diesmann et al. (1999) where it is shown that the response of cells to volleys of spikes is tighter than the volley itself. The firing of individual cells in the network becomes bursty in deeper layers and large correlations are manifested in tightly synchronized spiking events. Alternatively, one can predict the emergence of synchrony by observing that pooling increases correlations over finite time windows (see next section and Appendix) and therefore the analysis developed above can be adapted to correlations over small windows.

Balanced feedforward networks

In the simplified feedforward model above, when excitation balances inhibition, that is β ≈ 1, correlations between the pooled inputs to a layer are due to overlap alone, ${\rho }_{\text{k}}^{\text{in}}=P({\text{ρ}}_{k-1}^{\text{out}})\approx (p_{\text{e}}+p_{\text{i}})/2$ for all k. The correlating effects of this map are weak, and this would seem to imply that cells in balanced feedforward chains remain asynchronous. Indeed, our model of correlation propagation displays a stable fixed point at low values of ρ when β ≈ 1 (see Figure 6D). However, in practice, synchrony is difficult to avoid without careful fine-tuning (Tetzlaff et al., 2003), and almost always develops in feedforward chains (Litvak et al., 2003). We provide some reasons for this discrepancy.

Our focus so far has been on correlations over infinitely large time windows (see Materials and Methods where we define ρ_xy). Even when the membrane potentials are nearly uncorrelated over large time windows, differences between the excitatory and inhibitory synaptic time constants can cause larger correlations over smaller time windows (Renart et al., 2010). This can, in turn, lead to significant correlations between the output spike trains. We discuss this effect further in the Appendix and give an example in Figure 8. In this example, the correlations between the membrane potentials over long windows are nearly zero due to cancellation (see Figure 8A where ρ_VV = 0.0174 ± 0.0024 s.e. with threshold present), but positive over shorter timescales. The cross- covariance function between the output spike trains is primarily positive, yielding significant spike train correlations (ρ_spikes = 0.1570 ± 0.0033 s.e.). Therefore, the assumption that pairs of cells decorrelate their inputs may not be valid in the balanced case.

Figure 8.

Cross-covariance functions between membrane potentials and output spike trains. (A) The cross-covariance function between membrane potentials, scaled so that its maximum is 1. The linear approximation in Eq. (16) (blue, shaded) agrees with simulations of the full conductance-based model (black dashed line). Differences between simulations with and without threshold are too small to be observable (8000 simulations 10s each; simulations with and without threshold are shown). Parameters are as in Figure 1D. The cells are balanced with ρ_ee = ρ_ii = ρ_ei = 0.05 so that the correlation between the membrane potentials over long time windows is essentially zero (ρ_vv = 0.0085 ± 0.0024 s.e. unthresholded, and ρ_vv = 0.0174 ± 0.0024 s.e. thresholded). However, correlations over shorter time windows are positive as indicated by the central peak in the cross-covariance function. (B) The cross-covariance between the output spike trains is mostly positive. The correlation between the output spike trains was ρ_spikes = 0.1570 ± 0.0033 s.e. (500 simulations of 100s each with same parameters as in A.

Another source of discrepancies between the idealized model and simulations of feedforward networks are inhomogeneities, which become important when balance is exact. Note that Eq. (14) is an approximation obtained by ignoring fluctuations in connectivity from layer to layer. In a random network, inhomogeneities will be introduced by variability in input population overlaps. To fully describe the development of correlations in a feedforward network, it is necessary to include such fluctuations in a model of correlation propagation. The asynchronous fixed point that appears in the balanced case has a small basin of attraction and fluctuations induced by input inhomogeneities could destroy its stability (see Figure 6D). Other sources of heterogeneity can further destabilize the asynchronous state (see Appendix).

It has been shown that asynchronous states can be stabilized through the decorrelating effects of background noise (van Rossum et al., 2002; Vogels and Abbott, 2005). To emulate these effects, a third transfer function, N, can be added to our model. The correlation transfer map then becomes T(ρ) = S◦N◦P(ρ). Sufficiently strong background noise can increase decorrelation from input to output of a layer, and stabilize the asynchronous fixed point.

Discussion

We have illustrated how pooling and shared inputs can impact correlations between the inputs and free membrane voltages of postsynaptic cells in a feedforward setting. The increase in correlation due to pooling was discussed in a simpler setting in (Bedenbaugh and Gerstein, 1997; Super and Roelfsema, 2005; Chen et al., 2006; Stark et al., 2008), and similar ideas were also developed for the variance alone in (Salinas and Sejnowski, 2000; Moreno-Bote et al., 2008). The saturation of the signal-to-noise ratio with increasing population size observed in (Zohary et al., 1994) has a similar origin. Our aim was to present a unified discussion of these results, with several generalizations.

Other mechanisms, such as recurrent connectivity between cells receiving the inputs, can modulate correlated activity (Schneider et al., 2006; Ostojić et al., 2009). Importantly, the cancellation of correlations may be a dynamic phenomenon in recurrent networks, as observed in (Hertz, 2010; Renart et al., 2010). On the other hand, neurons may become entrained to network oscillations, resulting in more synchronous firing (Womelsdorf et al., 2007). A full understanding of the statistics of population activity in neuronal networks will require an understanding of how these mechanisms interact to shape the spatiotemporal properties of the neural response.

The results we presented relied on the assumption of linearity at the different levels of input integration. These assumptions can be expected to hold at least approximately. For instance, there is evidence that membrane conductances are tuned to produce a linear response in the subthreshold regime (Morel and Levy, 2009). The assumptions we make are likely to break down at the level of single dendrites where nonlinear effects may be much stronger (Johnston and Narayanan, 2008). The effects of correlated inputs to a single dendritic branch deserve further theoretical study (Gasparini and Magee, 2006; Li and Ascoli, 2006).

We demonstrated that the structure of correlations in a population may be difficult to infer from pooled activity. For instance, a change in pairwise correlations between individual cells in two populations causes a much smaller change in the correlation between the pooled signals. With a large number of inputs, the change in correlations between the pooled signals might not be detectable even when the change in the pairwise correlations is significant.

While we discussed the growth of second order correlations only, higher order correlations also saturate with increasing population size. For example, in a 3-variable generalization of the homogeneous model from Figure 3A, it can be shown that ${\text{ρ}}_{{\text{E}}_1{\text{E}}_2{\text{E}}_3}=1-\mathcal{O}(1/n_{\text{e}})$ where n_e is the size of each population and ${\text{ρ}}_{{\text{E}}_1{\text{E}}_2{\text{E}}_3}$ is the triple correlation coefficient (Stratonovich, 1963) between the pooled signals E₁, E₂, and E₃. The reason that higher order correlations also saturate follows from the generalization of the following observation at second order: Pooling amplifies correlations because the variance and covariance grow asymptotically with the same rate in n_e. In particular ${\sigma }_{\text{E}}^2$ and ${\text{γ}}_{{\text{E}}_1{\text{E}}_2}$ both behave asymptotically like $n_{\text{e}}^2{\text{ρ}}_{\text{ee}}{\sigma }_{\text{e}}^2+\mathcal{O}(n_{\text{e}}),$ and their ratio, ${\text{ρ}}_{{\text{E}}_1{\text{E}}_2}={\text{γ}}_{{\text{E}}_1{\text{E}}_2}/{\sigma }_{\text{E}}^2,$ approaches unity (Bedenbaugh and Gerstein, 1997; Salinas and Sejnowski, 2000; Moreno-Bote et al., 2008).

We concentrated on correlations over infinitely long time windows (see Materials and Methods where we define ρ_xy). However, pooling amplifies correlations over finite time windows in exactly the same way as correlations over large time windows. Due to the filtering properties of the cells, the timescale of correlations between downstream membrane potentials may not reflect that of the inputs. We discuss this further in the Appendix where the auto- and cross-covariance functions between the membrane potentials are derived.

To simplify the presentation, we have so far assumed stationary. However, since Eq. (2) applies to the Pearson correlation between any pooled data, all of the results on pooling can easily be extended to the non-stationary case. In the non-stationary setting, the cross- covariance function has the form R_xy(s, t) = cov (x(s), y(s + t)), but there is no natural generalization of the asymptotic statistics defined in Eq. (1).

Correlated neural activity has been observed in a variety of neural populations (Gawne and Richmond, 1993; Zohary et al., 1994; Vaadia et al., 1995), and has been implicated in the propagation and processing of information (Oram et al., 1998; Maynard et al., 1999; Romo et al., 2003; Tiesinga et al., 2004; Womelsdorf et al., 2007; Stark et al., 2008), and attention (Steinmetz et al., 2000; Mitchell et al., 2009). However, correlations can also introduce redundancy and decrease the efficiency with which networks of neurons represent information (Zohary et al., 1994; Gutnisky and Dragoi, 2008; Goard and Dan, 2009). Since the joint response of cells and recorded signals can reflect the activity of large neuronal populations, it will be important to understand the effects of pooling to understand the neural code (Chen et al., 2006).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Appendix

Derivation of EQ. (10)

Equation (10) can be derived from Eq. (2). However, we find that it is more easily derived directly. We will calculate the variance, ${\text{σ}}_{{\text{E}}_1}^{\text{2}}={\text{σ}}_{{\text{E}}_2}^{\text{2}},$ and covariance ${\text{γ}}_{{\text{E}}_1{\text{E}}_2}$ between the pooled signals.

The covariance is given by the sum of all pairwise covariances between the populations, ${\text{γ}}_{{\text{E}}_1{\text{E}}_2}={\Sigma }_{{\text{e}}_1\in {\text{E}}_1{\text{,e}}_{\text{2}}\in {\text{E}}_2}{\text{σ}}_{{\text{e}}_1}{\text{σ}}_{{\text{e}}_2}{\text{ρ}}_{{\text{e}}_1{\text{e}}_2}.$ Each cell receives $n_{\text{e}}+q_{\text{e}}{n}_{\text{e}}$ inputs so that there are (n_e + q_en_e)² terms that appear in this sum. However, the q_en_e “independent” inputs from each pool are uncorrelated with all other inputs and therefore don't contribute to the sum. Of the remaining $n_{\text{e}}^2$ pairs, n_ep_e are shared and therefore have correlation ${\text{ρ}}_{{\text{e}}_1{\text{e}}_2}=1.$ These shared processes therefore collectively contribute $n_{\text{e}}{p}_{\text{e}}{\sigma }_{\text{e}}^2$ to ${\text{γ}}_{{\text{E}}_1{\text{E}}_2}.$ The remaining $n_{\text{e}}^2-n_{\text{e}}{p}_{\text{e}}$ processes are correlated with coefficient ρ_ee and collectively contribute $(n_{\text{e}}^2-n_{\text{e}}{p}_{\text{e}}){\text{ρ}}_{\text{ee}}{\sigma }_{\text{e}}^2.$ The pooled covariance is thus

$${\gamma }_{{\text{E}}_1{\text{E}}_2}=\underset{\text{Correlated}}{\underbrace{(n_{\text{e}}^2-p_{\text{e}}{n}_{\text{e}}){\rho }_{\text{ee}}{\sigma }_{\text{e}}^2}}+\underset{\text{Shared}}{\underbrace{n_{\text{e}}{p}_{\text{e}}{\sigma }_{\text{E}}^2}}.$$

The variance is given by the sum of all pairwise covariances within a population, ${\text{σ}}_{{\text{E}}_1}^{\text{2}}={\Sigma }_{{\text{e}}_1\in {\text{E}}_1,{\text{e}}_2\in {\text{E}}_1}{\text{σ}}_{{\text{e}}_1}{\text{σ}}_{{\text{e}}_2}{\text{ρ}}_{{\text{e}}_1{\text{e}}_2}.$ As above, there are n_e + q_en_e neurons in the population, so that the sum has (n_e + q_en_e)² terms. Of these, n_e + q_en_e are “diagonal” terms (e₁ = e₂), each contributing ${\sigma }_{\text{e}}^2,$ for a total contribution of $(n_{\text{e}}+q_{\text{e}}{n}_{\text{e}}){\sigma }_{\text{e}}^2$ to ${\text{σ}}_{{\text{E}}_1}^2.$ The processes from the independent pool do not contribute any additional terms. This leaves n_e(n_e − 1) correlated pairs which each contribute ${\sigma }_{\text{e}}^2{\text{ρ}}_{\text{ee}}$ for a collective contribution of $n_{\text{e}}(n_{\text{e}}-1){\sigma }_{\text{e}}^2{\text{ρ}}_{\text{ee}},$ giving

$${\sigma }_{{\text{E}}_1}={\sigma }_{{\text{E}}_2}=\underset{\text{Correlated}}{\underbrace{n_{\text{e}}(n_{\text{e}}-1){\sigma }_{\text{e}}^2{\rho }_{\text{ee}}}}+\underset{\text{Diagonal}}{\underbrace{(n_{\text{e}}+n_{\text{e}}{q}_{\text{e}}){\sigma }_{\text{e}}^2}}.$$

Now, ${\text{ρ}}_{{\text{E}}_1{\text{E}}_2}={\text{γ}}_{{\text{E}}_1{\text{E}}_2}/{\text{σ}}_{{\text{E}}_1}$ can be simplified to give Eq. (10). Equations for ${\text{ρ}}_{{\text{I}}_1{\text{I}}_2}$ and ${\text{ρ}}_{{\text{E}}_1{\text{I}}_2}={\text{ρ}}_{{\text{I}}_1{\text{E}}_2}$ can be derived identically.

Finite-time correlations and cross-covariances

Throughout the text, we concentrated on correlations over large time windows. However, the effects of pooling described by Eq. (2) apply to the correlation, ρ_xy(t), between spike counts over any time window of size t, defined by ${\text{ρ}}_{xy}\left(t\right)=\mathrm{cov}\left(N_x\left(t\right),N_y\left(t\right)\right)/\sqrt{\mathrm{var}\left(N_x\left(t\right)\right)\mathrm{var}\left(N_y\left(t\right)\right)}$ where $N_x(t)={\int }_0^{t}x(s)ds$ is the spike count over [0,t] for the spike train x(t). The equation also applies to the instantaneous correlation at time t, defined by $R_{xy}\left(t\right)=C_{xy}\left(t\right)/\sqrt{C_{xx}\left(t\right)C_{yy}\left(t\right)}.$ Thus pooling increases correlations over all timescales equally.

However, the cell filters the pooled inputs to obtain the membrane potentials and, as a result, the correlations between membrane potentials is “spread out” in time (Tetzlaff et al., 2008). To quantify this effect, we derive an approximation to the auto- and cross-covariance functions between the membrane potentials.

The pooled input spike trains are obtained from from a weighted sum of the individual excitatory and inhibitory spike trains (see Materials and Methods). As a result cross-covariance functions between the pooled spike trains are just sums of the individual cross-covariance functions, $C_{XY}(t)={\Sigma }_{x\in X,y\in Y}{C}_{xy}(t)$ for X, Y = E₁, E₂, I₁, I₂ and x, y = e, i accordingly. Thus only the magnitude of the cross- covariance functions is affected by pooling. The change in magnitude is quadratic in n_e or n_i. This is consistent with the observation that pooling amplifies correlations equally over all timescales.

The conductances are obtained by convolving the total inputs with the synaptic filter kernels,

$$g_{{\text{E}}_k}(t)=({\text{E}}_k\ast {\alpha }_{\text{e}})(t),\text{\hspace{1em}and\hspace{1em}}{g}_{{\text{I}}_k}(t)=({\text{I}}_k\ast {\alpha }_i)(t);k=1,2.$$

The cross-covariance between the conductances can therefore be written as a convolution of the cross-covariance function between the input signals and the deterministic cross-covariance between the synaptic kernels (Tetzlaff et al., 2008). In particular,

$$C_{g_X{g}_Y}(t)=(C_{XY}\ast ({\alpha }_x\star {\alpha }_y))(t)$$ (15)

for X, Y = E₁, E₂, I₁, I₂ and x, y = e, i accordingly, where $({\alpha }_x\star {\alpha }_y)(t)={\int }_{-\infty }^{\infty }{\alpha }_x(s){\alpha }_y(t+s)\text{\hspace{0.17em}d}s$ is the deterministic cross- covariance between the synaptic filters, α_x and α_y. Note that total correlations remain unchanged by convolution of the input spike trains with the synaptic filters, since the integral of a convolution will be equal to the product of the integrals (Tetzlaff et al., 2008).

The total input currents, $J_K\left(t\right)=-\left(g_{E_k}\left(t\right)\left(V_L-V_I\right)\right)/C_m,$ obtained from the linearization of the conductance-based model described in the Section “Materials and Methods” are simply linear combinations of the individual conductances. The cross-covariance function between the input currents is therefore a linear combination of those between the conductances,

$$\begin{array}{l}{C}_{J_h{J}_k}(t)=|V_{\text{E}}-V_L{|}^2{C}_{g_{{\text{E}}_h}{g}_{{\text{E}}_k}}(t)+|V_{\text{I}}-V_L{|}^2{C}_{g_{{\text{I}}_h}{g}_{{\text{I}}_k}}(t)\\ -2|V_{\text{E}}-V_L||V_{\text{I}}-V_L|C_{g_{{\text{I}}_h}{g}_{{\text{E}}_k}}(t).\end{array}$$

Combining this result with Eq. (1), yields the correlation, ${\text{ρ}}_{\text{in}}={\text{ρ}}_{J_1{J}_2},$ between the total input currents given in Eq. (13).

Using the solution of the linearized equations described in the Section “Materials and Methods”, we obtain a linear approximation to the cross-covariance functions,

$$C_{V_h{V}_k}(t)\approx \frac{1}{2{\tau }_{\text{eff}}}(C_{J_h{J}_k}\ast (K\star K))(t)$$ (16)

for h,k = 1,2 where $(K\star K)(t)=\frac{{\tau }_{\text{eff}}}{2}{e}^{-\left|t\right|/{\tau }_{\text{eff}}}$ is the cross-covariance between the linear kernel, K, and itself. The convolution with (K⋆K)(t) scales the area of both the auto- and cross-covariance functions by a factor of ${\tau }_{\text{eff}}^2,$ and therefore leaves the ratio of the areas, ${\text{ρ}}_{V_1{V}_2}$ unchanged. Thus, the linear approximation predicts that ${\text{ρ}}_{V_1{V}_2}\approx {\text{ρ}}_{\text{in}}.$

When the total inputs are strong, τ_eff is small and we can simplify Eq. (16) by approximating (K⋆K)(t) with a delta function with mass ${\tau }_{\text{eff}}^2$ so that $C_{V_1{V}_2}(t)\approx {\tau }_{\text{eff}}{C}_{J_1{J}_2}(t)/2$ and similarly for $C_{V_1{V}_1}(t).$ This approximation is valid when the synaptic time constants are significantly larger than τ_eff, which is likely to hold in high conductance states. We compare this approximation to cross-covariance functions obtained from simulations in Figure 8.

In all examples considered, the cross-covariance functions have exponentially decaying tails. We define the correlation time constant, ${\tau }_{xy}=\underset{t\to \infty }{\lim }-t/\ln (C_{xy}(t)),$ as a measure of the decay rate of the exponential tail. If t ≫ τ_xy, then x(s) and y(s + t) can be regarded as approximately uncorrelated and ${\text{γ}}_{xy}\approx {\int }_{-t}^t(t-|s|/t)C_{xy}(s)\text{\hspace{0.17em}d}s$ (Stratonovich, 1963).

The time constant of a convolution between two exponentially decaying functions is just the maximum time constant of the two functions. Thus, from the results above, the correlation time constant between the membrane potentials is the maximum of the correlation time constants between the inputs, the synaptic time constants, and the effective membrane time constant ${\tau }_{V_1{V}_2}=\max \{{\tau }_{{\text{E}}_1{\text{E}}_2},{\tau }_{{\text{I}}_1{\text{I}}_2},{\tau }_{{\text{E}}_1{\text{I}}_2},{\tau }_{{\text{E}}_2{\text{I}}_1},{\tau }_{\text{e}},{\tau }_{\text{i}},{\tau }_{\text{eff}}\}$ where ${\tau }_{{\text{E}}_1{\text{E}}_2},$${\tau }_{{\text{I}}_1{\text{I}}_2},$${\tau }_{{\text{E}}_1{\text{I}}_2},$ and ${\tau }_{{\text{E}}_2{\text{I}}_1}$ are the time constants of the input spike trains and τ_e and τ_i are synaptic time constants. Thus the cross-covariances functions between the membrane potentials are generally broader than the cross-covariance functions between the spike train inputs.

Derivation of EQ. (14)

Consider a feedforward network where each layer consists of N_e excitatory cells and N_i inhibitory cells; each cell in layer k receives n_e excitatory and n_i inhibitory inputs from layer (k − 1), and these connections are chosen randomly and independently across neurons in layer k. Then the degree of overlap in the excitatory and inhibitory inputs to a pair of cells in layer k is a random variable. Following the derivation in Derivation of Eq. (10) in Appendix,

$${\rho }_{{\text{E}}_1{\text{E}}_2}=\frac{{\gamma }_{{\text{E}}_1{\text{E}}_2}}{{\sigma }_{\text{E}}^2}=\frac{s_{\text{e}}{\sigma }_{\text{e}}^2+(n_{\text{e}}^2-s_{\text{e}}){\rho }_{\text{ee}}{\sigma }_{\text{e}}^2}{n_{\text{e}}{\sigma }_{\text{e}}^2+(n_{\text{e}}^2-n_{\text{e}}){\rho }_{\text{ee}}{\sigma }_{\text{e}}^2},$$

where s_e denotes the number of common excitatory inputs between the two cells. To understand the origin of s_e, suppose the n_e excitatory inputs to cell 1 have been selected. Then the selection of the n_e excitatory inputs to cell 2 involves choosing, without replacement, from two pools: the first, of size n_e, projects to cell 1, and the second, of size (N_e − n_e), does not. Therefore, s_e is follows a hyper-geometric distribution with parameters (N_e, n_e, n_e), and has mean $n_{\text{e}}^2/N_{\text{e}}=n_{\text{e}}{p}_{\text{e}}.$ In addition, this random variable is independently selected amongst each pair in layer k. Using the mean value of s_e, we obtain Eq. (10).

For simplicity, we assume that ℰ |V_E − V_L| = ℐ |V_I − V_L|, so that β = n_i/n_e. If we assume that the statistics in the (k − 1)^st layer are uniform across all cells and cell types (i.e., $\text{ρ}={\text{ρ}}_{\text{ee}}={\text{ρ}}_{\text{ii}}={\text{ρ}}_{\text{ei}}={\text{ρ}}_{k-1}^{\text{out}}$ and σ_e = σ_i) then by substituting Eq. (10) and the equivalent forms for ρ_II, ρ_EI in to Eq. (13), we may write the input correlations to the k^th layer as

$${\rho }_{\text{in}}=\frac{|V_{\text{E}}-V_L{|}^2{\gamma }_{{\text{E}}_1{\text{E}}_2}+|V_{\text{I}}-V_L{|}^2{\gamma }_{{\text{I}}_1{\text{I}}_2}-2|V_{\text{E}}-V_L||V_{\text{I}}-V_L|{\gamma }_{E_1{I}_2}}{|V_{\text{E}}-V_L{|}^2{\sigma }_{\text{E}}^2+|V_{\text{I}}-V_L{|}^2{\sigma }_{\text{I}}^2-2|V_{\text{E}}-V_L||V_{\text{I}}-V_L|{\gamma }_{{\text{E}}_1{\text{I}}_1}}.$$

Substituting the values of the covariances and variances, and dividing the numerator and denominator by (ℰ|V_E − V_L|σ_e)², we get

$${\rho }_{\text{in}}=\frac{s_{\text{e}}+(n_{\text{e}}^2-s_{\text{e}})\rho +s_{\text{i}}+(n_{\text{i}}^2-s_{\text{i}})\rho -2n_{\text{e}}{n}_{\text{i}}\rho }{n_{\text{e}}+(n_{\text{e}}^2-n_{\text{e}})\rho +n_{\text{i}}+(n_{\text{i}}^2-n_{\text{i}})\rho -2n_{\text{e}}{n}_{\text{i}}\rho }.$$

Rearranging terms and dividing numerator and denominator by $n_{\text{i}}^2,$ along with the substitution β = n_e/n_i, we have

$${\rho }_{\text{in}}=\frac{\rho {(1-\beta )}^2+\frac{1}{n_{\text{i}}^2}\left(s_{\text{e}}+s_{\text{i}}\right)(1-\rho )}{\rho {(1-\beta )}^2+\frac{1}{n_{\text{i}}^2}(n_{\text{e}}+n_{\text{i}})(1-\rho )}$$ (17)

This equation takes into account the variations in overlap due to finite size effects since s_e and s_i are random variables. Eq. (14) in the text represents the expected value P(ρ) = 〈ρ_in〉 which can be obtained by replacing the variables s_e and s_i in Eq. (17) with their respective means, 〈s_e〉 = n_ep_e and 〈s_i〉 = n_ip_i. The expectation above is taken over realizations of the random connectivity of the feedforward network.

To calculate the standard deviation for the inset in Figure 6D, we ran Monte Carlo simulations, drawing s_e and s_i from a hypergeometric distribution and calculating the resulting transfer, $S({\text{ρ}}_{\text{in}})={\text{ρ}}_{\text{in}}^{\text{2}}$ using Eq. (17). Note, however, that Eq. (17) and the inset in Figure 6D, do not account for all of the effects of randomness which may destabilize the balanced network. In deriving Eq. (17), we assumed that the statistics in the second layer were uniform. However, variations in the degree of overlap in one layer will cause inhomogeneities in the variances and rates at the next layer. In a feedforward setting, these inhomogeneities are compounded at each layer to destabilize the asynchronous fixed point.