Direction-selective circuits shape noise to ensure a precise population code

Summary

Neural responses are noisy, and circuit structure can correlate this noise across neurons. Theoretical studies show that noise correlations can have diverse effects on population coding, but these studies rarely explore stimulus dependence of noise correlations. Here, we show that noise correlations in responses of ON-OFF direction-selective retinal ganglion cells are strongly stimulus dependent and we uncover the circuit mechanisms producing this stimulus dependence. A population model based on these mechanistic studies shows that stimulus-dependent noise correlations improve the encoding of motion direction two-fold compared to independent noise. This work demonstrates a mechanism by which a neural circuit effectively shapes its signal and noise in concert, minimizing corruption of signal by noise. Finally, we generalize our findings beyond direction coding in the retina and show that stimulus-dependent correlations will generally enhance information coding in populations of diversely tuned neurons.

Introduction

Basic biophysical considerations mean that sensory signals are inevitably corrupted with noise. Divergence of these noisy signals to multiple downstream neurons will cause those neurons’ response to covary. The noise correlations that result from such common circuit mechanisms can have diverse effects on coding, ranging from redundant codes, in which groups of cells encode less information than would be predicted from studying the individual cells they contain, to synergistic codes, in which they encode more (Averbeck et al., 2006; Hu et al., 2014; Schneidman et al., 2003; Shamir, 2014; Zohary et al., 1994; Romo et al., 2003; Jeanne et al., 2013; Wilke and Eurich, 2001; Wu et al., 2004; Shamir and Sompolinsky, 2004). Understanding the impact of noise correlations on coding is essential for understanding the fidelity with which neural circuits can compute and direct behavior.

Observed noise correlations are diverse in magnitude and structure. In cortex, average noise correlations are often positive, small, and depend on similarities between the cells’ tuning to different stimuli (Ecker et al., 2014; Gawne and Richmond, 1993; Bair et al., 2001; Reich et al., 2001; Cohen and Kohn, 2011; Ecker et al., 2010; Shamir, 2014). The small amplitude of noise correlations has been attributed to circuits operating in a balanced state, in which correlated fluctuations in excitatory and inhibitory inputs cancel (Renart et al., 2010; Graupner and Reyes, 2013; Hansen et al., 2012). However, the balanced state does not always hold (Hansen et al., 2012; Cafaro and Rieke 2010), and noise correlations can be quite strong. Moreover, noise correlations can depend on neural firing rate (de la Rocha et al., 2007), and on the stimulus presented (Kohn and Smith, 2005; Cohen and Kohn, 2011; Lin et al., 2015). Because of these issues, the extent of correlations between cells and how those correlations are constrained by the synaptic input cells receive is unclear.

Theoretical work provides guidelines for how noise correlations can affect sensory coding: noise that mimics the signals being conveyed by the population will be deleterious to the population code, whereas noise with different statistical structure than the signal is relatively benign. Most theoretical work considers the case wherse correlations are constant across stimuli and across neural firing rates (e.g., Zohary et al. 1994; Abbott and Dayan, 1999; Abbott and Dayan, 2001; Panzeri et al. 1999; Oram et al. 1998; Shamir and Sompolinsky 2006; Averbeck et al. 2006; Shamir, 2014). Other work suggests that stimulus-dependence can alter the impact of correlations on sensory coding (Josic et al. 2009; Wu et al., 2004; Montani et al. 2007). The importance of this issue is highlighted by studies showing that correlations between cells can be strongly modulated by neural firing rates and stimuli (de la Rocha et al., 2007; Binder and Powers 2001; Franke et al 2015; Lampl et al., 1999; Samonds and Bonds, 2004; Granot-Atedgi et al., 2013; Ponce-Alvarez et al., 2013; Lin et al., 2015; see also Kohn and Smith 2005). Previous theoretical work, however, did not isolate the impact of stimulus-dependence of the correlations in neural populations from other factors such as the diversity of correlation coefficients across the population (Josic et al. 2009; Wu et al., 2004).

Thus, the extent, origins and coding impact of stimulus-dependent correlations remain unclear. This is largely because few experimental preparations permit direct investigation of the circuit mechanisms shaping signal and noise for physiologically relevant stimuli. A notable exception is the population of ON-OFF directionally-selective retinal ganglion cells (ooDS cells), in which the relevant stimulus space (direction of moving objects) is simple and well-described (Barlow and Levick, 1965; Oyster and Barlow, 1967). Knowledge of the stimulus parameters that these cells encode and the accessibility of the retina to mechanistic investigations allowed us to answer several general questions about the role of noise correlations in neural population codes: What is the structure (including stimulus-dependence) of correlated noise in a neural population? How are these correlations generated by the circuitry? What impact do correlations have on the fidelity of sensory encoding? Computational modeling allowed us to generalize our findings beyond the ooDS system. This reveals that correlation structures that would be harmful to the population code in the absence of stimulus dependence lead to significant improvements in coding accuracy in the presence of stimulus dependence.

Results

Our aim is to understand the impact of noise correlations in ooDS cell populations. First, we show that the direction of a moving stimulus modulates noise correlations between pairs of ooDS cells. Second, we provide a simple example for why stimulus dependence might change how correlations affect population codes. Third, we use intracellular recordings to uncover the circuit mechanisms underlying stimulus-dependent noise correlations. Using this mechanistic description, we develop a computational model to extrapolate beyond our paired recordings and demonstrate that the observed stimulus-dependent correlations improve direction encoding by the ooDS cell population. Fourth, we perform theoretical calculations that suggest that stimulus-dependent noise correlations of the form that are observed in the ooDS cell population will enhance population codes in other neural circuits.

Correlated variability in the spiking responses of ooDS cell pairs

To characterize noise correlations in ooDS cell spike outputs, we recorded simultaneously from cell pairs while projecting moving bars of light onto the mouse retina. There are four subtypes of ooDS cells based on direction tuning, each of which responds preferentially to motion in one of the cardinal directions (Fig. 1A; Barlow and Levick, 1965; Oyster and Barlow, 1967). This means that there are three types of ooDS cell pairs as defined by the separation in their tuning curve peaks (0°, 90°, or 180°). We recorded from neighboring ooDS cell pairs with each possible tuning relation; these neighboring cells have partially overlapping receptive fields, and thus encode motion in the same region of visual space.

Fig. 1. Stimulus-dependence of ooDS cell pair responses.

(A) We performed simultaneous recordings from pairs of nearby ooDS cells with overlapping receptive fields. (B) Raster plot of spike responses from an ooDS cell pair in response to bars moving in eight different directions. See Supplemental Information and Fig. S1 for details on ooDS cell identification. (C) Mean +/− S.E.M. spike count as a function of bar direction (“tuning curve”) for each cell in the pair - these cells have a ~180° offset in their preferred directions. (D) Spike counts of the ooDS cell pair over many repeats of the same stimulus, shown for two directions of motion. Marginal distributions along axes show trial-to-trial variability in spike responses, while full response distributions show that this variability is correlated between cells (left distribution: linear correlation coefficient = 0.52, p < 10⁻⁴; right distribution: linear correlation coefficient = 0.80, p < 10⁻¹¹). (E) We computed the geometric mean spike count for the pair by taking the square root of the product of the two tuning curves. For the cell pair in (B-D) (left panel), the noise correlation coefficient (red: error bars show 95% confidence intervals) varies with bar direction and follows the geometric mean +/− S.E.M. spike count (grey, dotted line). The same is true for a pair of cells with the same preferred direction (right panel). Corresponding data for all 14 recorded cell pairs is shown in Fig. S2. (F) Summary of the relationship between the geometric mean responses and the noise correlation coefficient. Before computing the geometric mean responses, we first normalized the tuning curve of each cell. Each point corresponds to one stimulus presented to one cell pair (8 stimuli per pair, 14 recorded pairs). Filled symbols denote the mean +/− std. dev. correlation in each bin. Note that the spike count correlation increases with geometric mean spike count (linear correlation coefficient = 0.49, p = 5×10⁻⁸). Filled, colored symbols represent the example pairs in (E). (G) Mean +/− S.E.M. correlation coefficient over all eight directions for cell pairs with approximately 0, 90, and 180° offset in their preferred directions.

We presented moving bars in eight different directions and measured the spike count during the 2-3 second response to each stimulus (Fig. 1B). We computed the mean (neural tuning curves; Fig 1C) and covariability (noise correlations; Figs. 1D-G) of the spike count across many repeats of each bar direction. Both measures varied systematically with bar direction (see also Amthor et al., 2005; Franke et al., 2015). Moreover, there was a clear relationship between the mean responses of the two cells and their correlation coefficient (Fig. 1EF, Fig. S2): higher geometric mean responses (square root of the product of the tuning curves) occurred together with higher correlation coefficients (see also Franke et al., 2015). This effect was independent of the type of ooDS pair recorded and all three types had mostly positive noise correlations (Fig. 1E-G). Thus, correlations were strongest when both cells were strongly spiking, regardless of their tuning.

Cell pairs with similar tuning curves (0° pairs) have many stimulus values where both cells fire strongly, and thus have high correlation coefficients when averaged across stimuli (Fig. 1G). The dependence of the (average) strength of noise correlations on tuning curve overlap is consistent with the notion of “limited range” correlations in the literature (Averbeck et al., 2006; Shamir, 2014; Zohary et al., 1994; Cohen and Kohn, 2011; Ecker et al., 2011; Abbott and Dayan, 1999; Sompolinsky et al., 2001). These “limited range” correlations are typically thought to be harmful to neural population codes (Averbeck et al., 2006). However, most previous examinations of these correlations have only considered the case where they are independent of the stimulus (see Lin et al., 2015 for an exception). Below, we show that stimulus-dependent correlations can lead to different conclusions.

Why might stimulus dependence of noise correlations affect neural population codes?

Noise correlations can be strongly stimulus dependent (Fig. 1EF; de la Rocha et al., 2007; Binder and Powers 2001; Franke et al 2015; Lampl et al., 1999; Samonds and Bonds, 2004; Granot-Atedgi et al., 2013; Ponce-Alvarez et al., 2013; see also Kohn and Smith 2005). Nonetheless, reported noise correlations are often averaged across stimuli. Figure 2 below provides a simple example, based on cell pairs, to illustrate how stimulus dependence can change the impact of noise correlations; (also see Franke et al. 2015).

Fig. 2. Stimulus-dependence of noise correlations may change our perspectives on neural population coding.

(A) Tuning curves of two model neurons (solid lines), along with the noise correlation coefficient (dashed blue line), which is assumed to be proportional to the geometric mean of the neural tuning curves (as in Fig. 1F). For comparison, we show the average of that correlation coefficient over stimuli (horizontal dashed red line). (B) For the tuning curves in (A), we computed the pairs of mean responses elicited by all possible stimulus angles: this “signal” is indicated by the black curve. To investigate the relationship between the trial-to-trial variability and this signal curve, we assumed Poisson-like variability (variance equal to mean), and computed the stimulus-conditioned response distributions assuming either either stimulus-dependent correlations (as in (A); solid blue ellipses), or stimulus-independent correlations that matched the average of the stimulus-dependent correlations (dashed red ellipses). Ellipses show 1 standard-deviation probability contours for 3 different stimulus values (stimulus values indicated on panel (A)). By changing the projections of the stimulus-conditioned distributions onto the signal curve (indicated, for example, by the double-headed arrows for the responses to stimulus 1), the stimulus-dependence of the correlations impacts the population code.

The impact of noise correlations on neural encoding depends on the structure of the signal space (Averbeck et al., 2006), which describes how the mean population response changes as the relevant stimulus parameter changes. The black line in Fig. 2B illustrates the signal space for a hypothetical pair of neurons encoding motion direction with tuning curves shown in Fig. 2A (solid curves). Noise spreads responses out along the signal space, leading to ambiguity in the mapping between stimuli and neural responses. The impact of this noise is represented by the projection of the noise onto the signal space (Fig. 2B). Noise oriented along the signal space limits the precision with which the signal is encoded, while noise along axes orthogonal to the signal space does not (Averbeck et al, 2006; Shamir et al, 2014; Panzeri et al., 1998).

Stimulus dependence of correlations alters coding by shaping the noise relative to the signal space. To illustrate this effect, we compare stimulus-dependent noise correlations resembling those in our data (Fig. 2A, blue line) with stimulus-independent noise correlations (Fig. 2A, red line); since the two types of correlations are equal when averaged across stimuli, we refer to the stimulus-independent correlations as “matched” to the stimulus-dependent ones. Stimulus dependence causes the projection of the noise along the signal space to be smaller than the stimulus-independent case for stimuli between the two cells’ tuning curve peaks (Fig. 2B, “Stim. 1”); other stimuli yield similar projections for stimulus-dependent and matched stimulus-independent noise correlations (“Stim. 2” and “Stim. 3”).

This example emphasizes that the responses of populations with stimulus-dependent and matched stimulus-independent correlations differ, and that this difference can affect the fidelity of the population code. Sensory information is generally encoded by neural populations with more than two cells, and intuitions about the overall impact of correlations do not necessarily extrapolate from cell pairs to larger populations (Hu et al., 2014). Nevertheless, the general rule still holds that fluctuations along axes orthogonal to the signal space do not limit coding fidelity.

Below, we use intracellular recordings and computational modeling to characterize the structure of both signal and noise in larger ooDS cell population responses. We then present theoretical calculations showing that, in general, stimulus-dependent correlations of the type seen in our data improve population coding in diversely-tuned neural populations.

Mechanisms underlying stimulus-dependent correlations

Biophysical origins of stimulus-dependent correlations

Several considerations suggest that noise correlations in the ooDS population result from common synaptic input rather than recurrent coupling. First, only one of the four ooDS cell types is gap-junction coupled to ooDS cells with the same direction tuning, while the other 3 sub-types have no such coupling (Trenholm et al., 2013; Trenholm et al., 2011). Second, there is no known coupling between ooDS cells of different sub-types. Hence neither 90° nor 180° pairs are coupled. Correlations for 0° pairs resembled those of 90° and 180° pairs (Fig. 1F), suggesting that recurrent coupling is not strongly influencing the observed correlations. Third, a feed-forward model of ooDS cell populations (Fig. 6) can account for the observed correlations, supporting the idea that feed-forward circuit architectures suffice for understanding them.

Fig. 6. A computational model captures the ooDS cell population response statistics and reveals that stimulus-dependent correlations significantly improve the population’s direction code.

(AB) Spiking statistics (black: ooDS cell population mean +/− S.E.M.) to which the the model (red: average over 25 8-cell model populations) was fit. (A) Single-cell statistics. (B) Pairwise statistics (see Supporting Information). (C) Tuning curves of an example 8-cell population generated by the fitted model. (D) The rate-correlation relationship (Fig. 1F) was not used in training the model; it serves as an independent test. Correlation coefficient and geometric mean response for 250 model cell pairs (red dots) and 14 experimentally observed ooDS cell pairs (black circles), each in response to 8 different stimuli. The two distributions are not significantly different (2-dimensional KS test, KS statistic 0.15, p=0.2). Overlain are the mean correlation coefficients in the experimental data (black symbols: mean +/− S.E.M.) in different bins of geometric mean response and in the computational model (solid red curve). (E) Fisher information provided by model 8-cell population responses about the stimulus direction. Colors indicate the assumed correlation structure: stimulus-dependent correlations, as in the experimental data (red); no correlations (yellow); or “matched” stimulus-independent correlations that, for each cell pair, match the stimulus-average of their stimulus-dependent correlation coefficients (blue). Error bars in (E) are the S.E.M. over ensembles of 10 randomly-generated model populations.

To elucidate the circuit mechanisms responsible for the noise correlations, we characterized the synaptic inputs to ooDS cells. We measured all four inputs to a cell pair (i.e. excitatory and inhibitory inputs to each cell) simultaneously using an alternating-voltage clamp technique (Cafaro and Rieke, 2010; Fig. 3A-B; see Methods). We switched the holding potential between the excitatory and inhibitory reversal potentials every 5 ms and measured the excitatory or inhibitory current at the end of each such period (Fig. 3A-B). Linear interpolation of these sampled currents yields a near-simultaneous estimate of both synaptic currents (Fig. S3).

Fig. 3. Alternating voltage clamp measurements reveal sources of variability in single-cell synaptic inputs.

(A) Method of paired alternating voltage clamp recordings: The voltage command (top) for both cells recorded simultaneously was alternated between the excitatory and inhibitory reversal potentials while the current (black traces) was recorded. Interpolating between the last value measured on each cycle yields an estimate of each cell’s excitatory (blue) and inhibitory (red) current nearly simultaneously. (B) We simultaneously measured these four synaptic currents (excitatory and inhibitory inputs to each cell) while presenting moving bar stimuli. Highlighted region corresponds to the traces in (A). (C) Excitatory (blue) and inhibitory (red) conductances in a pair of ooDS cells measured simultaneously in response to moving bars. Mean responses (shading indicates S.E.M.) reveal tuning of input conductances. Subtracting the mean conductance traces (over nearby trials) from single-trial traces, we obtained residuals, which are shown for a single trial. (D) Using conductance residuals, we computed three single-cell (co)variances of the inputs (as a function of time delay) for each stimulus and each cell in the pair: inhibitory variance (red), excitatory variance (blue) and single-cell excitatory-inhibitory covariance (black). Shown are the (co)variance functions for cell 1 of the example pair. (E) Schematic of stimulus-dependent gain model. A portion of upstream noise is shared between cells and among excitatory and inhibitory channels. Upstream noise is multiplied by a stimulus-dependent gain factor: g_E,1(s) or g_I,1(s), defined as the measured mean conductance in each channel for each stimulus. Thereafter, independent, post-gain, noise is added to each channel (N_E,1 and N_I,1, respectively), yielding the noisy conductance inputs to the cell. Measured simultaneously, these inputs reveal the three possible (co)variances into a single cell. (F) Single cell peak (co)variance as a function of gain product for the responses of cell 1 above to 8 different stimuli. Solid line shows the linear best fit: data confirm the model’s prediction that (co)variance should be linearly related to the relevant gain product. (G) Linear correlation coefficients between (co)variance and gain product for the three single-cell input (co)variances measured in 21 ooDS cells. Thick lines and error bars indicate mean +/− S.E.M. and filled points indicate the example cell.

This approach allows measurement of all relevant means, variances, and covariances for inputs to two ooDS cells. We start by describing the means and (co)variances of inputs to single cells (Fig. 3). We then measure the covariances of the inputs to different cells (Fig. 4).

Fig. 4. Alternating voltage clamp experiments reveal the sources of co-variability between synaptic inputs to ooDS cell pairs.

(A) Paired recordings as in Figure 3 allow measurement of all four types of pairwise input covariances: EE (blue), II (red), EI (dotted green), and IE: (solid green). The network model in Fig. 3 was generalized to account for pairwise input correlations. A fraction of upstream noise is shared between cells in a pair (“Common noise”, N_c) (B) In response to two illustrative bar directions, the pairwise input covariance functions are shown for the example cell pair in figure 3. As with single-cell input (co)variances, pairwise covariances are modulated by bar direction. (C) Peak covariance of all four types of input correlations as a function of bar angle for the example cell pair. (D) The stimulus-dependent gain model in (A) predicts a linear relationship between pairwise input covariance and the relevant gain product - this prediction is borne out in this example cell pair. Solid lines show linear best fit. EI and IE covariances have been combined for reasons of symmetry. (E) Population data from 9 ooDS cell pairs showing linear correlation coefficients between the three pairwise covariances and their respective gain products. Thick lines and error bars indicate mean +/− S.E.M. and filled points indicate the example pair.

Variability in synaptic inputs: single cells

The mean excitatory (E) and inhibitory (I) conductances had opposite preferred bar directions but I inputs were more strongly and consistently tuned (Fig. 3C). Consequently, E and I inputs do not remain balanced as the stimulus varies.

We estimated the variability in the synaptic inputs elicited by a moving bar by subtracting an estimate of the mean response from the responses recorded on individual trials (Fig. 3C; see Supporting Information for details). The conductance residuals were used to compute the variances and covariances as a function of time delay (Fig. 3D). All three single cell (co)variances were modulated with bar direction - much as noise correlations in the spike outputs depended on direction (Figure 1DE).

Based on the stimulus dependence of the means and (co)variances, as well as prior work on the sharing of upstream noise in the retina (Khuc-Trong and Rieke, 2008; Ala-Laurila et al., 2011), we hypothesized a circuit architecture wherein a multiplicative stimulus-dependent gain acts on separate E and I pathways that share common, noisy input (Fig. 3E). In the model, upstream noise diverges into parallel E and I pathways and hence is shared between them. Each pathway includes a stimulus-dependent multiplicative “gain” term (g_E & g_I) equal to the mean conductance in that channel for the given stimulus. Finally, additional noise (N_E & N_I), independent in the two pathways and not subject to the multiplicative gain, is added. This model can be implemented with signal-independent noise properties, and a single exponential nonlinearity (Fig. S4).

The model of Fig. 3E predicts a linear dependence of the variance of E or I inputs on the square of the respective gain, and similarly a linear dependence of the covariance of E and I inputs on the product of the E and I gains. We estimated the means of the E or I conductances during short windows about the peak response to the stimulus (see Methods), set the gains in our model equal to these mean conductances, and measured the peak (co)variability in the synaptic inputs during the same time window. Our data supports the predicted linear dependence of (co)variance on squared gains (Fig. 3FG; Fig. S5). These results suggest that the ooDS cell synaptic inputs are well-described by the model of Fig. 3E, in which common input noise, modulated by stimulus-dependent gain factors, underlies the co-fluctuations of E and I inputs to single neurons.

Related models that explicitly couple gain (equal to the mean response to a given stimulus in our case) and (co)variance fluctuations across time or stimuli have been recently explored (Lin et al., 2015; Goris et al., 2014). Our model is equivalent to the “multiplicative” one considered by (Lin et al., 2015), where the gain is proportional to the variance, and is similar to the model of (Goris et al., 2014), where the gain and variance are related but not strictly proportional. Our model differs from that of (Ecker et al., 2014), which has no explicit relationship between gain and variance.

Variability in synaptic inputs: cell pairs

In addition to the converging (co)variances of the inputs to single cells, the paired alternating-voltage recordings characterized the four pairwise input covariances (Fig. 4A): covariance between excitatory input into cell 1 and excitatory input into cell 2 (“EE” covariance), covariance between inhibitory inputs (“II”), excitatory-inhibitory (“EI”) covariance, and inhibitory-excitatory (“IE”) covariance. We computed these covariances using the conductance residuals as in Fig. 3A. The pairwise input covariances, like the single cell covariances, showed a clear dependence on bar direction (Fig. 4BC).

We generalized the network model used to capture the stimulus dependence of the inputs to single cells to cell pairs (Fig. 4A). This generalized model similarly predicts that the covariance of each of the pairwise inputs will depend linearly on the product of the relevant gains. This prediction is confirmed by our paired recordings (Fig. 4DE): the circuit architecture in Fig. 4A offers a parsimonious circuit-level explanation for shared stimulus-dependent fluctuations in the inputs to neural populations; shared upstream noise, modulated by the stimulus-dependent gain, leads to co-fluctuations in the synaptic inputs experienced by ooDS cell pairs.

Relating input and output correlations

How do the different sources of input correlation collectively generate output correlations? The transfer of input correlations to output correlations depends on the relative strength of excitatory and inhibitory inputs and on nonlinearities in synaptic integration and spike generation (Fig. S6; de la Rocha et al., 2007). Of particular relevance here, the effects of EE and II input correlations (which positively correlate the cells’ spiking responses) can be at least partially canceled by EI input correlations (Renart et al., 2010; Graupner and Reyes, 2013; Hansen et al., 2012; Shadlen and Newsome, 1994; Shadlen and Newsome, 1998; Ly et al., 2012). Such cancellation is particularly effective for neurons operating in a balanced regime where E and I inputs are similar in magnitude, and this is often cited as a reason for the weak average correlations exhibited in cortical circuits. However, the tuning of ooDS cells relies on modulation of I relative to E (Fig. 3A), that in turn modulates the EI correlations (Fig. 4BC). This suggests that EI correlations may not always cancel EE and II input correlations.

Linear predictions of output correlations based on the measured synaptic inputs should reveal a role of EI correlations in limiting output correlation strength. However, such linear predictions failed to capture output correlations (Fig. S6). Hence we used dynamic clamp techniques to determine how manipulating input correlations affected output correlations (Cafaro and Rieke, 2010; Sharp et al., 1993). The dynamic clamp approach measures a cell’s voltage while injecting current calculated from the measured time-varying conductances. We manipulated the correlations in the inputs by injecting either simultaneously-recorded conductances (correlated) or non-simultaneously-recorded conductances (uncorrelated), measured in response to the same stimulus (e.g. Fig. 5AB). By comparing spike responses recorded under different conditions (Fig. 5C), we isolated the impacts of different types of input correlations on the correlations in the cells’ spiking responses. Spike count correlations were measured as in Fig. 1.

Fig. 5. Pairwise input correlations differentially shape output correlations.

(A) Example input conductance traces measured in alternating-voltage experiments using three different bar directions, and injected into recipient cells in dynamic-clamp experiments. By shuffling conductance combinations we can exclude or include specific sets of input correlations. For these inputs to cell 1 (top) and cell 2 (bottom) on this trial, we have taken the excitatory conductances measured on trial 2 and combined them with the inhibitory conductances measured on trial 1, leaving only pairwise EE & II input correlations intact, while breaking all other sources of input correlations. (B) Combined conductances are injected into a recipient ooDS cell, whose output voltage is measured, and spike correlations are measured as in Figure 1. (C) Schematic showing the logic of shuffling input conductances to include specific sets of pairwise input correlations. (D) Correlation coefficients for dynamic clamp trials where all input correlations were left intact versus those where only pairwise EE & II correlations remained. Each point corresponds to one stimulus for one pair, shown are data from 11 cells, three stimuli each (52-88 trials, mean 72 trials). Dotted line is unity. Removing pairwise EI correlations increased output correlations (p = 2×10⁻⁵, paired t-test) (E) Same as (D) but showing trials where all input correlations remained versus those wherein only pairwise EI correlations were intact. Removal of pair-wise EE & II correlations decreased output correlations (t-test, p = 2×10⁻¹⁰, paired t-test).

Spikes produced when EI correlations were removed and EE and II input correlations were retained showed higher output noise correlations than did control data in which all input correlations were left intact (Fig. 5D). Spike responses in which pairwise EI input correlations were retained and EE and II input correlations were removed showed near-zero or negative correlations (Fig. 5E). These observations support the general notion that EE and II correlations increase, whereas EI correlations reduce, correlations in spiking responses. However, the suppressive impact of EI correlations was relatively small compared to the role of EE and II correlations— the inclusion of EI correlations (Fig. 5D) reduced output correlations by just 34 +/− 8% (mean +/− S.E.M.). Thus, the impact of EI correlations may be relatively modest during stimuli that modulate the EI balance.

Impact of stimulus-dependent correlations on direction coding

The mechanistic understanding from the above experiments allowed us to construct a model of the ooDS cell population in which we could manipulate the noise correlations and investigate their impact on direction coding. We then generalized our study of the impact of stimulus-dependent correlations on population coding to arbitrary heterogeneous neural populations.

A mechanistic model of a ooDS cell population

Ganglion cell tiling and dendritic overlap (Amthor and Oyster, 1995) suggest that a given region of visual space is sampled by at least eight ooDS cells (two of each of the four subtypes). We constructed a mechanistic model to understand direction coding by these 8-cell populations. The model allowed us to explore stimulus space more completely than we could experimentally and to manipulate otherwise inaccessible features like the stimulus dependence of spike correlations.

The model follows the architecture revealed by our experiments (Fig. 4A; see Supporting Information and Fig. S7). The model has 13 free parameters describing basic network and intrinsic cellular properties (the amplitudes of inhibitory and excitatory inputs, the stimulus dependence of inhibitory input, the variances of common vs. independent noise sources, etc.). We fit these parameters to match 15 different experimentally-measured quantities describing the single-cell and pairwise response statistics (Fig. 6AB).

The model has realistic tuning curves and realistic levels of trial-to-trial response variability (Figs. 6AC). Moreover, although the model was not directly fit to the rate-correlation relationship, (Fig. 1F), it recreates it with high fidelity (Fig. 6D). We next used the model to study how the noise correlation structure affects direction encoding.

Correlations, stimulus dependence, and direction coding by ooDS cell populations

We used the model to generate responses to 500 different directions of motion; these responses captured the experimentally observed first- and second-order spiking statistics (see Supporting Information). Using these statistics (stimulus-dependent means and covariances), we computed the linear Fisher information (see Methods), which quantifies the population’s coding ability. The Fisher information places an upper bound on the precision with which the stimulus can be recovered from the neural responses by an unbiased linear estimator (Rao, 1945; Cramer, 1946), and is a standard way to assess neural population coding (Abbott and Dayan, 1999; Shamir, 2014; Moreno-Bote et al., 2014; Hu et al., 2014). We first compared coding performance for model responses with the stimulus-dependent correlation structure with that for trial-shuffled uncorrelated data (Fig 6E); correlated responses provided >100% more information. Analyses of coding based on simultaneous recordings from larger populations (Franke et al., 2015) show similar improvements in coding performance.

What are the key features of the correlations that give rise to their beneficial impact on coding performance? Observed noise correlations were strongly modulated with the stimulus (Fig. 1F). To investigate the role of this stimulus dependence, we generated responses in which the noise correlations for a given cell pair, for all stimuli, were maintained at the average value of the correlation for that pair (Fig. 6E “Matched Stim.-Indep. Corr.”). Like the uncorrelated data, these responses had relatively low information. This suggests that the stimulus dependence of the correlations is an important feature of the population code, and that the average (over stimuli) level of correlation does not necessarily capture how correlations affect coding.

One limitation of the analysis above is that linear Fisher information cannot extract information encoded in, for example, the stimulus-dependent variances of the neural activities (Shamir and Sompolinsky, 2004). To verify the robustness of our findings (Fig. 6), we repeated the above investigations using different measures of coding performance. We quantified the error of the Maximum Likelihood Estimator (MLE) and the Optimal Linear Estimator (OLE; Salinas and Abbott, 1994) of the stimulus given the neural activities (see Supporting Information). The results (Fig. S8) qualitatively match that which we obtained using Fisher information.

Generalization to larger and more heterogeneous neural populations

The 8-ooDS-cell model indicates that stimulus-dependent noise correlations can significantly boost the neural direction code. How do these observations apply to larger and possibly more heterogeneous populations? To answer this question, we first performed calculations of the amount of stimulus information conveyed by neural populations with identically-shaped and evenly-spaced tuning curves (Figs. 7ABC). For each population and stimulus, the mean responses were given by the tuning curves, and the variability was assumed to be Poisson-like (variances equal to means). Finally, we assumed that the correlations were stimulus dependent, and proportional to the geometric mean of the neural responses (as in the experimental data, Fig. 1F). The magnitude of the correlations was parameterized by ρ_max, which determines the largest possible correlation coefficient in the population. Our general setup and computation of information in the case of stimulus-dependent correlations follows (Josic et al. 2009; their IF-_mean), where explicit analytical formulae are derived. The results for these homogeneous populations (Fig. 7B) indicate that stimulus-dependent correlations lead to much more information than is contained in independent populations with the same tuning curves and noise levels.

Fig. 7. Stimulus-dependent correlations typically improve population coding.

We generated model neural populations of different sizes, with either identically-shaped and evenly displaced tuning curves (example population shown in panel (A)), or with randomly shaped and located tuning curves (example population shown in panel (D)). For each population, we assumed Poisson-like variability, and let the noise correlations coefficients be stimulus-dependent (being proportional to the geometric means of the cells’ tuning curves), with the overall magnitude of correlations in the population being determined by the parameter ρ_max (maximum possible correlation coefficient, obtained for cell pairs where both cells are firing at their maximum possible rates). For populations of different sizes, and with different ρ_max values, we then computed the Fisher information those populations provide about the stimulus. On balance, stronger stimulus-dependent correlations yield better coding performance, over a wide rage of population sizes (BE). (CF) To understand how the stimulus-dependence of the noise correlations affects the population code, we repeated our calculations from panels (BE), but instead of keeping the “full” stimulus-dependent correlation structure, we replaced the correlation coefficients for each cell pair with the average over stimuli of that pair’s stimulus-dependent correlations: average correlations are “matched” between (B) and (C), and between (E) and (F). With stimulus-independent correlations (in stark contrast to stimulus-dependent ones), larger correlations correspond to weaker population codes (CF). Error bars in (EF) are the S.E.M. over ensembles of 20 randomly-chosen model populations. Average correlations values in the legends of (BE) are averages over all stimuli and all cell pairs. The values are larger for the heterogeneous population because the heterogeneous population has, on average, wider tuning curves.

To directly test the importance of stimulus dependence, we repeated our calculations while holding the correlation coefficient for each cell pair constant at that pair’s stimulus-averaged value (as in red curve in Fig. 2A). With these “matched” stimulus-independent correlations, coding performance is typically worse than is obtained by independent cells (Fig. 7C). Similar results are seen for populations with randomly-shaped and randomly-located tuning curves (Figs. 7DEF). For this case of stimulus-independent correlations, the correlation coefficients have a “limited-range” structure (Averbeck et al., 2006; Shamir, 2014; Zohary et al., 1994; Cohen and Kohn, 2011; Ecker et al., 2011; Abbott and Dayan, 1999; Sompolinsky et al., 2001) in which the correlations between pairs of cells decrease as the cells’ tuning curves become more widely spaced. These limited-range correlations are typically thought to be harmful to neural population coding, although that result can depend on the heterogeneity of the neural tuning curves, and the speed with which correlations fall off as tuning curve overlap decreases (Shamir, 2014; Ecker et al., 2011; Shamir and Sompolinsky, 2006). Our results indicate that, if the limited-range structure arises due to stimulus dependence of the correlation coefficients (as in our DS cell recordings; Fig. 1EF), then those correlations can significantly improve the population code; this effect might be missed if the stimulus dependence of the correlations were ignored.

Do the coding benefits of stimulus-dependent correlations occur in populations with small levels of correlation, as often exhibited (on average) by cortical neurons (Ecker et al., 2014; Gawne and Richmond, 1993; Bair et al., 2001; Reich et al., 2001; Cohen and Kohn, 2011; Ecker et al., 2010; Shamir, 2014)? While the largest coding benefits are obtained for large ρ_max values, the average correlation coefficients are much smaller than ρ_max. To highlight this, we show in the legend to Figs. 7BE the average correlation coefficient (averaged over stimuli and cell pairs) obtained for each ρ_max value. Even for ρ_max=0.8, the average correlation coefficient is only 0.11 (for the homogeneous population), which is similar to the values typically reported in cortical recordings.

While one previous study analyzed the role of stimulus-dependent correlations in neural population coding in a similar setting, that work (Josic et al., 2009) compared the stimulus-dependent correlations to the case where the correlation coefficients were stimulus independent and uniform across the population. That investigation did not resolve whether coding improvement in the stimulus-dependent case should be attributed to the stimulus dependence of the correlations, or to the diversity of correlations across the population. For contrast, we compared populations with stimulus-dependent correlations to populations with matched stimulus-independent correlations, where those correlations varied between cells so as to match the average (over stimuli) of the correlations in the stimulus-dependent case. Our observations (Fig. 7) thus resolve the ambiguity left by (Josic et al., 2009), and point to significantly larger effects.

In summary, our calculations show that stimulus-dependent noise correlations may be a generally beneficial feature of neural population codes — and a feature that could be missed by averaging correlations over stimuli when reporting spike correlations.

Geometrical intuition: orthogonality between signal and noise

Why does the stimulus dependence of the noise correlations have such a striking impact on the neural population code? The impact of noise depends on its projection onto the signal space (Fig. 2). First, consider the signal space itself. For a population of cells with tuning curves uniformly tiling the space of stimulus direction (e.g. Fig. 6C, Fig. 7A), each stimulus direction leads to strong activation of some cells and weak activation of others. As the stimulus direction changes, the identities of the active and inactive cells change, but the overall level of activity in the population is relatively constant. Geometrically, this means that the signal space will lie on or near the surface of a (hyper-) spherical shell in the space of neural responses (Fig. 8A). Vectors pointing radially are orthogonal to this spherical signal space. Thus, any trial-by-trial fluctuations that are radial in response space (i.e. leading to changes in the magnitude but not direction of the response vector) will minimally impact the neural code for direction.

Fig. 8. Why (and when) do stimulus-dependent correlations improve neural population codes?

(A) The space of possible neural activities is shown. Each axis in this space is the firing rate of a single cell. Within that space, each stimulus elicits a distribution of responses (ellipses show 1 standard-deviation probability contours). For populations (like those in Fig. 7AD and 6C) whose tuning curves tile the response space, the stimulus-averaged population responses (and thus the “signal” curve) tends to lie near a spherical shell in the space of possible neural activities. In this case, trial-to-trial variability in the radial direction is relatively benign to the population code, whereas variability in the tangential direction (along the signal curve) is especially pernicious. (B) The mean response to a given stimulus is indicated by the green arrow. The amount of variability along the direction indicated by the mean response vector (“radial direction”) is shown in orange. (C) For heterogeneously tuned populations (as in Fig. 7D), we computed the fraction of overall variability in the population responses that lies along the radial direction as a function of the strength of the stimulus-dependent correlations (ρ_max). The calculation was repeated for several different population sizes; error bars are +/− 1 S.E.M. over 20 randomly-drawn populations. (D) To predict which populations will show coding benefits from stimulus-dependent correlations, we measured the cosine of the angle (θ) between the stimulus-induced changes in neural population responses, and the mean (trial-averaged) population response vector. (E) We considered 200 randomly-generated populations of 100 cells each (blue data points). For each population, the stimulus-averaged cosine value (defined in D) is shown (horizontal axis). For the same neural populations, we also computed the Fisher information, either in the presence of stimulus-dependent correlations (ρ_max = 0.8) or with uncorrelated noise, and computed the percentage by which the population code formed in the presence of stimulus-dependent correlated noise exceeds the performance of the population code formed in the presence of independent noise. For comparison, we repeated this calculation with 8-cell populations (red data points), and we show the corresponding values for our 8 ooDS-cell model populations (black square; model is described in Fig. 6; data shown is averaged over 10 model ooDS cell populations, of 8 cells each). The ooDS cell population appears as an outlier to the point clouds in (E) because the ooDS cell populations have a larger ρ_max value (of roughly 1; see Fig. 6D) than do the other populations shown here.

Increasing the magnitude of the stimulus-dependent correlations increases the fraction of the trial-to-trial variability that is in the radial direction (Fig. 8BC; see Supporting Information for mathematical explanation). This effect can be observed in the pairwise model example in Fig. 2 and can be seen in our recorded ooDS pair population (Fig. S9). Thus, stimulus-dependent correlations enhance the population code by shaping the noise in the population responses such that it is orthogonal to the signal space. Interestingly, the stimulus-dependent gain mechanism we uncovered in our ooDS cell populations (Fig. 4A, Fig. S7) can also be shown to orient noise in the radial direction in the space of possible neural responses (see Supporting Information for details).

Generality for populations with heterogeneous tuning curves

The geometrical picture discussed above identifies the conditions under which stimulus-dependent correlations will improve the population code: the benefits of stimulus-dependent correlations arise whenever the “signal” curve is orthogonal to the radial direction (i.e. when signals lie along a spherical shell). To test this intuition, we developed a metric to measure the extent to which the signals are orthogonal to the radial direction and tested this metric across several neural populations with differing degrees of tuning curve regularity.

The metric we used measures the cosine of the angle between the mean response vector for a particular stimulus and the stimulus-induced changes in the mean response vector (Fig. 8D). This cosine is averaged over all stimuli to yield our metric. When the signal direction is typically orthogonal to the radial direction, the average-cosine metric is near 0 (cosine of 90°), and stimulus-dependent noise correlations are expected to enhance the population code. When the signals lie predominantly along the radial direction in the response space, the average cosine metric will be near 1 and the stimulus-dependent correlations may be damaging to the population code.

We assembled many neural populations with randomly-drawn tuning curves (as in Fig. 7C; see Supporting Information for details). For each population, we computed the Fisher Information (as a measure of the ability of the population to encode stimulus direction) either in the presence of stimulus-dependent noise correlations (with ρ_max = 0.8), or in the presence of uncorrelated noise. For each population, we computed the percentage improvement in population coding performance that is obtained in the presence of stimulus-dependent correlations vs uncorrelated noise. By comparing the coding benefit of stimulus-dependent correlations (vertical axis of Fig. 8E) to the average-cosine metric (horizontal axis of Fig. 8E) for these populations, we see that the average-cosine metric is a good indicator of when stimulus-dependent correlations improve neural population codes: low average-cosine values correspond to large coding benefits from stimulus-dependent correlations. We repeated our calculations on populations of both 8 and 100 cells, and observed the same trends. This serves to confirm the geometric intuition (Fig. 8A) underlying the coding benefits of stimulus-dependent correlations.

We emphasize that, in generating Fig. 8E, we took extra care to draw model populations that had a wide range of average-cosine values. To do this, we forced the tuning curves to be clustered together (not fully spanning the stimulus space) to varying degrees (see Supporting Information). If we had let the tuning curves be randomly positioned within the stimulus space (as in Fig. 7D), we would have observed predominantly low values of the average-cosine metric (not shown): when the tuning curves randomly tile the stimulus space, the overall population activity level varies relatively little as the stimulus changes. This effect is magnified in larger populations. Thus, in large neural populations, the geometrical picture presented in Fig. 8A is more likely to apply, and stimulus-dependent noise correlations are more likely to improve population coding.

Discussion

Studies of population coding in retinal ooDS ganglion cells provide a unique opportunity to investigate the extent, origins and impacts of correlated variability because the relevant circuit mechanisms can be studied directly in the context of physiological stimuli and because the space of relevant stimulus parameters encoded by these cells is clear. Our central finding is that the circuitry shaping the synaptic inputs to ooDS cells generates stimulus-dependent correlations in the cells’ trial-to-trial variability; these correlations enhance the precision of direction coding roughly 2-fold compared to the case of independent noise (see also Franke et al., 2015).

Spike outputs of ooDS cells exhibit positive and strongly stimulus-dependent noise correlations (Fig. 1; see also Franke et al., 2015; Amthor et al., 2005). The seemingly complex correlation structure falls naturally out of a direction-tuned circuit with shared upstream noise (Figs. 3,4). Thus divergence of noisy signals produces correlated fluctuations in downstream neurons; stimulus dependence of those correlations arises due to circuit nonlinearities that control the gain of shared noise. Such circuit mechanisms are common, and other neural circuits are likely to share the beneficial shaping of noise described here. Indeed, recent work shows that noise correlations in visual cortex resemble those of the ooDS cells and that the resulting shaping of noise can benefit coding (Lin et al., 2015).

In agreement with previous work (Renart et al., 2010; Graupner and Reyes, 2013), correlations between excitatory and inhibitory synaptic inputs decorrelated spike outputs (Fig. 5DE). However, this decorrelating mechanism did not prevent large output correlations in ooDS cells because excitatory and inhibitory synaptic inputs are not always balanced (Fig. 3). Indeed, retinal direction selectivity is known to rely on a strongly modulated ratio of excitatory to inhibitory input (Fried et al., 2002; Taylor et al., 2000). Temporal EI imbalance has similarly been shown to shape stimulus selectivity in many areas of the brain (Wilent and Contreras, 2005; Ferster, 1988), and an EI imbalance has been suggested to play a role in gating noise correlations in cortex (Hansen et al., 2012).

The observed stimulus-dependent correlations enhance the ability of the neural population to convey direction information, and these findings generalize to large heterogeneous populations (Figs. 7BE; also see Josic et al., 2009). Moreover, the strong population coding effects we see in the retinal ooDS cell population emerged only when the stimulus dependence of the correlations was accounted for. Thus, correlations that look (on average) to be small and incon-sequential for coding may in fact be important features of the population code.

One clear way for noise correlations to improve population coding is if, for all cell pairs, the noise correlations have opposite signs relative to the signal correlations (“sign rule”; Averbeck et al., 2006; Hu et al., 2014; Jeanne et al., 2013; Panzeri et al., 1999; Oram et al., 1998). This phenomenon is not responsible for the strong boost in coding performance observed here (Figs. 6,7). For example, in the 8-cell population, with two cells of each sub-type, there will be four cell pairs in which both cells have the same direction tuning (like the pair shown in Fig. 1B-E), and correspondingly positive signal correlations. For these cell pairs, the “sign rule” indicates that negative noise correlations would improve the population code. However, we typically observed positive noise correlations for these cell pairs. Thus, the sign rule is strongly violated for some cell pairs, yet noise correlations still improved the population code on balance (Fig. 6E).

What matters most in understanding how correlations impact population coding in cases like these is the orientation of signal and noise within the larger population, not within cell pairs (Hu et al., 2014; Shamir, 2014; Abbott and Dayan, 1999; da Silveira and Berry, 2014; Shamir and Sompolinsky, 2006). In other words, the coding effect occurs at the population level, and cannot always be understood by looking at cell pairs in isolation. Indeed, while some pairs showed reductions in coding precision when noise correlations were removed (Fig. S10, cf. schematic in Fig. 2), the effect for pairs of cells was modest. Related theoretical work shows that the sign rule identifies only one of the myriad patterns of correlation that can substantially improve population coding (Hu et al., 2014; Shamir, 2014; Franke et al., 2015; da Silveira and Berry, 2014) relative to the case of independent cells. The current study provides a clear example of how populations in which some pairwise responses violate the sign rule may nevertheless have noise correlations that are beneficial to the population code.

The structure of the signal space depends on what is being encoded. Here, we considered the encoding of direction of motion and found that the stimulus-dependent correlations were quite beneficial. This is not generally true for all stimulus parameters. For encoding contrast information (where the tuning of all ooDS cells is similar), correlations may well be harmful. This emphasizes that correlations do not remove noise from the population responses: rather, they orient noise along axes that may be beneficial for some computations, but potentially harmful to others.

The circuit mechanisms that underlie the observed stimulus-dependent noise correlations -- diverging, stimulus-tuned excitatory and inhibitory inputs to multiple cells -- are not unique to the retina. Moreover, many sensory populations are composed of neurons with heterogeneous tuning properties that together span the relevant parameter space. These facts suggest that the relation between the circuit mechanisms shaping noise correlations and their impact on coding that we explored here may have correlates in other circuits in the nervous system.

Experimental Procedures

Tissue preparation

We used 6-10 week old mice (c57/BL6 or TRHR-GFP [Rivlin-Etzion et al., 2011] or BDxTSY [Kim et al., 2010]). These mice express GFP in ooDS cells, facilitating cell identification. BD mice were injected with tamoxifen (100μg, Sigma) intraperitoneally at postnatal day 0-1. The mice were dark adapted at least 2 hours. Under IR illumination, animals were euthanized by cervical dislocation and the eyes were removed and hemisected. The vitreous humor was removed mechanically and eye cups were stored in a light-tight container with 32° C bicarbonate-buffered Ames’ solution (Sigma), continuously bubbled with 95% O₂/5% CO₂. For recording, wedges of retina were removed from the sclera and retinal pigment epithelium and mounted photoreceptor-side down onto a poly-D-lysine coated glass coverslip (BD Biosciences). Retinal mounts were perfused with 30-36° C oxygenated Ames’ at 6-9 mL/min.

See Supplemental Information for more details on cell identification.

Visual stimulation

Visual stimuli were designed using MATLAB (MathWorks) and the Psychophysics Toolbox and delivered through an OLED monitor (eMagin, Bellevue, WA) focused on the photo-receptor outer segments. All light-driven responses were recorded at a background generating approximately 50 rhodopsin isomerizations/rod/second. To probe direction selectivity of ooDS cell pairs (Figs 1,3,4,5), stimuli were centered midway between the two somas and ~100% contrast bars were swept through their receptive fields. Bars were 120-180 μm wide (perpendicular to the axis of motion) and moved ~850 μm/s on the retina (except two cell pairs: 575 μm/s).

Information Calculations

To compute the information content of the model ooDS cell populations (Fig. 6), and the larger model populations (Figs. 7,8), we used the linear Fisher information. This is a fairly standard way to quantify neural population codes [Averbeck, Latham, and Pouget, 2006; Hu et al.,, 2014; Beck et al., 2011], and it determines the precision with which the neural activities can be decoded to recover the stimulus. Mathematically, the Fisher information uses the vector of tuning curves (where, for a given stimulus angle θ, each element of the vector is one neuron’s mean firing rate), and the covariance matrix Σ(θ) of the trial-to-trial variability in responses to stimulus θ. The Fisher information I(θ) is then calculated as

$$I\left(\theta \right)={\frac{d_{\underset{f\left(\theta \right)}{\to }}}{d\theta }}^T\Sigma {\left(\theta \right)}^{-1}\frac{d_{\underset{f\left(\theta \right)}{\to }}}{d\theta }$$ (1)

For our information calculations, I(θ) was computed for many different stimuli (500 for the data in Fig. 6; 100 for the data in Fig. 7) uniformly spanning the range of [0, 2π], and the reported information quantities are averages over all such stimuli.

Model Containing Stimulus-Dependent Correlations

For the data shown in Fig. 7, we generated the neural tuning curves via Von Mises functions (as in [Ecker et al., 2011; Hu et al., 2014]), and assumed Poisson variability (vector of spike count variance equal to vector of mean spike counts given by tuning curves f(θ)). The correlation coefficient between cells i and j was assumed to follow the functional form

$$\begin{array}{cc}\hfill {\rho }_{\mathit{ij}}& =a_{\mathit{ij}}\sqrt{f_i\left(\theta \right)f_j\left(\theta \right)},\hfill \\ \hfill a_{\mathit{ij}}& ={\rho }_{\mathit{max}}{\left[\sqrt{{\mathit{max}}_{\theta }\left(f_i\right(\theta \left)\right){\mathit{max}}_{\theta }\left(f_j\right(\theta \left)\right)}\right]}^{-1}\hfill \end{array}$$ (2)

where the maxθ(.) operation selects the peak amplitude of the tuning curve. This function ensures that the correlation coefficient varies between 0 and 1, and increases with increasing geometric mean firing rate (as in our experimental data).