Synergistic Coding of Visual Information in Columnar Networks

SUMMARY

Incoming stimuli are encoded collectively by populations of cortical neurons, which transmit information by using a neural code thought to be predominantly redundant. Redundant coding is widely believed to reflect a design choice whereby neurons with overlapping receptive fields sample environmental stimuli to convey similar information. Here, we performed multielectrode laminar recordings in awake monkey V1 to report significant synergistic interactions between nearby neurons within a cortical column. These interactions are clustered non-randomly across cortical layers to form synergy and redundancy hubs. Homogeneous sub-populations comprising synergy hubs decode stimulus information significantly better compared to redundancy hubs or heterogeneous sub-populations. Mechanistically, synergistic interactions emerge from the stimulus dependence of correlated activity between neurons. Our findings suggest a refinement of the prevailing ideas regarding coding schemes in sensory cortex: columnar populations can efficiently encode information due to synergistic interactions even when receptive fields overlap and shared noise between cells is high.

In Brief

Nigam et al. show that synergistic interactions represent an integral part of cortical computations in laminar circuits during wakefulness. Synergistic interactions allow columnar neural populations to efficiently encode sensory information even when receptive fields overlap and shared noise between cells is high.

INTRODUCTION

A prevailing view in sensory neuroscience is that nearby neurons exhibit significant redundancy in their code. This view is based on studies in a variety of animal models (Berry et al., 1997; Dan et al., 1998; Doi et al., 2012; Montani et al., 2007; Nirenberg et al., 2001; Puchalla et al., 2005; Reich et al., 2001) and across different sensory regions (Chechik et al., 2006; Reinagel and Reid, 2000) and is believed to reflect the high degree of redundancy of stimuli in the environment (Kersten, 1987; Simoncelli and Olshausen, 2001). For instance, natural scenes have strong spatial correlations that cause wide groups of cortical cells with overlapping receptive fields to participate in encoding any single individual feature present in the image. This view, which remains highly influential, has led to the idea that a major goal of sensory systems is to reduce the redundancy of neuronal responses (Attneave, 1954; Barlow, 1961). Indeed, theories of efficient coding have long predicted that sensory processing acts to reduce the redundancy of the neural code by removing correlations present between neurons (Atick and Redlich, 1992; Barlow, 1961; Field, 1989; Gutnisky and Dragoi, 2008; van Hateren, 1992; Laughlin, 1989; Rieke et al., 1995). Although redundancy may offer possible advantages (Barlow, 2001; Puchalla et al., 2005), it is achieved at a cost in coding efficiency, as more neurons participate to represent the same information than would be necessary.

Although the influential finding of redundant coding in sensory cortex may suggest that this coding scheme has been firmly established, several limitations of previous studies prevent this conclusion. First, previous work has relied on neuronal recordings in vitro in the retina (Doi et al., 2012; Puchalla et al., 2005) or in vivo in sensory cortex during anesthesia (Montani et al., 2007; Reich et al., 2001). However, when animals are anesthetized, neurons are highly correlated (Ecker et al., 2014; Poulet and Petersen, 2008; Schölvinck et al., 2015), and correlated activity has been typically linked to redundant coding schemes (Attneave, 1954; Barlow, 1961; Gutnisky and Dragoi, 2008). Second, redundancy was previously observed when multiple neurons were recorded tangentially across the cortical surface and not along the cortical column (Montani et al., 2007; Reich et al., 2001). Indeed, we confirmed previous findings and observed similar redundancy-dominated information encoding in tangential recordings from the superficial layers of macaque primary visual cortex (Figure S1). Cortical columns have been hypothesized to be the elementary functional and computational module of the neocortex (Rakic, 1988; Mountcastle, 1997), and columnar structure coupled with laminar morphology (Hubel and Wiesel, 1972; Lund and Boothe, 1975) have been intriguing subjects of study over the past several decades. Although we know much about anatomy, cell types, input-output architecture, and recurrent connectivity (Adesnik and Scanziani, 2010; Briggs and Callaway, 2005; Douglas and Martin, 2004; Markram et al., 2004; Nassi and Callaway, 2009), whether cells within a cortical column process information redundantly or synergistically is unknown. Third, the timescales used to compute redundancy in previous studies were much shorter than those normally experienced during wakefulness, such as those occurring during natural viewing when each gaze fixation lasts for hundreds of milliseconds.

We re-examined the idea of redundant coding in neocortex by measuring the amount of information about visual stimuli transmitted by neurons within a microcolumn in awake monkey primary visual cortex (V1) at a timescale relevant for visual perception. We found significant synergistic interactions between nearby neurons that were clustered non-randomly across cortical layers to form synergy and redundancy hubs. Neuronal populations comprising only synergy hubs were significantly better at decoding stimulus information compared to redundancy hubs or populations comprising both synergy and redundancy hubs. These findings suggest that the generality of the redundant coding idea in sensory cortex needs to be revisited. That is, columnar neuronal populations can efficiently encode information due to synergistic inter-neuronal interactions, even when receptive fields overlap and shared noise between cells is high.

RESULTS

Spike trains were recorded from multiple V1 neurons (Figure 1A; n = 201 single units) in two fixating monkeys using multi-contact laminar arrays advanced perpendicularly to the cortical surface. Stimuli consisted of five sine-wave gratings flashed for 300 ms and presented at equally spaced orientations within 20° of the cells’ optimal orientation. Gratings were 5° in diameter, i.e., large enough to cover the receptive fields of all the neurons recorded by the linear array (Figure 1B). Spatial frequency was chosen to maximize the number of V1 neurons that were driven in a session (average number of presentations of each stimulus per session was 84 ± 13). Because recordings were performed in a cortical column, cells had largely overlapping receptive fields and were tuned to the same stimulus orientation (Figures S2A and S2B). Representative examples of V1 responses of five single units to multiple presentations of sine wave gratings are shown in Figure 1C. From these responses, we calculated the bias-corrected amount of mutual information that spike timing and spike counts carried about stimulus orientation (STAR Methods) as a function of time, from 200 ms before stimulus onset to 1,000 ms poststimulus onset. Because the responses of cortical neurons are noisy, it is difficult to estimate the precise time of the first spike in response to the stimulus. Therefore, for each time t, we computed the information conveyed by the preceding spike and interspike interval (Figure 1D). To compare these information values with those based on spike counts, at each instant of time t, we calculated the information conveyed by spike counts in windows of different sizes: 10; 50; 100; 200; 300; and 400 ms.

Figure 1. Synergistic and Redundant Coding in Visual Cortex.

(A) Schematic of laminar recording using 16-channel U-Probe in macaque primary visual cortex.

(B) Position of receptive fields (blue circles) of neurons recorded with a laminar probe with respect to the displayed sinusoidal grating and fixation point (red dot).

(C) Raster plots of five representative cells for multiple presentations of each sinusoidal grating. The orientations of the gratings ranged from 115° to 155° in steps of 10°.

(D) Schematic of possible strategies for encoding stimulus information. A sliding time window starting from an instant t and extending to t +Δt was used to calculate the following: spike count (SC); spike time (ST): the elapsed time since the last spike with respect to t +Δt; and spike interval (SI): the inter-spike interval of the last pair of spikes corresponding to the instant t +Δt.

(E) Time course of shuffle-corrected mutual information (MI) values averaged over all pairs for different coding strategies.

(F) Average MI time courses for pairs of neurons (blue line) and independent sum of MI for two neurons (purple) and single neurons (orange). Spike counts in 300-ms time bins were used to calculate MI.

(G) Average time course of shuffle-corrected ΔI value for synergistic pairs (red-shaded line) and redundant pairs (blue-shaded line). Inset shows ΔI when averaged over all pairs. Solid line and shaded region represent mean ± SEM, respectively.

(H) Percentage of synergistic, redundant, and independent pairs obtained in the analysis.

(I) Time course of MI for synergistic and redundant pairs. Inset shows the peak value of MI (mean ± SEM) encoded on average by each class of pairs. The gray bar above the x axis indicates the duration for which the stimulus was on.

(J) Peak synergy and redundancy (mean ± SEM) calculated for different bin widths used to calculate spike counts for MI and ΔI analysis.

(K) Mean ± SEM of cortical distance between neurons forming synergistic and redundant pairs.

Despite the great variability in the amount of information conveyed by pairs of neurons, there was a clear increase in mutual information about 40 ms after stimulus onset (Figure 1E), which is expected given the time the visually evoked neural impulses need to reach primary visual cortex. Contrary to findings in the retina (Berry et al., 1997; Gollisch and Meister, 2008), spike counts were more informative than spike times and spike intervals in neuronal pairs (this was also true for single neurons; Figure S2C). Hence, we focused our subsequent analysis on the 300-ms time window, as spike counts in this window were most informative and matched the duration of the flashed stimulus (Figure S2D). As expected, the peak mutual information carried by pairs of neurons represents an almost two-fold increase in information relative to that carried by single cells (Figure 1F).

We further examined whether combinations of cells jointly convey less information (redundant coding), the same information (independent coding), or more information (synergistic coding) in their spike counts compared to the sum of the information conveyed independently by individual neurons. On average, the amount of information encoded jointly by a pair of cells was not significantly higher (p > 0.05; Wilcoxon rank-sum) than the sum of the information provided by the two neurons forming the pair (Figure 1F; the difference between these two quantities was labeled ΔI). Although this may suggest that pairs of neurons encode stimuli in an independent manner, when ΔI was analyzed separately for each pair (STAR Methods), we observed a significant number of pairs exhibiting synergistic ΔI > 0; p < 0.05; Wilcoxon signed rank test; Holm-Bonferroni correction for multiple comparisons; Figure 1G) and redundant interactions (ΔI < 0; p < 0.05; Wilcoxon signed rank test; Holm-Bonferroni correction for multiple comparisons). However, when ΔI was averaged across all the pairs, we failed to observe either net synergy or redundancy (Figure 1G, inset). Out of 1,031 pairs, 43% were synergistic, 47% were redundant, and the remaining 10% were independent (Figure 1H). Thus, contrary to previous findings originating from tangential recordings across the cortical surface (e.g., Figure S1), cortical columns in awake macaque V1 are characterized by a large proportion of synergistic pairs. Although synergistic pairs were slightly less abundant than redundant pairs, they carried significantly more information about incoming stimuli (Figure 1I; p < 0.0001; Wilcoxon rank-sum test). To examine whether these results extend beyond pairwise interactions, we calculated synergy and redundancy for triplets (n = 4,034) and quadruplets (n = 12,526; Figures S3A–S3F). Indeed, approximately 40% of triplets and quadruplets were synergistic. These synergistic groups carried more stimulus information compared to their redundant counterparts, which is consistent with the results found in pairs.

To quantify the relative strength of synergy and redundancy, we calculated the peak synergy and redundancy fraction (Montani et al., 2007; STAR Methods) for the two separate groups of pairs. We found that, on average, synergistic pairs provided approximately 45% more information than the sum of the individual information values provided by the neurons forming the pair, and redundant pairs provided approximately 40% less information. The strength of synergistic and redundant interactions gradually increased as spike counts were calculated over larger windows (Figure 1J). Contrary to previous studies, these findings provide evidence for strong synergy and redundancy, and relatively little independence, within the cortical columns of V1.

In principle, one might expect that nearby neurons within a column would have similar tuning curves, and hence, cells separated by smaller distances would mostly have redundant interactions. In that case, our results would simply reflect a distance-based encoding rule. However, we found no significant difference in cortical distance between the neurons forming synergistic and redundant pairs (Figure 1K; p > 0.05; Wilcoxon rank sum test). It might also be possible that small differences in tuning preference or mean firing rates between the cells in a pair could be associated with synergistic or redundant interactions. Therefore, we calculated the difference in tuning preferences and the mean firing rates but did not observe significant differences between the distributions for pairs corresponding to synergistic and redundant interactions (tuning preference: Figure S3G, Watson-Williams test; p > 0.05; firing rates: Figure S3H, one-way balanced ANOVA followed by multiple comparisons test; p > 0.05). Altogether, these results demonstrate that synergistic interactions cannot simply be explained in terms of spatial proximity, tuning differences, or firing rate differences between the cells in a pair.

Synergy and Redundancy Hubs

We further examined whether we can identify neurons preferentially engaging in synergistic or redundant interactions with the rest of the cells in a column or whether interactions are mixed (Figure 2A). That is, do some neurons have a higher probability than others to establish synergistic or redundant interactions? To examine this issue, we calculated the participation coefficient (PC) of a neuron as the fraction of interactions that were either synergistic or redundant. For example, a neuron with 7 synergistic and 3 redundant pairwise interactions would have a participation coefficient of 0.7 for synergy and 0.3 for redundancy. To assess the statistical significance of the PC values, we calculated the normalized participation coefficient (PC_norm) by randomly assigning the same number of redundant and synergistic interactions between neurons (STAR Methods). The normalized coefficient was obtained by dividing the actual PC for a neuron by the mean PC calculated from the randomized set. Neurons that had a participation coefficient >0.5 and normalized participation coefficient significantly >1 (see STAR Methodsfor statistical testing) were labeled as synergy or redundancy “hubs,” depending on which interaction dominated (Figure 2B). Neurons not satisfying this criterion were labeled as “non-hubs.” We found that approximately 26% of the neurons in our population were synergy hubs, whereas 37% were redundancy hubs. This demonstrates that synergistic and redundant interactions do cluster such that some neurons preferentially participate in one or the other type of interaction with their neighbors. However, there was no significant difference in mean cortical distance (Figure 2C) between the neurons forming synergy and redundancy hubs (p > 0.05; Wilcoxon rank-sum test).

Figure 2. Synergy and Redundancy “Hubs”.

(A) Schematic of neuron classification into two functional classes: synergy and redundancy hubs. Top represents a neuron (filled red triangle) that has synergistic interactions (red bands) with all other neurons (synergy hub). Bottom represents a neuron (filled blue triangle) that has redundant interactions (blue bands) with all other neurons (redundancy hub).

(B) Scatterplot of the participation coefficient (PC) and its value normalized with respect to chance (PC_norm) for all the recorded units. Each circle represents a single neuron. Filled red circles represent synergy hubs, filled blue circles represent redundancy hubs, and open gray circles are non-hubs.

(C) Mean distance between neurons forming synergy (red) and redundancy hubs (blue). Error bars represent SEM.

(D) Schematic representation of three types of sub-populations used for the decoding analysis. Homogeneous populations were constructed either with 100%synergy (red) or 100% redundancy hubs (blue). Heterogeneous populations were constructed by combining varying numbers of synergy and redundancy hubs.

(E) Time course of decoder accuracy above chance levels (20%) for a sub-population comprising 7 neurons with varying percentages of synergy and redundancy hubs (different colors) for a representative session. Each line represents the mean; shaded region represents SEM.

(F) Normalized peak decoder accuracy averaged over all sessions for sub-populations with varying percentages of synergy hubs. Solid red line is a fit generated with a smoothing spline. Error bars represent SEM.

What could be the functional role of such hubs? To answer this question, we built sub-populations of neurons consisting of varying numbers of synergy and redundancy hubs (STAR Methods) and examined their impact on stimulus decoding. For instance, to construct a homogeneous sub-population consisting of 5 neurons, all five neurons were randomly sampled from the pool of either synergy or redundancy hubs in a particular session. Heterogeneous sub-populations of five neurons were constructed by selecting different numbers of synergy and redundancy hubs, e.g., 3 synergy hubs were combined with 2 redundancy hubs or 1 synergy hub was combined with 4 redundancy hubs, etc. (Figure 2D). All possible drawings of neurons from the synergy and redundancy pools were used to construct these sub-populations. We subsequently used a linear discriminant classifier to decode stimulus identity using the spike count responses of the neurons in each sub-population. We normalized decoder accuracy with respect to that of sub-populations containing only redundancy hubs for each session. There was an increase in peak decoder accuracy as the percentage of synergy hubs was increased (Figure 2E shows one example session). The lowest decoder accuracy was obtained for homogeneous populations exclusively consisting of redundancy hubs (Figure 2E). To compare across sessions, we normalized decoder accuracy for each sub-population (homogeneous or heterogeneous) by the lowest performance (hence, the normalized decoder accuracy had values >1 for different sub-populations). Across sessions (Figures S3I–S3K), there was a clear increase in mean normalized decoder accuracy with the increase in the number of synergy hubs in the populations (Figure 2F; n = 11 sessions; p < 0.001; one-way balanced ANOVA followed by multiple comparisons test). Normalized decoder accuracy was not significantly greater than 1 (p > 0.05; t test) for sub-populations where less than half of neurons were synergy hubs. There was a 20% increase in decoder accuracy between sub-populations with only synergy compared to only redundancy hubs. Altogether, these results indicate that there is a specialized subgroup of neurons encoding more stimulus information, and these cells tend to form hubs of synergistic interactions.

Next, we examined the nature of the interactions between synergy and redundancy hubs (Figure 3A). Specifically, are synergy-synergy hub interactions predominantly synergistic or redundant? Across sessions (STAR Methods), we found that the probability that synergy hubs interact synergistically with each other is 75% ± 5%, and the probability that redundancy hubs interact synergistically is only 26% ± 6% (Figure 3B). However, one might expect that, by chance, a synergy hub would interact primarily through synergistic interactions with other synergy hubs. Therefore, the probability of a synergistic interaction between two synergy hubs would be high, and the same could be expected for redundancy hubs. To control for this possibility, we calculated the probability of different types of hub-hub interactions by shuffling the interaction labels across the population of pairs while holding the total number of synergistic or redundant interactions for each neuron the same as in the actual data. From these “randomized” networks, we calculated a distribution of interaction percentages for each type of hub-hub interaction, and if the actual interaction percentage was greater than the mean of this distribution (p < 0.05; right-tailed Wilcoxon sign rank test), that session was included for analysis (STAR Methods). Across sessions, the probabilities of 2^nd-order interaction between hubs were statistically different from 50% (p < 0.001; two-tailed Wilcoxon sign rank test). However, interactions between synergy and redundancy hubs are equally likely to be synergistic or redundant (not significantly different from 50%; two-tailed Wilcoxon sign rank test; p > 0.05). This analysis reveals a higher degree of organization of hub networks whereby synergistic-synergistic and redundancy-redundancy interactions are more prevalent between hubs than mixed interactions.

Figure 3. Synergistic Interactions between Synergy Hubs.

(A) Schematic of interactions between synergy hubs and redundancy hubs.

(B) Percentage of synergistic interactions between various hub types: synergy-synergy hubs (red circles); redundancy-redundancy hubs (blue circles); and mixed synergy-redundancy hubs (magenta circles). Each circle corresponds to an individual session. Filled black circles and error bars represent the mean and SEM across sessions. The asterisks denote statistically significant differences of mean values from 50% (filled black circles; Wilcoxon sign rank test; p < 0.001).

Synergistic and Redundant Interactions Are Present across All Cortical Layers

We further examined whether synergistic or redundant interactions are confined to a specific cortical layer or they are spread out across the entire cortical column. We thus performed current source density (CSD) analysis (Hansen and Dragoi, 2011; Hansen et al., 2012; Schroeder et al., 1998; STAR Methods) to identify the polarity of the inversion accompanied by the sink-source configuration at the base of the granular (G) layer and then assigned electrode contacts above and below the sink to supragranular (SG) and infragranular (IG) layers (Figure 4A). We calculated the degree of synergy and redundancy for cell pairs within (intra) and between layers (inter). In both cases, averaging the ΔI values for all the pairs within a group yielded a mean that was not significantly different from zero (p > 0.05; t test) to indicate independent encoding of information (Figure 4B, inset). However, when pairs were analyzed separately within each group, we mostly found synergistic (ΔI > 0; p < 0.001; Wilcoxon signed rank test) and redundant (ΔI < 0; p < 0.001; Wilcoxon signed rank test) rather than independent interactions (Figures 4C and 4D) across the layers of V1. Peak values of synergy and redundancy (Figures 4C and 4D, insets) were not significantly different across inter- or intra-layer pairs (p > 0.05; one-way balanced ANOVA). The presence of synergy and redundancy in inter-layer pairs suggests that these interactions are not confined locally within layers but extend across layers for distances up to hundreds of microns.

Figure 4. Laminar Distribution of Synergy and Redundancy.

(A) Schematic of laminar probe used for recording neural activity along a cortical column. The probe has 16 contacts with 100-μm inter-contact spacing. Representative example of current source density (CSD) analysis for identifying layers is shown.

(B) Time course of ΔI values averaged across all pairs within each respective layer and for inter-layer pairs. Inset shows peak ΔI values for each layer and for inter-layer pairs.

(C) Time course as in (B) but only for synergistic pairs within each layer and across layers. Inset shows peak ΔI values.

(D) Same as in (C) but for redundant pairs. Solid lines represent mean values, and shaded region represents SEM.

(E) Percent of synergistic and redundant pairs in each layer out of the total number of pairs in that layer.

(F) Percent of synergy and redundancy hubs within each layer out of the total number of neurons in each layer.

Additional synergy and redundancy analysis was performed by separating our inter-layer pairs into SG-G, IG-G, and G-IG pairs (Figures S4A–S4D). Inter-layer pairs between neurons in superficial (SG) and deep (IG) layers had higher synergy (p < 0.001; one-way balanced ANOVA followed by multiple comparisons test) compared to inter-layer pairs between neurons in superficial (SG) and granular (G) layers. However, there was no difference in redundancy values for different classes of inter-layer pairs. The percentage of synergistic or redundant pairs and synergy or redundancy hubs were roughly the same in the supragranular, granular, and infragranular layers (Figures 4E and 4F). Further, there was no significant difference in the peak information encoded by either intra- or inter-layer cell pairs (Figures S4E and S4F; p > 0.05; one-way balanced ANOVA), which indicates that information processing is relatively uniform across cortical layers. These analyses reveal that synergistic and redundant interactions constitute a general property of V1 cortical microcolumns.

Stimulus-Dependent Correlated Activity Gives Rise to Synergistic Interactions

To explore the sources of synergistic and redundant interactions between cortical neurons (Magri et al., 2009; Pola et al., 2003), we decomposed synergy and redundancy into components that reflect how signal and noise correlations separately influence stimulus coding. Specifically, the difference (ΔI) between the joint mutual information encoded by a pair of cells and the sum of the information transmitted by each cell can be decomposed into components determined by signal similarity (I^ss), stimulus-independent correlations (I^CI), and stimulus-dependent correlations (I^CD) (STAR Methods; Equation 6). We calculated the time course of each of these components and averaged them across pairs (Figure 5A), irrespective of whether they were synergistic or redundant. We also separated the cell pairs into synergistic and redundant groups and then evaluated the time course of these components (Figures 5B–5D). Signal similarity quantifies the amount of redundancy due to the similarity of neuronal responses across stimuli. This component can only contribute to redundancy in information encoding. However, we found that signal similarity did not play a significant role in generating the observed redundancy in these populations (I^ss not significantly less than 0; p < 0.05; Wilcoxon signed rank test; Figure 5B).

Figure 5. Stimulus-Dependent Correlations Lead to Synergistic Interactions.

(A) Time course of information breakdown of synergy or redundancy term ΔI in terms of signal similarity: I^SS (green; redundancy caused by similarity of responses between neurons), stimulus-independent correlations: I^CI (purple), and stimulus-dependent correlations: I^CD (orange) averaged over all pairs.

(B) Time course of the contribution of signal similarity to synergy and redundancy obtained by averaging I^SS separately for synergistic (red trace) and redundant pairs (blue trace).

(C) Time course of the contribution of stimulus-independent correlations for synergistic and redundant pairs.

(D) Same as in (C) except for stimulus-dependent correlations.

The effect of correlated activity between neurons on information processing was quantified by the stimulus-independent and stimulus-dependent components of information. The stimulus-independent component is positive when the cross-correlation between neurons and signal similarity have opposite signs and vice versa. We found that, although stimulus-independent correlations did not contribute to synergistic interactions (I^CI not significantly different from 0 for synergistic pairs; p > 0.05; Wilcoxon signed rank test), it did contribute to redundancy (I^CI significantly <0 for redundant pairs; p < 0.05; Wilcoxon signed rank test; Figure 5C). The stimulus-dependent component of information is non-zero when the strength of correlations between neurons is modulated by the stimulus, and it is zero otherwise. We found that only synergistic pairs had a non-negative I^CD (p < 0.05; Wilcoxon signed rank test), indicating that the synergy observed in this population originates from the stimulus-dependent noise correlation component (Figure 5D). Thus, even though neurons within a cortical column have overlapping receptive fields and similar orientation preference, the stimulus dependence of correlated activity gives rise to efficient synergistic computations.

DISCUSSION

A prevailing view in sensory systems is that nearby neurons exhibit significant redundancy in their code, mainly originating from the high degree of redundancy of natural stimuli. This view has significantly influenced research in sensory neuroscience, which has postulated that a major goal of sensory systems is to reduce the redundancy of neuronal responses. In contrast to previous studies, our results reveal a surprisingly high degree of synergy within V1 cortical columns during wakefulness when stimuli are presented at the timescale of visual fixation. Synergistic interactions are clustered non-randomly across cortical layers to form synergy hubs. Synergy is beneficial for stimulus coding, as sub-populations of synergy hubs perform significantly better than redundant or mixed synergy and redundancy hubs. Our findings have implications for the coding schemes employed by columnar neuronal populations, which have the ability to efficiently encode information due to synergistic interactions.

In the visual system, most previous studies measuring the amount of information transmitted by sensory neurons have been performed in the retina and lateral geniculate nucleus (LGN) in a variety of model systems (Brenner et al., 2000; Dan et al., 1998; Meister et al., 1995; Nirenberg et al., 2001; Puchalla et al., 2005). The macaque visual cortex has been the focus of only a handful of studies (Montani et al., 2007; Reich et al., 2001) that reported mostly redundant and independent interactions. Contrary to previous results, we found significant synergy within columnar circuits, despite the fact that neurons have largely overlapping receptive fields and similar tuning preferences. This could be due to several key differences from prior work, such as (1) previous recordings were performed in a two-dimensional plane along the cortical surface using tetrodes or microelectrodes (Figure S1), whereas we focused on recordings across the entire depth of a cortical column. It might seem contradictory that tangential recordings that involve neurons in superficial layers do not show synergy, whereas superficial layers in laminar recordings do. This may be due to an important difference between columnar and lateral intracortical circuits. That is, although the neurons recorded tangentially reside in superficial layers, they belong to different cortical columns (inter-electrode spacing up to 4 mm) and hence different preferred orientations. In that case, neuronal responses are primarily influenced by lateral (short- and long-range cortical connections). On the other hand, the neurons in superficial layers recorded by our laminar array have identical stimulus preference and overlapping receptive fields, and connectivity is dominated by short-range vertical intracortical connections (less than 1 mm). Differences could also arise from the stimulus dependence of correlated activity. In agreement with studies performed in the retina (Franke et al., 2016; Zylberberg et al., 2016) and cortex (Kohn and Smith, 2005; Kreiter and Singer, 1996), columnar V1 neurons sharing the same preferred orientations have strong stimulus-dependent correlations, which are linked to synergistic interactions. Consistent with previous studies in V1, neurons recorded along the cortical surface exhibit correlations that show only weak stimulus dependence (Gutnisky and Dragoi, 2008; Smith and Kohn, 2008). (2) Our recordings were performed in awake animals, whereas previous studies used anesthetized animals. There is considerable evidence that population activity and correlations between neurons depend on cortical state (Ecker et al., 2014; Poulet and Petersen, 2008; Schölvinck et al., 2015). Because synergy and redundancy depend on the strength of correlated activity (Figure 5), global brain state could play a significant role in determining whether interactions are synergistic or redundant, and (3) we measured synergy and redundancy for spike counts at the timescale of visual fixation (Dragoi and Sur, 2006; Dragoi et al., 2002) in 300-ms bins, i.e., a larger scale than the time intervals used in prior studies.

One of our novel results is the diversity of neuron classes in V1, ranging from cells engaged in strongly synergistic interactions (synergy hubs) to cells engaged in strongly redundant interactions (redundancy hubs). These sub-populations differ significantly in their stimulus decoding accuracy: sub-populations consisting of synergy hubs offer superior decoding performance compared to sub-populations of redundancy or mixed hubs. This functional sub-division into neuronal functional groups within the cortical microcolumn comprising neurons with similar stimulus preferences is surprising. Additionally, synergy hubs were present in all cortical layers, hence suggesting that synergistic coding constitutes a general strategy employed within cortical columns. Given that neuronal populations comprising only synergy hubs are significantly better at decoding stimulus information compared to redundancy hubs, what could be the advantage of encoding schemes in which both synergistic and redundant interactions are present? We reasoned that, although there is a significant reduction of redundancy from retina to the cortex (Laughlin, 1989; Atick and Redlich, 1992; van Hateren, 1992), a large fraction of pairwise intracortical interactions remain redundant. Hence, redundancy can be viewed as a necessary consequence rather than a limitation. Redundancy is functionally important in that it allows a dense sampling of stimuli by neurons with overlapping receptive fields. This oversampling of visual information could be necessary for error correction (Hamming, 1950) in the case of noisy stimuli to facilitate the extraction of behaviorally relevant information. Furthermore, redundant coding may compensate for unwanted effects of synaptic unreliability to ensure robust and precise firing rates. Given the importance of both synergistic and redundant coding schemes, columnar circuits may be designed to operate in a regime balancing the efficiency of synergistic interactions and the robustness of redundant interactions.

One issue of great interest is the functional coupling between synergy and redundancy hubs. Past studies have shown the existence of rich clubs (Colizza et al., 2006; van den Heuvel and Sporns, 2011; Nigam et al., 2016) in the functional architecture of brain networks. Thus, neurons or brain regions acting as hubs of functional connectivity preferentially connect to similar hubs to form a densely connected sub-network defined as a rich club. These specialized networks have been shown to play an important role in the integration and transfer of information across brain regions (Faber et al., 2019) and regulating network dynamics (Senden et al., 2014), which are important requirements of efficient information processing. Our analysis shows that synergy hubs interact synergistically with other synergy hubs, and redundancy hubs interact redundantly with other redundancy hubs. This raises the intriguing possibility that synergy hubs may be part of a rich club network critically involved in efficient information processing relevant for perception and cognition.

One could argue that the raw ΔI values of synergy and redundancy measured in columnar networks (Figure 1G) are too small to significantly impact information coding. First, it is noteworthy that the ΔI values are small because the recorded columnar neurons have very similar tuning preferences, which implies homogeneous tuning curves for the recorded population. Second, we computed mutual information using stimuli centered close to the preferred orientation of the cell (within 20°). Hence, the range of stimulus-evoked responses of our recorded cells is small compared to that associated with the full range of orientations spanning 0°–180°. As a result, the mutual information values were low, and thus synergy and redundancy were low too. However, relative to the mutual information encoded, the values for synergy and redundancy were not small. That is, synergistic interactions provided 45% more information than the sum of information transmitted by individual neurons. Previous work in neocortex (Montani et al., 2007) has estimated this value to be around 13.7% and concluded that synergy was very weak. Thus, our values are more than three times higher than previously reported. Furthermore, we show that even small changes in mutual information impact stimulus coding, as our decoder analysis reveals that synergy hubs provide more information than redundancy hubs. We also show that synergistic pairs, triplets, and quadruplets carry significantly higher information than their redundant counterparts (Figures S3D and S3F).

Our study uses synthetic stimuli (oriented gratings) for activating V1 neurons, but could the implications of our work be extended to feature encoding during natural viewing? There are two important distinctions between our stimuli and those encountered during natural viewing. First, stimuli viewed in natural conditions have a “full field” spatial context, whereas the gratings used in our experiments are restricted in diameter. When stimuli extend well beyond the classical receptive fields of V1 neurons, visual surround modulates neuronal responses to reduce firing rates (Angelucci et al., 2017) and correlations (Snyder et al., 2014; Vinje and Gallant, 2002) and increase sparseness, such as to improve the efficiency of information transmission (Vinje and Gallant, 2002). However, it is unknown whether and how contextual modulation of neuronal responses during natural viewing influences the stimulus dependence of correlated activity and, hence, synergy and redundancy. Second, natural viewing is a history-dependent process by which individual neuron and population responses adapt at the timescale of visual fixation (Dragoi et al., 2002; Gutnisky and Dragoi, 2008). Although adaptation reduces neuronal correlations (Gutnisky and Dragoi, 2008), how temporal context influences the degree of synergy and redundancy in V1 columnar circuits remains unknown. Future studies can further elucidate the role of spatial and temporal context of visual information in determining the relative balance of synergy and redundancy in columnar circuits. It also remains to be seen whether other sensory cortical areas exhibit a similar degree of synergy as that found in V1.

STAR★METHODS

LEAD CONTACT AND MATERIALS AVAILABILITY

Further information and requests should be directed to and will be fulfilled by the Lead Contact, Valentin Dragoi (Valentin.Dragoi@uth.tmc.edu).

EXPERIMENTAL MODEL AND SUBJECT DETAILS

All experiments were performed in accordance with protocols approved by the U.S. National Institutes of Health Guidelines for the Care and Use of Animals for Experimental Procedures and were approved by the Institutional Animal Care and Use Committee at the University of Texas Health Science Center at Houston. Data from two adult male rhesus macaques (Macaca mulatta) were collected for this study.

METHOD DETAILS

Experimental protocol

Two male rhesus macaques (Macaca mulatta), monkeys C and W, were trained to fixate on a small fixation point (0.2 deg) in the center of a cathode ray tube (CRT) monitor while the head was fixed. After 200 ms of fixation, a visual stimulus was presented for 300 ms. If monkeys break fixation anytime during this 500 ms period, the trial was aborted. Monkeys were rewarded with juice if they held fixation for the entire duration of the trial. Stimuli were generated with Psychophysics toolbox using MATLAB. Static 5-deg circular sine-wave gratings of 5 different orientations were flashed at the center of the receptive fields for 300 ms. During each trial we presented a single grating chosen randomly from one of the five different orientations. The 5 orientations displayed during the trials were chosen such that they covered a range of ± 20° in steps of 10° centered on the preferred orientations of the cells recorded in that session. The location of the presented stimulus was chosen so that it covered the position of most of the receptive fields. The average number of trials recorded per session was 424 ± 68.

Eye-movement control

Monkeys were trained to fixate on a dot (0.2 deg in size) at the center of the screen within a small rectangular window of size 1 deg. Eye position was constantly monitored by using an eye tracker operating at 1 KHz (EyeLink II; SR Research). If at any point during the trial, eye position exceeded 0.25 deg outside the boundaries of the rectangular box, then the trial was automatically aborted. This ensured that our trials were not contaminated by neural activity corresponding to large-scale fluctuations in eye movements.

Electrophysiological recordings

We recorded extracellular activity from populations of V1 neurons simultaneously across all layers, using laminar electrodes with 16 contacts (Plextrode ® U-Probe, Plexon Inc). These contacts were separated by 100 μm. We recorded both spiking activity and LFP signals by means of a Multichannel Acquisition Processor System (MAP, Plexon Inc) at a sampling rate of 40 KHz. Spike waveforms above threshold (~4 sd above the amplitude of the noise signal) were saved and sorted post data acquisition using Plexon’s Offline Sorter. Spike waveforms were manually sorted with Plexon’s offline sorter program using waveform clustering parameters such as spike amplitude and width, timing of the valley and peak. Clusters that clearly separated from the origin of the principal component space and from other clusters were classified as single units to be used for further analysis. Single units were subsequently analyzed using custom scripts in MATLAB. Post-sorting, we calculated auto-correlograms and cross-correlograms (Figures S5A–S5C) with the shift correction technique to further eliminate cases whereby a single neuron is recorded as two separate units on two different channels, hence avoiding an over-estimation and duplication of the units used in the analysis. For all the recorded neurons used in the analysis we compared the auto and cross correlogram values at 0 time lag (Figure S5C), and found that the distributions are significantly different (KS test; p < 0.001). The 0 time lag values in auto-correlograms were almost 100 times higher than the values in the cross-correlograms. This analysis shows that the neurons recorded on different electrodes were indeed individual single units and not multiple copies of each other. After a unit was isolated, its receptive field was mapped with dynamic gratings or using reverse correlation while the animal performed a passive fixation task. Reverse correlation stimulus consisting of 8 different orientations (equally spaced from 0 to 180°) were presented over multiple trials to extract the preferred orientation of the recorded units.

Only the units with significant modulation of responses during stimulus presentation compared to the pre-stimulus period were considered for analysis. We verified the verticality of laminar array penetrations by analyzing the receptive field maps and tuning curves of the neurons (Figures S2A and S2B). We calculated the pairwise difference in receptive field centers across the cells used in our analysis, and found a mean difference between receptive field centers < 0.5 deg (Figure S5D, Wilcoxon signed rank test; p < 0.001). We also calculated the standard error of mean (s.e.m) of preferred orientation for all the neurons recorded in a session. When averaged across sessions, s.e.m was 3.2 ± 0.2° (Figure S5E), which is significantly < 4° (Wilcoxon signed rank test; p < 0.001), in agreement with our previous work (Hansen and Dragoi, 2011; Hansen et al., 2012). The highly overlapping nature of the receptive fields and similarity of preferred orientations indicate that the neurons used in our analysis were recorded from the same cortical column. We eliminated sessions in which we found significant drifts (upward or downward) in the average population activity over the time course of the recording. Our analysis of spike waveform characteristics and firing rates indicates that the majority of the units that we recorded were putative excitatory cells.

Current source Density Analysis (CSD) for cortical layer identification

For each recording session we identified visual cortical layers using CSD analysis. We recorded LFP data using laminar probes while presenting a full field high contrast natural image. We corrected the filter-induced timing delays of LFP data by using the FPAlign utility from Plexon Inc. We then filtered the LFP channels with a band pass filter (0.5–100 Hz) and we applied an 8^th order Butterworth notch filter at 60 Hz. We averaged the filtered LFP signals across trials to obtain the evoked response potential (ERP) for each channel. We computed the current source density by using the second spatial derivative of the LFP time-series across equally spaced laminar contacts using the iCSD toolbox for MATLAB (Pettersen et al., 2006). The centroid of the primary sink observed in the CSD served as reference for the granular layer. All the contacts above and below this sink were analyzed and classified into one of three groups: supragranular (SG), granular (G) and infra granular (IG).

Information theoretic analysis

We examined spike count and spike timing-based coding schemes. For spike count schemes we considered the total spike counts in bins of size 10, 50, 100, 150, 200, 250, 300, 350, and 400 ms, and used a 10 ms sliding window. We performed the sliding window analysis starting from 200 ms prior to stimulus onset to 1000 ms post stimulus onset. For spike timing schemes, we considered spike time (ST) and spike interval (SI). ST at instant t was defined as the time of the last spike before time t. SI was defined as the inter-spike interval corresponding to the last two spikes with respect to time t.

All information theoretic quantities such as mutual information (MI), ΔI (synergy/redundancy), were evaluated using the Information Breakdown Toolbox (Magri et al., 2009). The Panzeri-Treves bias correction (Panzeri and Treves, 1996) method implemented in the toolbox was used to account for the bias in mutual information estimation caused by limited sampling of neuronal responses. Using the toolbox, we also calculated for each information theoretic measure 1000 bootstrap estimates. The bootstrapped estimates were calculated by randomly shuffling the neural responses evoked by the stimulus. The shuffling destroys the stimulus dependency of neural activity and provided an estimate of these quantities at chance level. The values reported in our analysis were obtained by subtracting the mean of the bootstrapped estimates from the bias corrected raw estimates. The following are the information theoretic quantities used in our analysis:

$$MI(S;R)=H(R)-H(R|S)$$ (1)

$$H(R)=-\sum _{r}P(r)lo{g}_{2}P(r|s)$$ (2)

$$H(R|S)=-\sum _{s=1}^{N_s}P(s)\sum _{r}P(r|s)lo{g}_{2}P(r|s)$$ (3)

Mutual information (Equation 1) is defined as the difference between the response entropy H(R) (Equation 2) and the noise entropy H(R|S)(Equation 3). P(r) is the probability that the neuronal response is r, P(r | s) is the conditional probability of responser given stimuluss. By definition MI > 0; however, since we reported bootstrap subtracted raw values, there were certain points along the time-course where the values were negative. This simply implies that the actual raw values are smaller than chance values.

$$\mathrm{\Delta }I=MI(S;R)-M{I}_{lin}$$ (4)

ΔI is the difference between the mutual information provided by the joint responses of N neurons and the sum of the information provided by each of the N neurons MI_lin. ΔI > 0 implies that the joint responses provide more information than the linear sum of the information provided by each single neuron, and this indicates the presence of synergistic interactions. On the other hand, ΔI < 0 implies that the joint responses provide less information than the linear sum of the information provided by each single neuron, indicating the presence of redundant interactions. If ΔI = 0 there is no loss or gain of information by considering the joint responses compared to individual responses. To classify whether a pair is synergistic or redundant we performed a right/left-sided Wilcoxon signed rank test on the distribution of values of ΔI in the time window starting at stimulus onset and extending 600 ms post stimulus onset (approximate time for which mutual information is greater than chance levels, see Figure 1A). The obtained p values of the above statistical test was then passed through a Holm-Bonferroni test to obtain corrected p values accounting for multiple comparisons. If ΔI was significantly greater than 0 (p < 0.05), the pair was classified as synergistic, if ΔI was significantly less than 0, then it was classified as redundant and finally if ΔI was not significantly different from 0, the pair was classified as independent. To quantify relative strengths of synergy and redundancy, we normalized the peak synergy and peak redundancy values between 0 and 600 ms by the mutual information of that particular pair (Montani et al., 2007),

$$\mathrm{\Delta }{I}_{norm}=\frac{MI(S;R)-M{I}_{lin}}{MI(S;R)}$$ (5)

ΔI_norm is positive for synergistic interactions and in that case it was labeled as the Synergy Fraction (SF). Redundant interactions have a negative ΔI_norm in which case it was labeled as Redundancy Fraction (RF). We calculated ΔI_norm separately for synergistic pairs and redundant pairs and we reported the mean for each category of pairs.

Information breakdown analysis (Pola et al., 2003) was performed to assess the role of correlations in the generation of synergy and redundancy. ΔI can be broken down into the following components:

$$\mathrm{\Delta }I=I^{ss}+I^{CI}+I^{CD}$$ (6)

I^ss represents the contribution to ΔI that arises due to the similarity in the distribution across stimuli of stimulus-conditional response probabilities of individual neurons. This component can be present even in the absence of correlations between neurons and is always less than or equal to zero. I^CI represents the contribution of stimulus-independent correlations to ΔI. I^CI is positive if signal and noise correlations have the opposite sign and negative otherwise. I^CD is the contribution of stimulus dependent correlations to ΔI. This term is strictly non-negative and is zero only if the stimulus does not modulate the correlations between cells. Just as in the case of mutual information and ΔI, we calculated bias corrected and shuffle corrected estimates of the information breakdown terms for spike counts in 300 ms time bins using the information breakdown toolbox. Although I^CD is strictly non-negative, the shuffle corrected values for I^CD could be negative if the shuffled values are greater than the actual raw values.

Participation coefficient

To classify neurons into synergy or redundancy hubs, we calculated participation coefficients for synergistic (PC_syn) and redundant interactions (PC_red) for each neuron. For instance, let’s consider a group of 10 neurons. Imagine a 10 × 10 symmetric interaction matrix (i) where the ij th entry of the matrix is +1 if there is a synergistic interaction between neuron i and j. On the other hand, if there is a redundant interaction between neuron i and j, the ij th entry is set to −1. For an independent interaction between pairs the corresponding matrix entry is set to 0. This yields a symmetric 10 × 10 matrix that summarizes the structure of interactions between all possible pairs. PC_syn was calculated by dividing the number of synergistic interactions a neuron had, by the total number of possible interactions that the neuron can have with the rest of the recorded neurons in a session. Participation coefficient for redundant interactions was calculated in a similar way, except considering the redundant interactions of a neuron. To generate values of PC expected by chance we created a new shuffled 10 × 10 matrix by assigning the same number of +1 s (synergistic interactions) and −1 s (redundant interactions) at random. This creates a shuffled interaction matrix which has the same number of interactions of each type, but assigned randomly between pairs. One can calculate the participation coefficient for synergistic and redundant interactions for each neuron in this new shuffled matrix. Multiple realizations of the shuffled interaction matrix (N = 500) can then be used to create a distribution of participation coefficient values expected from random assignment of synergistic and redundant interactions without any structure. If there is non-random structure in the actual data, the participation coefficients should be significantly higher than those calculated from the shuffled realizations of the interaction matrix. A distribution of PC_norm values was generated by dividing the actual PC value and the shuffled PC values. A Wilcoxon signed rank test was used to ascertain whether the median of the distribution was greater than 1. Neurons with PC > 0.5 and PC_norm significantly > 1 were assigned as synergistic or redundant hubs depending upon which interaction dominated. Hence, in our case a neuron is a synergy hub not only if it has more synergistic interactions compared to redundant, but also if its participation coefficient is significantly higher than what would be expected by chance for a neuron with the same number of synergistic and redundant interactions.

Hub interaction analysis

For each session we examined the pairwise interactions between synergy and redundancy hubs. The percentage of synergistic interactions was calculated by dividing the number of synergistic interactions within the group by the total number of possible interactions. The same analysis was done for within-group interactions for redundancy hubs. We also calculated the percentage of synergistic interactions between synergistic and redundancy hubs. To control for the fact that by chance the probability of synergistic interactions within synergy hub groups and redundant interactions between redundancy hubs groups would be high we calculated chance values of the % synergistic/redundant interactions in randomized interaction networks. To construct a randomized synergistic interaction matrix, we randomly assigned synergistic interactions between pairs while keeping the total number of synergistic interactions for each neuron constant. Hence the identity of the hubs was retained even post randomization. The percentage of synergistic interactions was then calculated for this randomized network. A distribution of randomized values was calculated by repeating the randomization process multiple times (n = 500). If the actual percentage exceeds the median of the randomized distribution (Wilcoxon signed rank test; p < 0.05), then that session was used for the analysis, otherwise it was rejected. Finally the distribution of values obtained for all sessions was tested against 50% using a Wilcoxon signed rank test to conclude whether the interaction percentage was significantly less than or greater than 50%.

Decoder analysis

We created sub-populations of different sizes with synergy and redundancy hubs for each session. The size of the sub-population depended on the number of synergy and redundancy hubs found in that session. For a fixed sub-population size, the number of synergy hubs and redundancy hubs was varied such that there were sub-populations with only redundancy hubs, only synergy hubs, or a mix of synergy and redundancy hubs in different proportions. Thus, we created homogeneous and heterogeneous populations. We explored all possible combinations of synergy and redundancy hubs for creating the heterogeneous sub-populations. We decoded neural activity by using a multi-class linear discriminant analysis (Averbeck et al., 2003; Pesaran et al., 2002). The classifier was trained on 70% of the trials and performance was tested on the remaining 30%. We repeated this classification process 20 times with different training and testing sets drawn randomly from the trials from a session. Decoder performance was averaged over all possible realizations of the sub-populations of a fixed size and over the number of iterations. The responses used for decoding were spike counts in 300 ms bins and we employed the sliding window approach as described in the previous section to generate decoder accuracy as a function of time. The maximum decoding accuracy above chance levels from this time course was used to quantify the decoding power of a sub-population. Finally, to account for sub-populations with different number of neurons from different sessions, we calculated the normalized decoding accuracy which was obtained by dividing the decoder accuracy of a sub-population by the decoding accuracy of the homogeneous population containing only redundant hubs.

QUANTIFICATION AND STATISTICAL ANALYSIS

We used non-parametric statistical tests throughout the analysis as mentioned in the main text. Specifically, for synergy and redundancy identification, Wilcoxon signed rank test was used. In other cases, we used Wilcoxon rank sum test for significance analysis. We applied the Holm-Bonferroni correction wherever multiple comparisons were performed, as mentioned in the main text.

DATA AND CODE AVAILABILITY

Information theoretic calculations were performed using the information breakdown toolbox which is available at https://github.com/sunnyneuro/Information_Breakdown_Toolbox.git

KEY RESOURCES TABLE.

REAGENT OR RESOURCE	SOURCE	IDENTIFIER
Experimental Models: Organisms/Strains
Macaque monkeys	Oregon Natl. Primate Research Center	Macaca mulatta
Software and Algorithms
Stimulus control	Psychtoolbox	http://psychtoolbox.org/
MATLAB	MathWorks	https://www.mathworks.com/
Information Breakdown Toolbox	Magri et al., 2009	https://github.com/sunnyneuro/Information_Breakdown_Toolbox.git
Other
16 channel U-Probes	Plexon	https://plexon.com/
Eye tracker	Eye-Link	https://www.sr-research.com

Highlights.

• Strong synergistic interactions underlie cortical computations in laminar circuits

• Non-random clustering of synergistic interactions form synergy hubs

• Synergistic interactions distributed uniformly throughout the cortical column

• Stimulus-dependent correlations give rise to synergy in laminar cortical circuits