Visual stimuli recruit intrinsically generated cortical ensembles

Significance

This study demonstrates that neuronal groups or ensembles, rather than individual neurons, are emergent functional units of cortical activity. We show that in the presence and absence of visual stimulation, cortical activity is dominated by coactive groups of neurons forming ensembles. These ensembles are flexible and cannot be accounted for by the independent firing properties of neurons in isolation. Intrinsically generated ensembles and stimulus-evoked ensembles are similar, with one main difference: Whereas intrinsic ensembles recur at random time intervals, visually evoked ensembles are time-locked to stimuli. We propose that visual stimuli recruit endogenously generated ensembles to represent visual attributes.

Abstract

The cortical microcircuit is built with recurrent excitatory connections, and it has long been suggested that the purpose of this design is to enable intrinsically driven reverberating activity. To understand the dynamics of neocortical intrinsic activity better, we performed two-photon calcium imaging of populations of neurons from the primary visual cortex of awake mice during visual stimulation and spontaneous activity. In both conditions, cortical activity is dominated by coactive groups of neurons, forming ensembles whose activation cannot be explained by the independent firing properties of their contributing neurons, considered in isolation. Moreover, individual neurons flexibly join multiple ensembles, vastly expanding the encoding potential of the circuit. Intriguingly, the same coactive ensembles can repeat spontaneously and in response to visual stimuli, indicating that stimulus-evoked responses arise from activating these intrinsic building blocks. Although the spatial properties of stimulus-driven and spontaneous ensembles are similar, spontaneous ensembles are active at random intervals, whereas visually evoked ensembles are time-locked to stimuli. We conclude that neuronal ensembles, built by the coactivation of flexible groups of neurons, are emergent functional units of cortical activity and propose that visual stimuli recruit intrinsically generated ensembles to represent visual attributes.

There is a growing consensus in neuroscience that ensembles of neurons working in concert, as opposed to single neurons, are the underpinnings of cognition and behavior (1–3). At the microcircuit level, the cortex is dominated by recurrent excitatory connections (4, 5). Such densely interconnected excitatory networks are ideal for generating reverberating activity (1, 6, 7) that could link neurons into functional neuronal ensembles. Moreover, most cortical neurons are part of highly distributed synaptic circuits, receiving inputs from and projecting outputs to, thousands of other neurons (8, 9). In fact, the basic excitatory neurons of the cortex, pyramidal cells, appear to be biophysically designed to perform large-scale integration of inputs (10). All of these structural features indicate the possibility that rather than relying on the firing of individual neurons, cortical circuits may generate responses built out of the coordinated activity of groups of neurons. These postulated emergent circuit states could represent the building blocks of mental and behavioral processes (1–3, 11).

In the visual cortex, there has been continuing progress in understanding functional properties and receptive fields of single neurons using single-unit electrophysiology and optical imaging (12–14). These single-neuron studies have provided a solid foundation for neuroscience. However, the focus on single neurons may provide an incomplete picture of this highly distributed neural circuit (3, 15). In fact, in recent years, the network activity patterns of the primary visual cortex (V1) in vitro and in vivo have been shown to be highly structured in spatiotemporal properties (13, 16–18). For example, using voltage-sensitive imaging, one can measure large-scale cortical dynamics with high temporal resolution, albeit without single-cell resolution (19–22). At this bird’s-eye view, wave-like spontaneous spatiotemporal patterns of activity appear similar to those patterns measured during visual stimulation (19, 21, 22). These findings imply that groups of neurons are active together in the absence of any visual input and that the same groups of neurons are also active together in response to visual stimulation. However, to test this hypothesis, one must measure the circuit activity with single-cell resolution.

With calcium imaging, multineuronal activity can be visualized with single-cell resolution (16, 23), so it has become possible to discern exactly which neurons are activated under spontaneous and visually evoked conditions, cell by cell. Indeed, calcium imaging of brain slices from mouse visual cortex has revealed that groups of neurons become coactive spontaneously (24, 25) and that the same groups of neurons can be triggered by stimulation of thalamic afferents (26, 27). However, the patterns of activity found in slices may differ from the patterns of activity in vivo. Therefore, to determine the relation between spontaneous and evoked cortical activity patterns properly, it is necessary to measure them in vivo.

Using two-photon calcium imaging in vivo, we have now mapped the spontaneous reverberating activity patterns in the V1 from awake mice with single-cell resolution and analyzed their relation to the activity patterns evoked by visual stimulation. We find patterns of coactive neurons that we term “ensembles,” defined as “a group of items viewed as a whole rather than individually” (28). Although the mere existence of these coactive neurons does not prove their functional importance, we provide converging lines of evidence that these ensembles are, in fact, functional units of cortical activity. This work provides a step in the progression of defining neuronal ensembles, rather than receptive fields of individual cells, as a building block of cortical microcircuits and suggests that these intrinsic neuronal ensembles are recruited when the cortex performs some of its most basic functions.

Results

Defining Ensembles of Coactive Neurons in Cortical Activity.

To investigate the spatiotemporal dynamics of the activity of networks of neurons in the V1, we used two-photon calcium imaging to record neuronal activity simultaneously from approximately 100 neurons at a time in layer 2/3 of V1 in awake mice standing on a floating trackball (13, 16, 29) (Fig. 1). With this preparation, we mapped the spatiotemporal dynamics of cortical activity in the presence or absence of visual input, using a black screen to measure spontaneous activity and either drifting gratings or a natural movie to evoke cortical activity.

Fig. 1.

Imaging neuronal ensembles. (A) Illustration of a head-fixed awake imaging setup. Mice were presented with a black screen or visual stimulation with drifting gratings or a natural movie. Head fixation was omitted from the drawing for clarity. (B) Two-photon microscopic image of a typical field of view from bolus-loaded cells in layer 2/3 of V1. The Oregon Green Bapta-1 AM (OGB-1) dye labeled both neurons and astrocytes, and the red SR101 dye labeled only astrocytes. (Scale bar, 50 μm.) (C) ROIs (yellow) overlaid on the image. (D) Spike probability (color-coded) of 102 neurons in an example of a single frame (frame 491) during spontaneous activity. Spike probability was inferred from calcium signals using a spike inference algorithm (Materials and Methods). Spike probability was then thresholded to a level of 3 SDs above 0, as detailed in Materials and Methods, and converted to 1 (active) or 0 (inactive). These binary activity data were used for the subsequent analyses unless otherwise indicated. (E) Ensemble of coactive neurons after applying a threshold to D. In a given frame, the red color denotes active neurons and the gray contour denotes inactive neurons. (F) ∆F/F trace from neuron 31 during spontaneous activity. (G) Inferred spike probability from the same neuron. (H) Raster plot of spontaneous activity constructed using thresholded spike probability data. Each row represents a single neuron, and each mark represents neuronal activity. a.u., arbitrary unit. (I) Percentage of neurons coactive in each frame. The red line indicates the threshold for a statistically significant number of coactive cells in a frame (Materials and Methods). A total of 4.07 frames were imaged per second, and the field of view was 317.44 × 317.44 μm.

To validate that our paradigm was consistent with the rich body of work on visual receptive field properties of single neurons, we examined the orientation-tuning properties of individual neurons in V1 by presenting mice with oriented gratings and then calculated the orientation-tuning curve for each neuron [Fig. S1; the mean orientation-selective index (OSI) was 0.45 ± 0.02 (mean ± SEM), and 602 neurons were imaged from seven awake mice]. In agreement with previous work, we found that 53.9 ± 5.5% of imaged neurons were orientation-selective (13, 30, 31) (Fig. S1C; mean ± SEM).

We then examined the potential existence of coactive groups of neurons, neuronal ensembles. At this early stage in this research, both the number of neurons and the extent of cortical volume that one must record from to capture presumptive ensembles are uncertain; however, one can start by analyzing a snapshot of the activity from a group of neurons in a given field of view. To investigate the joint activity from groups of neurons, we analyzed high-activity frames in which a statistically significant number of neurons were active (Materials and Methods). Activity was defined as crossing a threshold of spike probability inferred from calcium signals using a fast, nonnegative deconvolution method (32) (Fig. 1G and Materials and Methods). We found 415 ± 16 high-activity frames per imaging movie [average movie duration: 13.08 ± 0.3 min (mean ± SEM), n = 7 mice], which corresponded to 13.0 ± 0.3% of all image frames. Although the proportion of high-activity frames was relatively small, half of the total activity occurred during these high-activity frames [51.2 ± 2.4% of total thresholded spike probability (mean ± SEM), n = 7 mice; Fig. 1H]. We then defined coactivation of a group of neurons in frames with high activity as an ensemble.

Ensembles Occur Spontaneously and in Response to Visual Stimuli.

Cortical circuits are dominated by recurrent excitatory connections. Such densely interconnected networks of excitatory neurons are ideal for generating intrinsically driven reverberating activity that could link neurons into functional neuronal ensembles. If that is the case, we expect to see coordinated activity of groups of neurons even in the absence of any external input. We therefore analyzed intrinsic cortical activity in the absence of visual input, and we found spontaneously active ensembles (Fig. 2A, Top). Most of the spontaneous activity was recorded before any exposure to gratings or a natural movie, indicating that spontaneous ensembles did not result from the residual activity of preceding visual stimulation.

Fig. 2.

Ensemble properties. (A) Raster plots of the activity from 121 imaged neurons in an awake mouse under three different conditions [spontaneous (Top), drifting gratings (Middle), and natural movie (Bottom); not drawn to spatial scale for purposes of clarity]. The activity of an ensemble is marked in red. (B) Number of ensembles per second in spontaneous, gratings, and natural movie conditions. (C) Percentage of active neurons per ensemble. (D) Mean ∆F/F of neurons. In the gratings condition, only the stimulus period was included for analysis. Data are mean ± SEM [n = 7 mice, 86 ± 8 neurons per mouse (mean ± SEM)]. **P < 0.01, Wilcoxon signed-rank test.

Independently, we also found ensembles evoked by drifting gratings or a natural movie in the same awake mice and compared these evoked ensembles with spontaneous ensembles (Fig. 2A). Interestingly, there was no statistical difference in the number of ensembles per second between spontaneous activity and visually evoked activity with drifting gratings or with a natural movie (Fig. 2B). In addition, the percentage of active neurons per ensemble was similar in spontaneous and visually evoked activity (Fig. 2C).

The firing rates of cortical neurons during the presentation of a preferred stimulus are higher than the firing rates during spontaneous activity (33, 34). Consistently, we also found that the mean ∆F/F, fractional changes in fluorescence relative to the baseline, of neurons was significantly higher during the presentations of preferred oriented gratings compared with spontaneous activity (Fig. 2D). In contrast, the mean ∆F/F of neurons during the presentation of a natural movie was similar to that during spontaneous activity (Fig. 2D).

Ensembles Repeat.

The detected ensembles may be transient combinations of coactive cells or, alternatively, may be more stable groupings. To distinguish between these possibilities, we analyzed whether ensembles repeat (Fig. 3). To evaluate the spatial similarity between ensembles, we used the correlation coefficients between the ensembles as a similarity metric. A threshold for significant correlation was established for each comparison (Materials and Methods). We generated 50,000 independent surrogate datasets by randomizing active cells while preserving the total number of active cells per frame in one of the frames in every comparison. After establishing this threshold to evaluate similarity between pairs of ensembles, we analyzed the number of times that similar ensembles occur spontaneously. To evaluate whether similar ensembles occur more frequently than by chance, we generated 1,000 independent surrogate datasets by randomly exchanging activity between cells while preserving both the number of active cells per frame and the total amount of activity per cell (Fig. 3C).

Fig. 3.

Ensembles repeat. (A) Example of similar ensembles occurring during spontaneous activity. t, time during image acquisition. (Scale bar, 50 μm.) (B) Schematic illustrating significantly correlated frames as the number of frames compared increases. (C) Schematic illustrating the shuffling method. In surrogate data, activities between neurons were randomly exchanged while preserving both the number of active neurons in a given frame and the total amount of activity in a given neuron. Black lines denote the original activities conserved in the shuffled trace, red lines denote new activities after shuffling, and dotted lines denote activities removed by shuffling. (D) Correlated ensembles occur more frequently than by chance. The y axis shows the percentage of high-activity frames that participated in correlations, and the x axis shows the number of correlated frames. Dotted lines denote the mean of the 1,000 surrogate datasets. Data are mean ± SD [n = 7 mice (data from three mice are shown in Fig. S2A)]. ^†P < 0.05; *P < 0.005. spon, spontaneous. (E) Example of a histogram of the percentage of surrogate high-activity frames that participated in three-time correlations in the spontaneous condition from mouse 3 (red dotted circle in D). Note that surrogate and observed data do not overlap. Each experiment was recorded for 13.08 ± 0.3 min (mean ± SEM).

We applied this analysis to the spontaneous activity and found that correlated ensembles repeated far more frequently than by chance, and this result was highly statistically significant (Fig. 3D and Fig. S2A). We extended this analysis to visually evoked activity in response to gratings or a natural movie and again found highly significant repetition of correlated ensembles (Fig. 3D and Fig. S2A). We found the same results in anesthetized mice (Fig. S2B).

We chose to analyze the inferred spike probability because it represents neuronal activity better than ∆F/F (32). However, as a control, we also analyzed the raw ∆F/F traces to verify that our results were not an artifact of using a spike inference algorithm, and the results between the two analyses were consistent (Fig. S3 and S4).

In all, these data show that ensembles, both spontaneous and in response to stimuli, are not fleeting groupings of neurons but stable groups that are repeatedly active together.

Responses of Individual Neurons in Isolation Do Not Account for the Occurrence of Ensembles.

The detected ensembles may emerge simply from the firing properties of individual neurons acting in isolation. Alternatively, they may emerge from an added layer of functional connectivity, either intrinsic to the cortical microcircuit or arising from the thalamus. To determine if these ensembles could emerge from the individual firing properties of neurons in isolation, we computed the predicted probability of the occurrence of a “core” ensemble, defined as a group of coactive neurons that are conserved in all significantly correlated ensembles (Fig. 3D), by multiplying the observed probabilities of single-cell activation. We then compared this predicted probability with the observed probability of core ensemble occurrence (Fig. 4A, red neurons with green contours). If these probabilities are similar, the coactivation of a group of neurons in ensembles may simply result from the independent activation of individual neurons. However, when we performed this analysis for spontaneous activity, we found that the observed frequency of core ensemble occurrence was significantly higher than the frequency of core ensemble occurrence computed from the combined probability of individual neuron activation (Fig. 4B). We applied the same analysis to visually evoked activity with gratings or a natural movie, and we again found that the probability of group activation was significantly higher than the probability of group activation accounted for by the properties of neurons individually (Fig. 4B). Taken together, these data show that coactivation of groups of neurons likely emerges from functional interactions between neurons rather than from the individual firing properties of isolated neurons.

Fig. 4.

Ensembles cannot be explained by firing properties of individual neurons. (A) Example of correlated ensembles [spontaneous (Top), drifting gratings (Middle), and natural movie (Bottom)]. The red color denotes an ensemble in a given frame, and the green contour denotes a core ensemble, defined as a group of coactive neurons that are conserved in all significantly correlated ensembles. (Scale bar, 50 μm.) (B) Distributions of the predicted probability that a core ensemble would be coactive, calculated based on the individual firing properties of neurons in isolation (blue) vs. the observed probability that a core ensemble was coactive (red). Mean predicted probabilities were 0.0006 ± 0.0003, 0.007 ± 0.005, and 0.002 ± 0.001, and mean observed probabilities were 0.0020 ± 0.000, 0.017 ± 0.005, and 0.010 ± 0.002 for the spontaneous, drifting gratings, and natural movie conditions, respectively (P < 0.005 and n = 7 for the spontaneous and gratings conditions, and P < 0.05 and n = 4 for the natural movie condition; Wilcoxon signed-rank test; mean ± SEM).

Single Neurons Participate Promiscuously in Multiple Ensembles.

While analyzing correlated ensembles during spontaneous or visually evoked activity (Fig. 4), we noticed that individual neurons participating in one ensemble also participated in other ensembles with different sets of neurons (Fig. 5A). To quantify how flexibly individual neurons shift from ensemble to ensemble, we analyzed core ensembles conserved in the significantly correlated ensembles that were triggered by each oriented grating (Fig. 5A; total of four orientations). We then calculated the proportion of neurons that were shared in multiple core ensembles triggered by distinctly oriented gratings. We found that 8.42 ± 1.24% of neurons were shared in different core ensembles triggered by distinctly oriented gratings with a 45° difference in orientation (Fig. 5B). A smaller proportion of neurons were shared in different core ensembles when the difference in orientation was 90° (Fig. 5B; 4.12 ± 0.74%). This finding could be explained by the possibility that neurons shared in multiple core ensembles are more broadly tuned (i.e., less “specialized” neurons). In contrast, neurons that participated in core ensembles, but were not shared by different core ensembles triggered by distinctly oriented gratings, should be more specialized. To test this hypothesis, we analyzed the mean OSI of the shared neurons and compared it with the OSI of all imaged neurons or the neurons that participated in core ensembles but were not shared by the other core ensembles triggered by distinctly oriented gratings. We found no statistical difference (Fig. 5C). Therefore, individual neurons, regardless of how specialized they are for a stimulus at an individual level, can be part of multiple ensembles.

Fig. 5.

Individual neurons flexibly participate in multiple ensembles. (A) Example of correlated ensembles [spontaneous (Top), drifting gratings (Middle), and natural movie (Bottom)]. The red color denotes an ensemble in a given frame, the green contour denotes a core ensemble (also Fig. 4), and the blue contour denotes neurons that were shared in multiple core ensembles. (Scale bar, 50 μm.) (B) Percentage of neurons that were shared in multiple core ensembles evoked by distinctly oriented gratings with a difference in orientation of 45° vs. 90° (n = 7 mice). (C) Mean OSI (n = 7). (D) Percentage of neurons that were shared in multiple core ensembles evoked by distinct natural scenes (n = 4). Data are mean ± SEM.

Because natural scenes consist of complex visual features, we predicted that more neurons would be shared between the core ensembles that are evoked by distinct natural scenes. We found that 40.84 ± 5.15% of neurons were shared between the core ensembles that were activated by distinct natural scenes and 12.79 ± 4.01% of neurons were shared in up to five distinct core ensembles (Fig. 5D). Taken together, our findings demonstrate that when individual neurons are activated, they are more likely to be activated together with a specific set of other neurons as an ensemble. At the same time, individual neurons can participate in multiple ensembles, dynamically reorganizing their allegiance with different sets of neurons.

Ensembles Evoked by Visual Stimulation Are Similar to Spontaneous Ensembles.

We found ensembles that occurred spontaneously and in response to visual input. There may be two populations of ensembles: intrinsically generated spontaneous ensembles and visually evoked ensembles. Alternatively, visual stimuli may draw on the lexicon of intrinsically generated ensembles. To distinguish between these possibilities, we determined whether the correlated ensembles that are repeatedly evoked by visual stimulation are similar to the correlated ensembles that repeat spontaneously. A threshold for significant correlation was established for each comparison as described in Fig. 3. After establishing this threshold, we searched for the matching correlated ensembles between spontaneous and evoked activity, by either gratings or a natural movie, and plotted the percentage of the evoked high-activity frames with the matching correlated ensembles as the number of frames compared increases (Fig. 6). To determine if the number of correlated frames with the matching ensembles was significant, we generated 100 independent surrogate spontaneous datasets in which spontaneous activity was shuffled as described in Fig. 3C. We then performed the same analysis with the real evoked datasets and the surrogate spontaneous datasets. This analysis revealed that the matching correlated ensembles between spontaneous and evoked activity occur far more frequently than by chance in all seven mice (Fig. 6B and Fig. S5A). We found similar results in anesthetized mice (Fig. S5B). Our findings show that stimulus-evoked ensembles overlap substantially with intrinsically driven, spontaneously active ensembles.

Fig. 6.

Visually evoked ensembles are similar to spontaneous ensembles. (A) Schematic illustrating ensembles that are correlated in both spontaneous and visual stimulation conditions. (B) Correlated evoked ensembles are also similar to correlated spontaneous ensembles above chance level. The y axis shows the percentage of evoked high-activity (h.a.) frames that participated in matching correlations between evoked and spontaneous data, and the x axis shows the number of correlated frames. Dotted lines denote the mean percentage of evoked high-activity frames that participated in matching correlations between real evoked data and 100 spontaneous surrogate datasets. Data are mean ± SD (n = 7 mice; data from three mice are shown in Fig. S5A). ^†P < 0.05; *P < 0.005. (C) Example ensemble frames with a significant correlation between the natural movie and spontaneous conditions. Only two ensemble frames were included for purposes of clarity, although more were correlated. (Scale bar, 50 μm.)

Spontaneous Ensembles Repeat Randomly, Whereas Evoked Ensembles Are Locked to Visual Stimuli.

Our finding that stimulus-evoked ensembles are similar to spontaneously evoked ensembles suggests two distinct possibilities. The ensemble activity present during visual stimulation may simply reflect ongoing spontaneous activity unrelated to the visual input. Alternatively, stimuli may selectively recruit intrinsic ensembles that are also active spontaneously. If this second scenario is the case, a given stimulus should consistently evoke a specific ensemble. This is, in fact, what we found. First, we measured the occurrence of significantly correlated spontaneous ensembles, analyzed the time intervals between the significantly correlated ensembles, and plotted these time intervals for all significantly correlated ensembles (Fig. 7A; the significantly correlated ensembles are shown in Fig. 3D). We found that correlated spontaneous ensembles reoccurred in an apparently random temporal sequence. We then analyzed evoked ensembles in response to the repeated presentation of distinctly oriented gratings and found that the temporal sequence was not random at all. Correlated ensemble frequency peaked when identically oriented gratings were represented (Fig. 7B). Finally, we examined evoked ensembles in response to a natural movie played in a loop. We found that correlated ensemble frequency peaked when the corresponding natural scene repeated in the looped movie (Fig. 7C). These results demonstrate that a given stimulus, whether simple gratings or a natural scene, consistently evokes a specific ensemble. Taken together, our findings show that ensembles of neurons are active together spontaneously and that visual stimuli recruit the intrinsic ensembles that are relevant to incoming stimuli.

Fig. 7.

Temporal occurrence of ensembles. Correlated spontaneous ensembles reoccurred at random time intervals, whereas correlated evoked ensembles reoccurred when an identical stimulus was represented. (A) Spontaneous ensembles. (B) Ensembles evoked by drifting gratings (a session of four distinctly oriented gratings looped every 20 s). (C) Ensembles evoked by the looped natural movie (30 s in length). Data from six mice were pooled.

Discussion

We used two-photon calcium imaging to capture the network activity in V1 of awake mice. We found, first, that both spontaneous and evoked cortical activity are dominated by high-activity periods (50% of total activity) with groups of coactive of neurons that we defined as ensembles. Second, we found that these ensembles repeat, suggesting that they are stable groups of neurons and not simply fleeting pairings of neurons. Third, we found that ensembles cannot be explained by the individual firing properties of neurons in isolation, suggesting that neurons are functionally bound together as a group. Fourth, we found that individual neurons contribute to multiple ensembles, vastly expanding the cortical encoding potential beyond a single cell model. Fifth, we found that spontaneous ensembles and stimulus-evoked ensembles are highly similar, with one main difference: Spontaneous ensembles occur at random time intervals, whereas visually evoked ensembles are time-locked to stimuli.

In this work, we focused our analysis on the spatial structure of ensembles. Our imaging was performed with relatively poor temporal resolution, so it precludes us from a more detailed analysis of the temporal structure of the firing of the neurons within an ensemble. However, as in previous work in cortical brain slices (24–27, 35), it is likely that a given ensemble represents a sequence of events in time. Future work, using techniques with faster resolution, is necessary to explore the underlying temporal dynamics of ensemble formation.

Intrinsically Active Neuronal Ensembles.

We found that neurons tend to fire together with other neurons, forming ensembles of coactive neurons, even in the absence of any visual input. These ensembles are repeatedly active, and they can be evoked by specific stimuli, indicating that they do not arise from random neuronal pairings. In fact, the probability of a group of neurons being activated together is much higher than the probability predicted by their individual firing properties. This finding implies that these ensembles emerge from an added layer of functional connections, which links neurons together into groups. Although the spontaneous ensembles could emerge from spontaneous patterned feed-forward activity from the thalamus, the fact that spontaneous ensembles also repeat in cortical slices lacking thalamic input argues against this possibility (25, 26). Therefore, these ensembles likely emerge from intrinsic cortical connectivity. This second possibility is in agreement with the findings by Ko et al. (30, 31), which demonstrate that in mouse V1, neurons with the same preference for orientated gratings or naturalistic stimuli make more synaptic connections with each other than those neurons with a preference for orthogonally oriented gratings or different naturalistic stimuli.

Individual Neurons Contribute Flexibly to Multiple Ensembles.

We also found that neurons considered as a group (i.e., an ensemble) responded reliably to stimuli. However, when neurons were considered individually, they were promiscuous and participated in other ensembles evoked by different visual stimuli. This finding suggests that groups of neurons can encode visual features more reliably than individual neurons. This finding is also consistent with the findings showing that population responses perform much better at decoding tasks than single neurons in anesthetized mice (36).

These dynamic rearrangements of cortical activity may explain how a limited number of neurons can encode the ever-changing environment with reliability and without averaging, and this flexibility may be a fundamental property of cortical function.

Visually Evoked Ensembles Are Similar to Spontaneous Ensembles.

By performing voltage-sensitive dye imaging of large-scale cortical dynamics in V1 of anesthetized animals, Kenet et al. (19) demonstrated that spontaneously occurring cortical states resemble the cortical responses to visual inputs. Our results, using a technique with cellular resolution in awake mice, extend the study by Kenet et al. (19) and demonstrate that the ensembles that are active spontaneously are also activated by visual stimuli. Our results are also consistent with the finding that a pair of neurons with the same preference for oriented gratings exhibits higher cell-to-cell correlation during spontaneous activity than a pair of neurons with a preference for orthogonally oriented gratings in anesthetized animals (37).

Visual Stimuli Recruit Intrinsic V1 Ensembles.

What explains the close overlap between spontaneous ensembles and visually evoked ensembles? Visual experience could shape local synaptic connections in V1 during development (38, 39) and continuously throughout adulthood (40). Thus, the past activity could be responsible for the current state of synaptic connectivity that likely generates the spontaneous ensembles. Such ongoing intrinsic ensembles might be critical for maintaining and strengthening this synaptic connectivity. Our findings that visual stimuli recruit spontaneously active neuronal ensembles that are relevant to the incoming visual stimuli suggest that ensembles encode visual features. The reverberating, self-generated cortical activity that we found may therefore be important for preparing the circuit to receive incoming sensory input efficiently. Feed-forward thalamic input may then bias and amplify the intrinsic ensemble that is most relevant to the stimulus (Fig. 8).

Fig. 8.

Model illustrating that visual stimuli recruit ensembles from a spontaneous lexicon. In this proposed model, when a visual stimulus reaches the cortex, it activates individual components of an ensemble, each of which is relatively unreliable in isolation. Through recurrent connections, an entire ensemble is then activated, recruited from the spontaneously active lexicon of ensembles. The two examples shown are actual cortical responses to distinct visual stimuli and highlight the fact that individual neurons contribute to multiple ensembles. (Scale bar, 50 μm.)

Neuronal Ensembles as Functional Building Blocks of the Cortex.

What is the functional meaning of these intrinsic ensembles? We speculate that they could represent emergent states of cortical function because the structural principles of the cortical microcircuit are ideally suited to perform distributed computations (8–10). In fact, over the past decades, there have been many theoretical proposals postulating the existence of computational units that are built by joining together the activity of many neurons (1, 3, 6, 11, 15, 41, 42). Such emergent units of function, named differently by different authors (e.g., neuronal oscillations, reverberations, assemblies, ensembles, groups, synfire chains, clicks, attractors, flashes, songs, bumps) and with differences in the temporal precision that they could exert, all share the fundamental property of diluting the importance of the firing of individual neurons and of treating the circuit as a neural network (43). In the extreme case of a completely distributed circuit, the activity of any given neuron becomes irrelevant.

Based on our results, and in agreement with previous results from complementary experimental paradigms in brain slices (24–27, 35) and in vivo (19, 22, 44–51), we propose that neuronal ensembles are intrinsic circuit motifs of cortical activity that represent its emerging computational primitives. To test if these coactive responses are related to behavioral or intrinsic states, one needs to alter these ensembles in vivo. It would be ideal, in a future experiment, to generate or obliterate the ensembles, or to alter their cellular participants, online, as if one is “playing the piano” with the neural circuits during behavior (52). The development of novel optical techniques, such as two-photon holographic optogenetics (53, 54), could enable 3D spatiotemporal manipulation of the activity of neuronal populations with single-cell precision in awake animals, an ideal experimental platform with which to explore the functional significance of neuronal ensembles.

Materials and Methods

Animals, Surgery, and Training.

All experimental procedures were carried out in accordance with Columbia University institutional animal care guidelines. Experiments were performed on C57BL/6 mice (n = 6) or on parvalbumin-Cre (n = 2) or somatostatin-Cre (n = 2) × LSL-tdTomato transgenic mice, obtained from The Jackson Laboratory, at the age of postnatal day (P) 40–P80 (55–57). Seven mice were used for awake preparation, and three mice were used for anesthetized preparation. During surgery, mice were anesthetized with isoflurane (initially 2% (partial pressure in air) and reduced to 1%). A small circle (1–2 mm in diameter) was thinned over the left V1 using a dental drill to mark the site for craniotomy (centered at 2.5 mm lateral from the lambda, putative monocular region). A titanium head plate was attached to the skull using dental cement. Mice underwent training to maneuver on a spherical treadmill with their head fixed for 1–3 h each day for 2–3 d. This head-fixed awake preparation allows mice to move freely, but movement is not associated with vestibular stimulation.

Dye Loading and Two-Photon Calcium Imaging.

On the imaging day, mice were anesthetized with isoflurane and the craniotomy, marked previously, was completed for dye injection. For bulk loading of cortical neurons Oregon Green Bapta-1 AM (Molecular Probes) was first dissolved in 4 μL of freshly prepared DMSO containing 20% Pluronic F-127 (Molecular Probes) and then further diluted in 35 μL of dye buffer [150 mM NaCl, 2.5 mM KCl, and 10 mM Hepes (pH 7.4)] (58). Sulforhodamine 101 (50 μM; Molecular Probes) was added to the solution to label astrocytes (59). The dye was slowly pressure-injected into the left visual cortex at a depth of 150–200 μm at an angle of 30° with a micropipette (4–7 MΩ, 10 psi, 8 min) under visual control by two-photon imaging (20× water immersion objective, 0.5 N.A.; Olympus). The activity of cortical cells was recorded by imaging fluorescence changes with a two-photon microscope (Moveable Objective Microscope; Sutter Instrument) and a Ti:sapphire laser (Chameleon Vision II; Coherent) at 880 nm or 1,040 nm through a 20× (0.95 N.A.; Olympus) or 25× (1.05 N.A.; Olympus) water immersion objective. Scanning and image acquisition were controlled by Sutter software (4.07 frames per second for 512 × 512 pixels or 8.14 frames per second for 340 × 340 pixels, Mscan; Sutter Instrument).

Visual Stimulation.

Visual stimuli were generated using the MATLAB (MathWorks) Psychophysics Toolbox (60) and displayed on a liquid crystal display monitor (19-inch diameter, 60-Hz refresh rate) positioned 15 cm from the right eye, roughly at 45° to the long axis of the animal. Spontaneous calcium signals were measured for ∼13 min in the dark at the beginning of the experiments and sometimes in the middle of the experiments (with a monitor and room lights turned off). The imaging setup was completely enclosed with blackout fabric (Thorlabs). After spontaneous calcium signals were collected, mice were presented with either sequences of full-field grating stimuli or a natural movie (the order of presentations was alternated randomly). Square or sine wave gratings (100% contrast, 0.035 cycles per degree, two cycles per second) drifting in eight different directions in random order were presented for 5 s, followed by 5 s of mean luminescence gray screen (10 repetitions). A natural movie (Moose in the Glen, from the British Broadcasting Corporation’s Natural World documentary series) consisting of 10 distinct natural scenes in 30-s sequences was played using the MATLAB Psychophysics Toolbox (20 repetitions). In some experiments, a natural movie was played using the QuickTime Player (Apple). The sequences of gratings or a natural movie stimulation played in MATLAB were synchronized with image acquisition using Sutter software (Mscan; Sutter Instrument). Locomotion of a mouse was not associated with motion of the visual scene relative to the mouse.

Image Analysis.

The raw images were processed to correct brain motion artifacts using the enhanced correlation coefficient image alignment algorithm (61) or a hidden Markov model implemented previously (62, 63). Initial image processing was carried out using custom-written software in MATLAB (Caltracer 2.5, available at our laboratory website). Cell outlines were detected using an automated algorithm based on fluorescence intensity, cell size, and cell shape, and were adjusted by visual inspection. Cell-based regions of interest (ROIs) were then shrunk by 5–10% to minimize the influence of the neuropil signal around the cell bodies. All pixels within each ROI were averaged to give a single time course, and ∆F/F was calculated by subtracting each value with the mean of the lower 50% of previous 10-s values and dividing it by the mean of the lower 50% of previous 10-s values. For a cross-correlation analysis using ∆F/F, neuropil contamination was removed by first selecting a spherical neuropil shell (6.2-μm thickness) surrounding each neuron and then subtracting the average signal of all pixels within the spherical neuropil shell, excluding adjacent ROIs and pixels within 0.3 μm surrounding ROIs, from the average signal of all pixels within the ROI. Neurons with noisy signal with no apparent calcium transient were detected by visual inspection and excluded from further analysis.

Spike probability was inferred from calcium signals using a fast, nonnegative deconvolution method (32). Briefly, the baseline of calcium signals was detrended, and ∆F/F was then calculated before applying an algorithm to infer spike probability. The decay constant of calcium transients, τ, was set to 0.8 s. The output was normalized by a maximum value in each neuron. Spike probability was then thresholded to a level of 3 SDs above 0, determined from spike probabilities of the entire population in each experiment, to identify active cells not confounded from the noise in the recordings; the values above a threshold were set to 1, and the values below a threshold were set to 0. These binary activity data were then used for subsequent analyses unless otherwise indicated. Although most spikes resulted in significant somatic calcium transients with a calcium indicator and analysis threshold similar to our experiments (25), we likely underestimated the presence of action potentials, particularly when neurons fire a single action potential or at frequencies higher than 40 Hz (64).

To analyze the OSI, average inferred spike probability or ∆F/F was taken as the response to each grating stimulus. Responses from 10 trials were averaged to obtain an orientation-tuning curve or matrix. The preferred orientation was taken as the modulus of the preferred direction to 180°. The OSI was calculated as (R_best − R_ortho)/(R_best + R_ortho), where R_best is the best direction and R_ortho is the average of responses to the direction orthogonal to the best direction. Cells with an OSI <0.4 were considered to be unselective for orientation.

Definition of an Ensemble.

An ensemble was defined as coactivation of a group of neurons in a high-activity frame in which a statistically significant number of neurons were active. To establish a threshold for the significant number of coactive neurons, binary activity data (thresholded spike probability) were shuffled 1,000 times by randomly transposing intervals of activity within each cell (shuffling within cells). The threshold corresponding to a significance level of P < 0.05 was estimated as the number of activated cells in a single frame that exceeded only 5% of these surrogate datasets. The number of ensembles per second was calculated by dividing a number of high-activity frames by a number of total frames and then multiplying the quotient by a frame rate in each imaging movie. In the drifting gratings condition, only stimulus periods were included in this analysis. The mean ∆F/F of neurons was calculated by averaging ∆F/F during the presentation of preferred oriented gratings for each neuron. Only neurons with an orientation preference were counted in this analysis. From the same set of neurons, ∆F/F was averaged throughout the entire traces for the spontaneous and natural movie conditions.

Correlated Ensembles.

The similarity between ensembles was evaluated using Pearson’s correlation coefficient (r). To convert r to the normally distributed variable z, the Fisher z-transformation was applied to r according to the following:

$$z=\frac{1}{2}\ln \left(\frac{1+r}{1-r}\right).$$ [1]

A threshold for significant correlation was established for each pairwise comparison. Establishing a threshold for each comparison is important because in binary data the number of active neurons in a frame influences a correlation coefficient between a pair of frames. We generated 50,000 independent surrogate ensembles by randomizing active cells while preserving the number of active cells per frame in one of the frames in each comparison (shuffling across cells). The threshold corresponding to a significance level of P < 0.05 was estimated as the correlation coefficient that exceeded only 5% of correlation coefficients between these surrogate ensembles. After establishing this threshold to evaluate similarity between ensembles, we analyzed the number of times that similar ensembles occur. To evaluate whether similar ensembles occurred more frequently than by chance, we generated 1,000 independent surrogate datasets by randomly exchanging activity between cells while preserving both the number of active cells per frame and the total amount of activity per cell. Surrogate data were independently generated in all conditions (i.e., spontaneous activity and visually evoked activity with gratings or a natural movie) because the activity of individual neurons may differ in different conditions.

To search for the matching correlated ensembles between spontaneous and visually evoked activity, the similarity between spontaneous and evoked ensembles was calculated using Pearson’s correlation coefficient. A threshold for significant correlation was established for each pairwise comparison as above. To evaluate whether matching-correlated ensembles occur more frequently than by chance, we generated 100 independent surrogate spontaneous datasets by randomly exchanging activity between cells while preserving both the number of active cells per frame and the total amount of activity per cell in spontaneous activity and searched for the matching correlated ensembles between surrogate spontaneous and real evoked activity.

The predicted probability of a core ensemble being activated together was calculated by multiplying the probabilities of single neurons in the core ensemble being activated during spontaneous activity. The probability of a single neuron being activated was calculated by dividing the number of frames where the neuron was active by the number of total frames during spontaneous activity. The observed probability of a core ensemble being activated together was calculated by dividing the number of frames where the ensemble was coactive by the number of total frames during spontaneous activity. In the visual stimulation conditions, the probability of a single neuron being activated was calculated by dividing the number of frames where the neuron was active during the presentations of the same orientation or same natural scene by the number of total frames that were presented with the same orientation or same natural scene. Similarly, the probability of a core ensemble being activated was calculated by dividing the number of frames where the core ensemble was coactive during the presentations of the same orientation or same natural scene by the number of total frames that were presented with the same orientation or same natural scene. Note that the entire frames were used in this analysis.

To analyze the percentage of neurons shared in multiple core ensembles, the core ensembles that were conserved in significantly correlated ensembles evoked by each oriented grating (total of four orientations) or each natural scene (total of 10 scenes) were counted. After the core ensembles were identified, the number of neurons shared in different core ensembles that were evoked by distinctly oriented gratings or different natural scenes was counted (“shared neurons”) and divided by the total number of imaged neurons. Neurons that belonged to core ensembles, but were not shared with other core ensembles evoked by distinctly oriented gratings or different natural scenes, were defined as “unshared neurons.”

To analyze the time interval between significantly correlated ensembles, we first looked at the image acquisition time of a set of significantly correlated frames and calculated time intervals between all possible pairs of the significantly correlated frames. We then repeated this analysis for the entire sets of significantly correlated frames and plotted these time intervals as a histogram for spontaneous or visually evoked activity. Each frame pair was counted only once. In the gratings condition, each orientated grating was presented in random order during image acquisition, and we sorted activity traces of all neurons according to four differently orientated gratings in each session (90°, 135°, 0°, and 45° in order; total of 20 sessions per experiment). Because each session consisted of 20 s (5-s stimulus, four orientations), the identical orientation reoccurred every 20 s and lasted for 5 s. Note that only the stimulus period was included in this analysis. For the natural movie condition, because the 30-s movie was played in a loop 20 times during image acquisition, the identical scene reoccurred every 30 s and lasted for less than 0.25 s.

Statistical Analysis.

We used Wilcoxon rank sum tests to determine statistical significance (P < 0.05) unless otherwise indicated.