Playing the piano with the cortex: role of neuronal ensembles and pattern completion in perception and behavior

Abstract

Neuronal ensembles, i.e. coactive groups of neurons, have been long postulated to be functional building blocks of cortical circuits and units of the neural code. Calcium imaging of neuronal populations has demonstrated the widespread existence of spontaneous and sensory-evoked ensembles in cortical circuits in vivo. The development of two-photon optical techniques to simultaneously record and activate neurons with single cell resolution (“piano” experiments) has revealed the existence of pattern completion neurons, which can trigger an entire ensemble, and demonstrated a causal relation between ensembles and behavior. We review recent results controlling visual perception with targeted holographic manipulation of cortical ensembles by stimulating pattern completion neurons. Analyzing population activity as neuronal ensembles and exploiting pattern completion could enable control of brain states in health and disease.

Introduction

As the genetic code deciphered the relationship between the sequence of nucleotides in the genome and the structure of proteins, one can view breaking of the neural code, i.e., deciphering the relationship between the patterns of neuronal activity and behavior, as the central problem in neuroscience. And, just like the codon is the basic information unit of the genetic code [1], one may wonder what is the codon equivalent for the neural code. The traditional view, the neuron doctrine championed by Cajal [2] and Sherrington [3], states that individual neurons are the structural and functional units of the nervous system. In this view, each neuron has a specific function, defined, in sensory systems, by its receptive field. But an alternative paradigm [4] is that the functional units are groups of neurons acting together as ensembles, i.e. neurons firing together, and that these ensembles are the coding units [5]. This view was pioneered by Cajal’s own disciple, Lorente de Nó, who, after careful anatomical analysis of cortical circuits, proposed that recurrent excitatory connectivity would generate “neuronal chains” with ongoing “reverberant” activity that would make them get activated as a unit [6]. These endogenously active states could be used as symbols of objects in the physical world, similar to the representations (“Darstellungen”) postulated by Kant [7], and thus enable the mental manipulation of reality. The idea of groups of recurrently connected neurons working as units was further developed by Hebb, who argued that these chains, which he renamed “cell assemblies”, would naturally arise from synaptic learning rules where pre and postsynaptic cells would be coactive, and could be used to implement associative memories [8]. Applying insights from the physics of emergent systems, Hopfield provided a mathematical backbone for these ideas, arguing that recurrently connected neural circuits would settle on to dynamical “attractors”, i.e., stable or semistable states of the activity of the network, formed by the conjoint activation of a group of neurons [9]. Because of their settling dynamics, these attractors would have pattern completion properties, i.e. activation of a few of the neurons could trigger the entire pattern. Hopfield and Tank then generalized the function of these coactive states, arguing that they could implement, not only memories, but also solutions to computations such as optimizations [10]. Finally, using electrophysiological insights, Abeles argued that the sequential firing of synchronous groups of neurons (“synfire chains”) would preferentially propagate through cortical circuits, because among others excitatory connections are sparse, and because neurons have a non-linear activation function near the action potential threshold [11]. Thus, in spite of different approaches and nomenclatures, these pioneers predicted essentially the same idea: that the neural code is written with multicellular units, built by the coordinated activity of groups neurons. For clarity, for the rest of the piece we call these units neuronal ensembles.

Direct support for the ensemble hypothesis came from the development of optical methods to image and optically manipulate neural circuits with single cell resolution. Calcium imaging [12], when combined with two-photon microscopy [13], enabled the direct visualization of the action potential activity of neuronal populations, revealing spontaneous coactivations of neurons in cortical slices [14]. These endogenous ensembles could be triggered by activating thalamic inputs, as if the evoked response was built using a vocabulary of intrinsic ensembles [15]. Very similar results were found in awake animals: neuronal ensembles were prevalent in both visually-evoked and spontaneous activity, and evoked ensembles resembled spontaneous ones [16]. But to test if these ensembles are indeed functional, one needs to manipulate them. The development of two-photon photoactivation of neurons [17] and holographic optogenetics [18], has enabled doing this in all-optical experiments, as if one were optically “playing the piano” with the circuit, where each neuron is a piano key and one can activate them at will. These holographic “piano” methods are critical because they have single cell resolution and neuronal ensembles are built with specific sets of neurons that occupy the same territories, so one needs two-photon precision to stimulate the neurons from one ensemble selectively, without activating cells that belong to other ones. In close agreement with early predictions, these “piano “experiments have revealed that ensembles can be artificially imprinted by coactivating cortical neurons in vivo, following Hebbian paradigms [19••]. Moreover, imprinted ensembles have pattern completion properties, as predicted by Hopfield [9], and are stable for at least a day [19••]. Finally, recent results using holographic optogenetics have demonstrated that the activation of ensembles, by turning on individual pattern completion neurons, can actually control visually-evoked behavior in mice [20••,21••]. Even the stimulation of two neurons can change the behavior of a mouse. Thus, there is a causal link between neuronal ensemble activity and behavior, as ensembles appear necessary and sufficient to trigger visual perception [20••,21••]. These results have also demonstrated that targeting neurons with pattern completion capabilities can be an effective strategy to control behavior [19••,20••]. Although the idea of recalling a learned behavior by the stimulation of single neurons is not new [22,23], the precise targeting of visualized neurons paves the road to understand circuit mechanisms underlying different behaviors with single cell precision. In the following we review these recent results and briefly discuss their implications for neuroscience and medicine.

Computational identification of neuronal ensembles

Let’s go back to the neural code. Since ensembles are based on group activity, understanding how neuronal ensembles are spatially and temporally formed requires population analyses. To start, imaging the activity of neurons (Figure 1a) can be represented by a matrix [NxT] where N is the total number of neurons and T is the total number of time points (Figure 1b). Analyzing these matrices, one can explore the existence of neuronal ensembles, defining the population activity at a given time as a vector, which constitutes a computation unit (Figure 1b). Thus, each point in a multidimensional space represents a population vector instead of a single neuron (Figure 1c). Applying dimensionality reduction algorithms to population vectors then generates a low dimensional visualization of the original matrix of activity, with clusters of population vectors (Figure 1c). Each cluster of population vectors defines a neuronal ensemble that represents a given network state [5]. For example, in the case of primary visual cortex each cluster defines an ensemble that responds to a different orientation grating or a particular natural scene [16,24]. With this framework, a population vector at one time can be compared to subsequent population vectors at different times [20••,21••], capturing single-trial variability without averaging [25]. This type of neuronal ensemble analysis has generated new insights and interpretations for neuronal population data [19••,20••,24,26,27•,28,29].

Figure 1. Analysis of neuronal ensembles.

(a) Schematic representation of calcium imaging recordings where active neurons at different frames (f(1) to f(t)) are depicted by black dots. The activity profile from each frame defines a population vector (v(1) to v(t)). (b) Representation of population vectors as binary arrays where 0’s and 1’s indicate silent and active neurons. Colors depict population vectors form by similar groups of coactive neurons. Each row represents a neuron and each column represents a population vector. (c) Dimensionality reduction applied to population vectors reveals clusters in a low dimensional space, where each cluster defines a neuronal ensemble.

Pattern completion in neuronal ensembles

The ability to make conjectures from incomplete information sources represents a fundamental feature of brain computation [8,30,31]. This phenomenon, “pattern completion”, explained by dynamical properties characteristic of physical or biological systems, could be used to recall a past state, as a memory, from some of the elements involved [9]. In a neuronal ensemble context, pattern completion refers to the capacity to activate the entire neuronal ensemble by the targeted activation of one or few neurons. While long postulated, pattern completion it was only directly revealed in recent “piano” experiments with two-photon optogenetics in primary visual cortex [19••]. It was found that, when stimulating a random group of neurons together (Figure 2a) 50 to 100 times (as if one was playing the same cord in a piano), these neurons could be glued together and became spontaneously coactive (Figure 2b), building an ensemble. This artificially “imprinted” ensemble, which lacked any sensory or behavioral significance (since the neurons were chosen randomly), was still spontaneously active a day later, demonstrating long-term reprogramming of cortical circuits with a Hebbian paradigm [19••]. Interestingly, stimulation of neurons from this imprinted ensemble one by one (as if one were playing the piano with a finger), revealed that some of the imprinted neurons had pattern completion properties, activating the entire ensemble (Figure 2c), even a day after [19••]. The mechanisms of ensemble imprinting and pattern completion are still under investigation, although it was assumed to be mediated by Hebbian plasticity. Since neuronal ensembles are comprised by neurons with higher probability to share synaptic connections [26,32], pattern completion could be due to reactivation of recurrent connections between pattern completion neurons and the rest of the ensemble.

Figure 2. Pattern completion in imprinted ensembles.

“Piano” experiments in primary visual cortex of awake mice. (a) Two-photon population photostimulation of a group of neurons (red), using two-photon calcium imaging to detect the neuronal response to the stimulation laser. Scale bar 50 μm. (b) Imprinted ensemble active during spontaneous, ongoing activity. Red contours indicate activated neurons during the imprinting protocol. Filled neurons depict spontaneous activation of imprinted neuronal ensemble in the absence of any stimulation. (c) Recalling of imprinted ensemble by single-cell photostimulation of a pattern completion neuron (photostimulated neuron shown with arrow). Recalled neurons are shown in red. Modified from [19••], with permission from AAAS.

Control of a learned behavior with stimulation of pattern completion neurons

The activation of ensembles via pattern completion neurons, as tool, has enabled to start exploring the causal role of neuronal ensembles in behavior. Moreover, it has also shed light on the function of individual neurons in circuits, re-examining the basic tenet of the neuron theory. In fact, pattern completion could explain why earlier behavioral studies succeeded in electrically activating individual neurons to control behavior [22,23]. These studies were originally met with considerable skepticism, as they did not provide a plausible explanation of how the activation of individual neurons, in a brain composed of hundreds of millions (rat) or billions of neurons (monkey), could ever impact behavior. This difficult arithmetic could be explained if, in those experiments, the stimulated neurons had pattern completion capabilities, which triggered an ensemble. This initial ensemble could then activate a set of secondary pattern completion neurons in other ensembles, triggering an avalanche of ensemble activity, which resulted in a behavioral outcome. Interestingly, these reported studies were done in experimental conditions where the identification of driving neurons was biased by their responses to the task, and these neurons may have enhanced pattern completion capabilities.

The role of individual neurons in neuronal ensembles has been recently examined in piano experiments in layer 2/3 of primary visual cortex, where ensembles encode different orientations to drifting-gratings in non-trained mice [16,24]. To explore the role of individual neurons, a single-neuron was stimulated in the presence of visual stimuli, while the activity of the surrounding cells was recorded with two-photon imaging. This revealed feature competition properties where neurons with similar tuning were suppressed, apparently reducing information redundancy [33]. Such experiments did not find pattern completion, but the examined ensembles had not been trained to the visual stimuli. Thus, training, or imprinting (Figure 2), could be critical for pattern completion. Indeed, recent “piano” experiments with trained animals have confirmed that stimulation of pattern completion neurons in mouse visual cortex can drive behavior, by recalling specific ensembles generated by the training [20••,21••] (Figure 3). Two independent research groups used a visually guided Go/No-Go task and holographic optogenetics to demonstrate that single cell photostimulation can control behavior by triggering ensembles [20••,21••]. Mice were first trained to lick in response to drifting-gratings (Go stimulus) and two-photon calcium imaging of layer 2/3 in primary visual cortex revealed ensembles that responded to the Go stimulus. Then, pattern completion neurons from Go ensembles were computationally extracted from these responses, and their activation using holographic optogenetics, was able to recall Go ensembles and enhance behavioral responses to challenging low-contrast stimuli, or even trigger behavior in the absence of visual stimuli [20••]. In a second study, the simultaneous activation of Go ensemble neurons and visual stimuli was necessary to successfully evoke the learned behavior [21••]. Consistent with these studies, visual cortex stimulation using holographic opto-genetics enhances perception in a challenging visually-guided task [34•]. However, in less challenging scenarios, cortical stimulation actually suppresses stimulus detection, indicating that in some conditions the activity of primary visual cortex lacks meaningful information [34•]. Thus, the effect of single cell stimulation on behavioral performance could be dependent not only on the training, but also on the task demand or even on the brain state [5]. Although the mechanisms underlying these behavioral effects of single neurons activation are still under investigation, these experiments altogether demonstrate that holographic optogenetics can be used to manipulate ensembles, proving a causal relation between ensemble activity, perception and learned behaviors.

Figure 3. Recalling of neuronal ensembles by stimulating pattern completion neurons triggers visually-guided behavior.

“Piano” experiments in primary visual cortex of awake mice during a Go/No-Go visual discrimination task. (a) Neuronal Go ensemble activated by a visual stimulus (green neurons). (b) Recalling of Go ensemble by activating pattern completion neurons (red) with holographic optogenetics. (c) Recalling of Go ensemble by visual stimuli or holographic optogenetics evokes behavior. Modified from Carrillo-Reid et al., 2019. See also Marshel et al., 2019.

Future perspectives: is the neural code written with ensembles?

In summary, these recent experiments [20••,21••,34•], an outcome of the development of the “piano” combination of two-photon holographic optogenetics with two-photon calcium imaging [17,18,35,36], have started to provide the first causal evidence linking ensembles with behavior. Thus, they can be viewed as an initial confirmation of proposals of reverberating chains, assemblies, attractors or synfire chains [6,8,9,11]. The synaptic and circuit mechanisms by which this happens are still unclear and remain an area of active investigations. But regardless of the mechanisms, these recent results are consistent with the hypothesis that neuronal ensembles are functional units of the neural code. While these experiments have been performed in mouse primary visual cortex, the presence of coactive ensembles of neurons that are causally related to behavior has also been observed in cnidar-ians [37], given credence to the idea that, like the genetic code, some basic features of the neural code could be similar across phyla. These results also enable the reinterpretation of the neuron doctrine, as they have revealed the direct effect of individual neurons in behavior and perception. While this may seem a demonstration of the validity of the neuron doctrine, a critical detail from these recent results is that individual neurons only have a behavioral effect if they successfully recall an ensemble [20••]. Thus, echoing a proper Kuhnian paradigm shift [38], the new theory (the ensemble hypothesis) can explain the older one (the neuron doctrine), subsuming it.

A lot more needs to be done to explore the role of ensembles in the neural code. The neurobiology of neuronal ensembles, their anatomy, connectivity, development, synaptic mechanisms, and functional role in normal and diseased states, remains essentially virgin territory. To capture all the ensembles, a dream experiment for many neuroscientists, and one that inspired the BRAIN initiative [39], would be to visualize the spatial and temporal dynamics of every neuron involved in a behavior and then have the ability to precisely manipulate targeted neurons with single cell resolution. The development of all-optical reading and writing of neuronal activity with holographic “piano” experiments and two-photon calcium imaging has enabled an initial exploration of ensemble function, and has moved experimental capabilities one step forward toward such dream [17,35,40]. Instead of “playing the piano”, a different approach to study the effect of perturbations in local cortical circuits is the photoablation of neurons [41]. Such experiments could be useful to investigate damage sensitivity of ensembles but, like all lesion experiments, have the disadvantage of being irreversible and difficult to control. In addition, to investigate the role of ensembles of more complex behaviors it would be necessary to record and target multiple regions of the brain with single cell resolution. Thus, more powerful optical and analytical methods need to be developed, including voltage, volumetric and mesoscopic in vivo imaging methods [42–44]. Finally, the identification of neuron subtypes belonging to neuronal ensembles in conjunction with trascriptomics [45] will be necessary to target molecularly identified subtypes of neurons with holographic optoge-netics and to understand their role in functional microcircuits across brain areas.

The neuronal ensemble hypothesis could have significant repercussions for translational neuroscience and medicine. Methods to selectively activate ensembles could in principle also be applied to humans. Indeed, electrical stimulation of primary visual cortex in human subjects produces the perception of moving lines or spatially located images [46]. Scaling “piano” methods with single cell precision to humans could lead to replaying of natural scenes to guide learning. For this, it is necessary to develop methods that capture population activity on a single-trial basis, allowing the visualization of circuit mechanisms underlying brain states. Thus, to recreate a percept or a representation, the activation of specific neurons that recall neuronal ensembles in sequential order might be enough, instead of targeting hundreds or thousands of neurons. Such single cell methods could be used to manipulate memories or circuit states in mental or neurological diseases. The characterization of the circuit mechanisms underlying changes in cortical neuronal ensembles related to memory and behavior could be extended to different brain regions to define biomarkers of population activity to diagnose and intervene during pathological states [5]. In fact, ensembles are abnormally recruited in epileptic seizures [47], are less synchronized in mouse models of schizophrenia [48], have a reduced repertoire in medically-induced loss of consciousness [49] and are more numerous in a mouse model of autism [50]. Thus, deciphering the role of ensembles in cortical circuits could help to understand the pathophysiology of neurological and mental impairments and enable strategies to reprogram abnormal ensembles to rescue the functionality of patients.