Cortical computations via metastable activity

Abstract

Metastable brain dynamics are characterized by abrupt, jump-like modulations so that the neural activity in single trials appears to unfold as a sequence of discrete, quasi-stationary ‘states’. Evidence that cortical neural activity unfolds as a sequence of metastable states is accumulating at fast pace. Metastable activity occurs both in response to an external stimulus and during ongoing, self-generated activity. These spontaneous metastable states are increasingly found to subserve internal representations that are not locked to external triggers, including states of deliberations, attention and expectation. Moreover, decoding stimuli or decisions via metastable states can be carried out trial-by-trial. Focusing on metastability will allow us to shift our perspective on neural coding from traditional concepts based on trial-averaging to models based on dynamic ensemble representations.

Recent theoretical work has started to characterize the mechanistic origin and potential roles of metastable representations. In this article we review recent findings on metastable activity, how it may arise in biologically realistic models, and its potential role for representing internal states as well as relevant task variables.

Introduction

Brain circuits consist of large neural networks, where populations of neurons are recurrently coupled via synaptic connections. These circuits can be interpreted as dynamical systems with many coupled degrees of freedom and therefore capable of generating a wealth of dynamical behaviors on very diverse timescales [1]. Those include transient relaxations towards a point or line attractor, oscillatory patterns, chaotic dynamics or metastable activity [2,3]. These dynamical behaviors have been implicated in important brain functions. Transient relaxation to stable neural activity (such as a point or line attractor) may subserve memory [4,5] and perceptual decisions [6,7]. Oscillatory dynamics may subserve respiration, locomotion, and other rhythmic forms of behavior [8,9]. Chaotic dynamics amplify random perturbations and, when successfully tamed by learning, can generate complex computations [10,11]. Metastable dynamics, initially characterized in the presence of a sensory stimulus [12,13], might underlie a range of internal computations during cognitive tasks [14^••,15,16^••,17^••].

Relaxations to an attractor, oscillations and chaotic dynamics typically describe neural activity as smoothly varying over time [18,19]. In contrast, metastable dynamics are characterized by abrupt, jump-like modulations so that neural activity appears to unfold as a sequence of discrete, quasi-stationary ‘states’. Metastable activity is being found in an increasing number of brain structures of different species engaged in a variety of tasks, and is the focus of this opinion. The intuitive appeal of metastable activity is that it resonates with our intuition that our thoughts and actions proceed along a sequence of distinct episodes, as we scan alternatives and ponder potential options during everyday tasks. It is natural to think that such episodes are being represented in transient but well-defined neural patterns in our brains. Recent models have started to clarify how metastable activity could emerge spontaneously as a collective phenomenon via attractor dynamics in spiking networks [20^••,21,22^••,23^••]. These models have proved powerful tools to investigate the circuit origin of metastable states and their potential role in sensory and cognitive processes.

In this article we review recent progress in the analysis of metastable state sequences, focusing on issues of detection, modeling and interpretation.

Observing metastable activity

Metastable states were first characterized in electrophys-iological recordings from the prefrontal cortex of monkeys performing a delayed localization task [12,24,25]. Spike counts from simultaneously recorded neurons were analyzed with a hidden Markov model (HMM), a popular latent variable model for analyzing sequential data arising, for example, in time series analysis, speech recognition [26], DNA sequence analysis [27], behavioral analysis [28], and many other problems [29,30].

Under the most common application of HMM to neural data, population activity from simultaneously recorded neurons unfolds through a sequence of ‘hidden’ states, each state being a set of ensemble firing rates (Figure 1a). Stochastic transitions between states occur at random times according to an underlying Markov chain, and neurons discharge as Poisson processes with state-dependent firing rates, though the model can be extended to include refractory periods and other history-dependent factors [31]. The number of hidden states may be selected using information criteria [15,32,33^••], cross-validation [14^••], or nonparametric Bayesian methods [34,35^••]. Several measures have been used to assess the goodness-of-fit of inferred sequences, including variance explained [14^••], comparison to shuffled datasets [16^••], and comparison of single-neuron transitions to ensemble state transition [22^••]. The strength of HMM is to allow a principled, unsupervised method for segmenting neural activity into a sequence of discrete metastable states, and its use has led to intriguing new observations in ensemble dynamics, as we discuss next.

Figure 1. Observing metastability in neural populations.

(a) Schematics illustration of a hidden Markov model (HMM) with three states. Hidden states (squares) are collections of firing rates across neurons; arrows indicate transition rates among the states (thicker arrows denote larger transition rates). (b) HMM applied to electrophysiology recordings from the gustatory cortex of a behaving rat. The top panels show spike rasters from 10 simultaneously recorded neurons segmented via HMM analysis. Two trials are shown, one in which sucrose (left) and one in which quinine (right) was administered to the rat. HMM states were assigned to each bin of data when their posterior probability exceeded 0.8 (dashed horizontal line). The bottom panels show the state sequences in four different trials with either sucrose (left) or quinine (right) as a taste stimulus. Transitions may occur at variable times across trials, but the state sequences are reliable. Panel (b) modified from Ref. [13].

Consistent features of metastable activity

HMM and related procedures for detecting hidden states have established a number of consistent features across datasets, animals and tasks. Ensemble activity unfolds as a sequence of metastable states, each lasting from a few hundred ms to a second or more, with sharp transitions among the states [12,13,15,22^••] (Figure 1b). The transitions are typically one order of magnitude faster than the state durations, are close to their theoretically observable lower bound, and are not an artifact of the HMM [12,13,15]. When the start of the sequence is aligned to an external event such as stimulus onset, specific state sequences are observed (Figure 1b), raising the possibility that stimuli are coded by dynamic state sequences [12,13].

Among the most significant early results of applying HMM to the analysis of neural data were the demonstration that (i) the hidden states identified in response to a stimulus tend to recur during most of the recorded activity, not just in response to the stimulus, and (ii) pairwise correlations among simultaneously recorded neurons depend on an underlying global activity state, and not just on neural connectivity [12,24]. These two observations are very relevant for current issues in neuroscience, including the origin and role of ongoing activity [36,37], the importance of the connectome [38,39], and the meaning of correlations [40-42].

It is worth remarking that the ability of HMM analysis to segment the neural activity into distinct states in an unsupervised manner – that is, without knowledge of external triggers such as stimulus timings, – has been crucial to obtaining the results reviewed above. It is also important to point out that the HMM analysis on ensemble data allows to decode the neural activity on a trial-by-trial basis, avoiding the necessity of averaging across trials. Most findings discussed in this review would go undetected if data were averaged across trials, because ensemble transitions occur at different times in each trial. This latter feature also allows the time-warping of state sequences, explaining why, for example, the method is so useful for speech recognition and for the analysis of neural activity during birdsongs [43]. This could also explain part of the trial-to-trial variability observed in cortex after aligning spike trains across different trials to, for example, stimulus onset [13].

Metastability of ongoing cortical activity

One central question in contemporary neuroscience is why ongoing cortical activity is so rich in structure and resembles so closely stimulus-evoked activity [36,37,44-49]. The origin of ongoing activity and the nature of its interaction with evoked activity can potentially reveal much about the way cortical networks are functionally organized and how this may support flexible coding (see Refs. [36,37] for two recent proposals). However, quantifying precisely the similarities and differences between ongoing and evoked activity has proved difficult. A recent HMM analysis of ongoing spiking activity in the gustatory cortex of behaving rats has demonstrated that ongoing activity, similarly to evoked activity, unfolds as a sequence of metastable states [22^••]. Transitions among states are characterized by a partial degree of coordination, so that ongoing activity is neither completely asynchronous nor completely synchronized. Ongoing and stimulus-evoked activity share most of the same states, although some of the states occur mostly during ongoing activity and others during evoked activity [33^••]. About 50% of the ensemble neurons exhibit three or more different firing rates across states, that is, they are ‘multi-stable’ rather than bistable; and this fraction is substantially reduced by stimulation [22^••]. These findings have all been captured by a spiking network model of metastability that might shed light into the link between ongoing and evoked activity (see section ‘Spiking network models of metastable activity’).

Metastable sequences as a substrate for internal computations

Do metastable states have precise and specific meanings as neural correlates of external or internally generated events? Visual [12], gustatory [13,22^••] and vibro-tactile stimuli [15] seem represented by reliable sequences of HMM states in primate and rodent cortical areas in the context of different tasks.

Even more remarkably, though, aspects of the metastable dynamics seem to subserve internal states of attention, expectation, and internal deliberations during decisions, and can be used to predict behavior. In monkeys performing a selective attention task, the probability of detecting a change in the attended stimulus was significantly greater when occurring during an ‘ON’ state (characterized by vigorous multi-unit activity in area V4) than during an ‘OFF’ state (characterized by faint multi-unit activity) [14^••] (Figure 2a). Similarly, different hidden states were preferentially associated to different task conditions in a human working memory task, with occupancy rate in each state predicting better performance in the corresponding task [35^••]. Moreover, changes in patterns of functional connectivity across many brain areas co-occurred more reliably with state transitions than with external triggers, suggesting that metastable states supported by flexible patterns of functional connectivity may reflect internal representations of task demands.

Figure 2. Metastable activity and cognitive function.

(a) Left: Multiunit activity from monkey V4 during a selective attention task alternates between ON (green) and OFF (pink) HMM states. Right: Behavioral performance improves when the decision occurs during ON intervals compared to OFF intervals. (b) Left: Probability of being in a state representing the value of the chosen (red), unchosen (blue) and unavailable (gray) option from monkey OFC ensembles in a decision-making task. Right panels: Comparison of time-course of state probability in quick versus deliberative decisions as judged by the dynamics of eye movements (depicted at top). (c) Left: Dynamics of HMM states decoded from hippocampal neural activity (top) and position (bottom) of a rat running along a linear track (six runs; only data during sharp wave ripples, comprising 2% of the session, were used). Right: Mapping latent state probabilities to associated animal positions yields latent-state place fields describing the probability of each state for every position on the track. (a) panels modified with permission from Ref. [14^••]; (b) panels modified with permission from Ref. [17^••]; (c) panels modified from Ref. [16^••].

A similar relationship between state occupancy rate and decisions has been found in monkeys performing a choice task between differentially rewarded stimuli [17^••]. Ensemble activity in orbitofrontal cortex (OFC) mostly switched between two states representing the values of the stimuli currently being offered, with larger occupancy rate predicting the behavioral decision (Figure 2b, left). Intriguingly, slower decisions (reflecting longer internal deliberation) occurred when the states had similar occupancy rates and were found on trials of the same kind (thus reflecting subjective preferences rather than actual task difficulty; Figure 2b, right). These findings are reminiscent of those on dynamic changes of mind [50] and complement earlier intriguing results on the possible meaning of state transitions, which may underlie the sudden realization that task rules have changed [51] or reflect the difficulty of a vibro-tactile discrimination [15].

In the rat hippocampus, HMM states were recently found to represent the position in a linear track and in an open field during a navigation task [16^••] (Figure 2c), an alternative view to decoding spatial maps based on single neurons place fields [52]. The state sequences were observed in area CA1 during hippocampal sharp wave ripples when the animal was not exploring the track, and could be used to infer a spatial map of the environment without any reference to external locations [16^••].

In rat gustatory cortex, state sequences after stimulus delivery proceed faster when the delivery is expected compared to when it is not expected [33^••], which may explain the faster decoding of expected stimuli found in this area [53] (a spiking network model explanation of this phenomenon is discussed in the next section).

The examples discussed in this subsection make a convincing case that metastable sequences underlie a variety of important cognitive processes, and more examples will likely be found across different cognitive domains not yet explored with latent state models.

Spiking network models of metastable activity

In addition to statistical descriptions in terms of HMM and related latent state models, mechanistic accounts of metastable activity based on biologically plausible networks of spiking neurons have also been given. We shall mostly focus on efforts that have purposefully combined spiking network modeling with HMM [22^••,33^••,54,55], as we believe that such an approach is the most promising to understand the potential origin and function of metastable activity in neural circuits. Metastable activity naturally occurs when multiple hidden states are attractive fixed points of the neural dynamics which are either inherently unstable, or can be destabilized by internal noise or external perturbations [56-58]. Such fixed points would attract the dynamic trajectories of the neural activity and force them to linger in their neighborhood for a finite amount of time. It is natural, therefore, to look for models wherein metastability is caused by the coexistence of multiple attractor states.

Metastable activity in clustered networks of spiking neurons

Among the class of attractor neural networks with multiple attractor states, spiking network models allow to build biologically plausible models capable of generating metastable state sequences. Recurrent networks with strong synaptic connections can produce highly temporally fluctuating activity [59-67], often the signature of chaotic activity. However, to endow a network of spiking neurons with endogenously generated metastable states, a specific partition of the network in subpopulations of neurons seems required [20^••,21,22^••,23^••]. The clustered architecture has emerged as an effective way to generate metastability. The models studied so far comprise excitatory and inhibitory spiking neurons with (typically) the excitatory neurons organized in clusters, as seems the case in cortical circuits [68-70]. The common feature of these models is that the average synaptic strength (and/or connectivity) inside clusters is larger than between clusters (Figure 3a, right). A mean field analysis of these models [4,22^••,54] shows that multiple attractor states are possible depending on the average intra-cluster synaptic weight (ICW; Figure 3b). Above a critical ICW value (dashed green line in Figure 3b), multiple configurations of the network emerge wherein neurons in clusters tend to occupy several states with different values of mean firing rates (grey and red lines in Figure 3b), where the firing rates depend on the number of active clusters. In an infinite network, those configurations would be stable attractors; but in a finite network, or in the presence of sufficient external noise, those configurations become metastable. Note that if the ICW is large enough (e.g. full green line in Figure 3b), the configuration with no active clusters is unstable even in the infinite network, and at least one cluster must be active at any given time.

Figure 3. Spiking network model of metastable activity.

(a) Spike rasters of a spiking network model of excitatory (E) and inhibitory (I) integrate-and-fire neurons with two different architectures. Left: homogeneous network. Right: clustered network. The homogeneous network has uniform connectivity among the E neurons. In the clustered network, the E neurons are organized in clusters, so that synaptic connectivity is stronger inside each cluster than among clusters. The spiking activity in the clustered network is metastable. Insets: graphical description of the topology of each network. (b) The mean field theory of a clustered spiking network with 30 E clusters shows coexistence of several attractors for intra-cluster synaptic weight (ICW) J₊ beyond the critical value 4.2 (dashed vertical green line; J₊ is in units of the baseline synaptic weights outside clusters). The curves represent the firing rates of neurons in each cluster according to the number of active clusters (numbers from 1 to 8). Grey and purple curves: E neurons; red curves: I neurons. A state on the purple curve is ‘globally inactive’ in the sense that no clusters are active and firing rates of E neurons remain low. (c) Raster plot from a typical simulation of the model in (b) with ICW corresponding to the full vertical green line. Thirty E neurons (one for each separate cluster) are shown, together with the segmentation of the rasters in HMM states. Panel (a) adapted with permission from Ref. [20^••]; panel (b) adapted from Ref. [54]; panel (c) adapted from Ref. [22^••].

It is unclear whether suitable connectivity patterns exist such that metastable itinerant dynamics is possible also in a deterministic network of infinite size [71]. In the following, we assume that fluctuations originate in finite size effects in a deterministic network [20^••,22^••], and later consider alternative models (see subsection ‘Variations on the clustered architecture’).

In finite clustered networks above the critical ICW value, a combination of erratic spiking activity and recurrent inhibition can ignite an itinerant exploration of multiple configurations (as in Figure 3a, right), whose number and features depend on network parameters such as the ICW. When an HMM analysis is run on these networks, segmentation of the spontaneous ongoing activity in discrete states is found (Figure 3c). Such metastable activity is modified, but not removed, by an external stimulus. Since the firing rates depend on the number of active clusters, the neurons exhibit multiple firing rates across different HMM states, as found experimentally in rat gustatory cortex [22^••]. Note that the neurons themselves can have graded activity; hence, their multistability is an emergent property of the collective network’s dynamics and not an intrinsic property of the units.

Predictions of the clustered spiking network

Clustered spiking networks capture many statistical and dynamical features of the data, including a stimulus-induced reduction of: (i) trial-to-trial variability [20^••,21,72], multistability of the firing rates across states [22^••], and neural dimensionality [54], that is, the minimal number of effective dimensions (between one and the number of neurons) necessary to describe the ensemble dynamics [73-75]. It is worth noticing that while the stimulus-driven reduction of trial-to-trial variability has been a catalyst for the development of the clustered network model [20^••,21], the reduction of multistability and dimensionality was found to be a naturally emergent property of the model which led to finding the corresponding properties in the data.

These predictions capture aspects of the data and thus help to corroborate the validity of the model. However, two other recent predictions link metastability directly with sensory and cognitive functions. The first is that visiting metastable states during ongoing activity will help to maintain learned stimulus responses and improve performance, suggesting yet another role for ongoing activity. This prediction emerged from an attempt to understand how neural clusters and metastable states can emerge during learning, and revealed the need to complement synaptic plasticity with homeostatic mechanisms [76-78]. The second prediction has led to the proposal that a state of expectation could be understood as an acceleration of metastable dynamics [33^••]. When rats are trained to respond to taste stimuli, the identity of the stimuli can be decoded faster from the neural activity if stimulus delivery is expected compared to when stimuli are randomly, and passively, administered [53]. In the model, the onset of a predictive cue modifies the network’s landscape of metastable configurations so as to make state transitions more frequent. Faster state transitions, in turn, induce faster onset of stimulus-coding states after stimulus delivery, an effect uncovered by an HMM analysis of the model and confirmed in the data.

Variations on the clustered architecture

Variations of the clustered network architecture and its generated dynamics can explain related kinds of metastable activity such as perceptual bistability [79,80], the emergence of specific slow oscillations known as ‘up’ and ‘down’ states [81], or networks that can sustain bidirectional sequence propagation at slow and tunable speed [82]. These types of metastable dynamics are different from those reviewed earlier in subtle, but significant, ways.

Clustered spiking networks have also been used to uncover subtle differences between mechanisms for decision making. Specifically, spiking networks with multiple attractors built for decision making [7,83] can work in two different modes, the ‘ramping’ mode, characterized by gradual stimulus-driven firing rate increases, and the ‘jumping’ mode [55,56], wherein the stimulus ignites a sequence of metastable states. By leveraging a mechanism reminiscent of stochastic resonance, the jumping mode can outperform the ramping mode (underlying the so-called ‘integration-to-bound’ models [84]) during perceptual decision-making in the face of sensory noise.

Finally, we briefly mention that models with endogenously generated activity are not necessary to produce metastable activity. External fluctuations or the presence of firing rate adaptation both can create instability and lead to metastability in a finite network with multiple attractors [85,86] (including ‘up’ and ‘down’ states at low firing rate, [87]). Sometimes both ingredients are required to reproduce the statistics of the empirical data [87,88]. Also, it is possible to obtain stimulus-evoked quenching of variability in non-metastable networks, such as networks with chaotic attractors [89] or the supralinear stabilized network (SSN) [90-92]. These models capture the reduction of variability via different mechanisms and make different predictions, such as the tuning-dependent modulation of variability observed in primary visual cortex and captured by the SSN [92].

Conclusions

We have reviewed recent progress made in elucidating the nature and the role of metastable states in brain function. Thanks to the availability of ensemble data from behaving animals, the study of metastable activity is changing the way we think about neural coding for both external events and internal representations.

We have also reviewed a class of spiking network models that promise to provide a mechanistic understanding of the origin and function of metastability. These models have demonstrated the importance of a clustered architecture for the spontaneous generation of metastability, the coexistence of ongoing and evoked metastability, the benefits of metastable sequences to decision making, and the explanation of faster coding of expected events via the acceleration of metastable dynamics. Because of their biological plausibility, not only can these models shed light on the mechanisms, but they can also be powerful tools for analyzing metastable states, predict causal relationships to behavior, and help to design new experimental studies.

In the future, it will be important to establish a causal relationship between metastable states and behavior, by manipulating neural activity during behavioral tasks while recording from large ensembles across multiple sites. It will also be important to establish the emergence of neural clusters and metastable dynamics though development or learning. Recurrent spiking network models, in conjunction with latent-state models such as HMM, are likely to play an important role in his endeavor.