Itinerancy between attractor states in neural systems

Abstract

Converging evidence from neural, perceptual and simulated data suggests that discrete attractor states form within neural circuits through learning and development. External stimuli may bias neural activity to one attractor state or cause activity to transition between several discrete states. Evidence for such transitions, whose timing can vary across trials, is best accrued through analyses that avoid any trial-averaging of data. One such method, hidden Markov modeling, has been effective in this context, revealing state transitions in many neural circuits during many tasks. Concurrently, modeling efforts have revealed computational benefits of stimulus processing via transitions between attractor states. This review describes the current state of the field, with comments on how its perceived limitations have been addressed.

Graphical abstract

Introduction

Attractor states come in many forms, the simplest of which is a point attractor. For example, the firing rate of a neuron responding to a constant stimulus is likely to approach a constant value after a transient period. The constant final rate is a point attractor, because if the rate were temporarily altered (say by a brief pulse of excitation) it would return to its prior constant value. Importantly, at a more detailed level of description—that of the membrane potential, rather than the firing rate—the point attractor is revealed to be a cyclic attractor as the neuron spikes at regular intervals following a periodic voltage trace. If perturbed from the periodic trace, the membrane potential would return to the same orbit, albeit with a phase shift. So the notion of a point attractor can depend on the level of description: a point attractor at one level can be comprised of other types of attractor at a more detailed level. In this review we focus on systems possessing many point attractors when described at the level of neural firing rates. Systems with multiple point attractors are important because the system’s state—i. e. which point attractor it dwells at—becomes history-dependent, so the system can represent memories and respond to stimuli in a sequence-specific manner.

Models of neural circuits containing multiple point attractor states have a long history in theoretical neuroscience [1–8]. These circuits can learn discrete input patterns and later reproduce them upon presentation of altered or incomplete versions of the original pattern. In these circuits different sets of neurons—whose coordinated activities comprise the neural representations of the original stimuli—can maintain their activity in the absence of direct external stimulus. Evidence from many short-term memory tasks indicates that neural circuits are indeed capable of producing such stimulus representations that persist in the absence of external stimuli [9,10].

When neural activity transitions between different attractor states, the term attractor-state itinerancy is used. The transitions can be prompted either by transient external stimuli or due to internal processes. A trivial example of the latter is loss of persistent activity representing short-term memory during a delay due to internal noise fluctuations [10–13].

A more interesting example is the bistable percept, models of which invoke negative feedback such as firing rate adaptation to weaken an active cluster of neurons until an inactive cluster takes over (Fig. 1) [14,15]. Noise causes the timing of transitions between states to be variable, leading to an important observation: the average firing rate across trials, being a weighted mean of the activities corresponding to the two percepts, would not reveal any switching between percepts, which is the most relevant aspect of the activity. Trial-specific analyses, such as hidden Markov modeling (HMM) [16–19], are essential for uncovering such switches between states when the transition times are variable.

Figure 1. Noise-induced transitions generate itinerancy between the two attractor states of a bistable system, accounting for perceptual alternation.

A) Three separate trials (vertically aligned panels) indicate back and forth switching between states where one unit’s activity dominates over the other, but with across-trial variability in the transition times. B) Activity averaged over ten trials, as is common for neural data, obscures all signs of the underlying dynamics. Blue and red curves denote activity of separate excitatory cell-groups.

Evidence for multiple attractor states and itinerancy

Existence of discrete attractor states

Both perceptual reports and electrophysiological measurements provide evidence for multiple attractor states [20–24]. By its definition, a point attractor requires that in the presence of a fixed stimulus, if neural activity is perturbed a little, it returns or is attracted back to its prior pattern. However, given the difficulty of such perturbations, the presence of attractors has been inferred by responses to gradual variation of the stimulus [20]. As a stimulus is parametrically altered, the neural response or percept changes by small degrees until a transition point is reached, at which a small change in stimulus causes a large change in neural response and percept as the prior attractor state destabilizes and a new one is reached [21]. The observation of hysteresis, meaning that the transition point depends on recent history, strongly suggests the presence of discrete attractor states [22]. This type of evidence for attractor states has been found for simple auditory percepts [23], for face recognition [24], and for spatial environments [22] across a number of species.

Evidence from neural activity for state transitions in the absence of stimulus change

HMM is a method for ensemble data analysis that can reveal any reliable structure underlying distinct states of neural activity as well as any pattern in the order that states are visited by a network. A description of the operation of HMM is presented in Figure 2. Hidden Markov models of multiple simultaneously recorded in vivo spike trains have revealed transitions between relatively stable discrete activity states in premotor and prefrontal cortex of monkey performing a spatial short-term memory task [17,25,26] or freely viewing a scene [27], and in gustatory cortex of rats processing tastants [28]. The timing of such state transitions both correlates with ongoing behavior [27,29] and is predictive of an ensuing response [30].

Figure 2. Hidden Markov modeling (HMM).

A) HMM operates on sets of simultaneously recorded spike trains, from either a single time-series or (as in this case) multiple trials. B) HMM assumes a number of discrete states (colored ovals) with a matrix of transition probabilities per unit time between the states (represented by the thickness of arrows). Each state is defined by the probability of emission of a particular pattern in a time-bin (inset histograms). For neural spike trains an emitted pattern is the combination of neurons producing a spike in a small time bin, so a state’s emission probabilities are directly related to the firing rate of each neuron in that state. HMM proceeds iteratively in a two-step process: 1) Given a possible model of transition and emission probabilities, it calculates the likelihood the observed spike trains are produced by the model, giving the probability the system is in any given state at any point in time on any trial. 2) Given the fitting of state probabilities to the spike trains, both the emission probabilities of each state and transition probabilities between states are recalculated. Repeated iteration of steps 1) and 2) increases the log likelihood of the match to the data until a local optimum is reached. Typically numerous random initial conditions for the emission and transition probabilities are used in case one particular set of iterations reaches a local optimum that is not the global optimum. C) Given the optimized HMM, the posterior probabilities for each state are ascribed to the time-series of spike trains (colored solid lines overlaying the spike trains).

Single spike trains from monkeys have been used to provide evidence for state transitions that do not appear in trial-averaged data. During motor planning, within-trial spike counts taken from anterior cingulate cortex over a time-window near the mid-point of a delay yielded a bimodal distribution across trials [31]. A more sophisticated analysis of lateral intraparietal cortex data taken during a decision-making task used a latent variable model of single spike trains to uncover state transitions [32].

Switching between attractor states is necessary for an internally generated switch between motor output patterns [33]. Transitions between discrete motor states have been revealed by HMM for the crawling of c. elegans [34], the sequences of syllables in bird songs [35], and lever-pressing rates in rats [36]. Since the observed motor patterns are produced by underlying neural activity, these studies lend indirect evidence to transitions between attractor states within neural circuits.

Since attractor states are produced by the internal structure of a neural circuit it is likely that spontaneous activity, in the absence of a stimulus to drive activation of a particular state, would randomly sample the network’s activity states [37–41]. This feature combined with synaptic plasticity, can stabilize discrete attractor states [42]. In support of the hypothesis that spontaneous activity samples attractor states, multi-electrode data from awake ferrets shows that spontaneous activity in vivo approaches activity evoked by natural stimuli over the course of development [43], with statistics more compatible with wandering between multiple states than with fluctuations about a single state [44]. Recently, direct evidence from HMM demonstrates that spontaneous activity in rodents can exhibit rapid transitions between discrete states [45].

Whole-brain data from fMRI or MEG [46,47], and large-scale simulations of the brain based on tractography studies [47,48], provide evidence for rapid transitions between a few discrete functional connectivity states in the resting state of humans. These transitions and the underlying discrete states would not be revealed by time-averaged data [47–49].

State transitions without point attractors

The wealth of evidence for state transitions does not necessarily imply stochastic switching between point attractor states. Indeed, in EEG measurements, state transitions are observed as changes in the amplitude, phase, or frequency of oscillatory patterns [50,51]. While a transition between point-attractor states in one circuit could trigger a change in oscillation in another (via e. g. neuromodulator release [52]), it is also possible for a circuit possessing multiple oscillatory (cyclic) attractor states [53,54] to transition stochastically between the states.

Similar state transitions can also arise in deterministic systems, either through chaos [55,56] in which case states have an oscillatory structure [57], or through heteroclinic sequences [58–61] in which case states are weakly unstable fixed points. When comparing these deterministic dynamical models with point-attractor state itinerancy, both the statistics of state-durations across trials and the dynamics of neural activity within a state should differ. Extensions of HMM can extract such internal state dynamics [62] to help determine which dynamical system is present.

Computational models with attractor-state itinerancy

Network multistability can arise from distinct bistable units or as an emergent property of connected monostable units

Multistability can arise from recurrent feedback in the circuit when the firing rate of single neurons varies continuously with input (i. e., when single neurons taken in isolation show neither jumps in firing rate nor hysteresis so are monostable). In such cases, the representations of stimuli are distributed, meaning each pattern requires coordinated firing of many neurons and each neuron can take part in many patterns, typically with independent groups of other cells [6,8]. Chaotic networks comprised of standard monostable single units can be trained to produce a discrete number of stable network states whose activity can be switched by transient inputs [33,63,64]. Randomly connected oscillators can also produce multiple stable states, distinguished by phase relationships between oscillators, as well as amplitude and frequency of oscillation [65].

In another approach, several groups have focused on the behavior of clusters of neurons, also called assemblies [66], within which preferentially strong connections [67–69] generate a single bistable unit. Such bistable units could arise from a Hebbian strengthening of connections among coactive excitatory cells, eventually leading to similarly responsive cells being preferentially connected to each other [42,70], as observed empirically [71].

In models, the level of cross-inhibition between bistable units determines the number of coactive units. If only a single unit can be active due to strong cross-inhibition, i. e., in a winner-takes-all circuit, information is stored very sparsely and the number of stable states equals the number of units. If multiple units can be active at once due to relatively weak cross-inhibition, there is a combinatorial explosion in the number of potentially stable states, suggesting great computational benefits. The information capacity of such circuits is demonstrated in their ability to dissociate the number, duration and amplitude of repeated identical stimuli [72] while the richness of their dynamics is revealed in their ability to produce chaotic transients while also containing many stable states [73].

Attractor-state transition models for decision-making

Decision-making models with attractor-states representing the final choice yield many of the neural dynamics observed in perceptual choice tasks [74–77]. In a variant of such models, the initial “undecided” state can itself be an attractor state, in which case decisions are made by noise-induced transitions [78,79].

Evidence for a delayed rapid transition from one discrete attractor state to another during decision making has been accrued during taste processing (where the decision is to expel or ingest a tastant) [28,30], in a somatosensory task [80], and in a detection of motion direction task [32]. However, an alternative analysis of the latter task revealed ramping as the typical dynamics during stimulus acquisition, with rapid transitions corresponding only to changes of mind [81].

Surprisingly, noise-induced transitions between point attractors can outperform the optimal method of integration for deciding between noisy stimuli [79], if biological constraints such as the limited range of neural firing rates and the presence of within-circuit noise are taken into account. In such circuits a particularly strong fluctuation can kick the activity out of an “undecided” state into one of the decision states, such a transition being more likely to the decision state with greater underlying activation. The advantage of such attractor-based methods is that internal noise, which limits the usefulness of perfect integration [82] (it accumulates as Brownian motion in standard integration models of decision making), can actually boost performance by attractor transitions, generating a decision within an appropriate time window, in a method resembling stochastic resonance [78].

Bayesian sampling with multiple attractor states

In tasks with a constant stimulus, noise-driven transitions between attractor states can account for the distribution of durations of individual competing percepts [14,15,83]. Moreover, point attractor models of perceptual rivalry are compatible with the hypothesis that percepts represent the outcome of a Bayesian sampling process if the fraction of time spent in an attractor state is equal to the posterior probability of that percept corresponding to the stimulus [84]. Importantly, synaptic plasticity in circuits with multiple attractor states can provide a basis for Bayesian computations [85–87].

Recall of sequences via attractor-state itinerancy

Models of word recall, within which each word is stored as a particular attractor state, have explained statistical features of recall patterns from word-lists [88–91]. Within these models, transitions are facilitated by adaptation—as proposed by others in the more general framework of latching models [92–96]—combined with oscillations in external drive. The sequence of recall or the sequence of words within a grammar [97] can be related to the connectivity between neurons comprising the corresponding attractor states, or the number of units shared between different states. The connectivity necessary for recall of sequences within winner-takes-all circuits can be achieved by spike-timing dependent plasticity [98–100] or other bidirectional Hebbian plasticity rules [61].

Are attractor-state models contradicted by physiological data?

Variability of neural activity in multi-attractor systems

One argument against taking point-attractor models seriously is they appear to preclude the obvious variability in neural activity apparent at all spatial and temporal scales. Since the attractor state is a fixed point of the dynamics, from a mathematical viewpoint once in an attractor state neural activity is fixed and static. Therefore one has to be careful about the extent to which an attractor description is valid [101]. In particular, as mentioned in the opening paragraph, the level of description is important—a high-level variable (such as mean firing rate of a group of neurons) can be static, so in an attractor state, even when the microscopic constituents of that high-level quantity vary incessantly.

From an empirical viewpoint, when HMM produces a better description of spike trains than does the PSTH (e. g., as measured by log likelihood [28]) one should take the attractor state model seriously. From a modeling perspective, one can separate time-scales, averaging over the very fast time-scale of single spikes (a few ms), while taking slowly varying properties as parameters for a model of the dynamics of the intermediate time-scale (tens of ms). If the intermediate time-scale model possesses attractor states then much can be learned from an attractor-state formalism, even if these states vary slowly and disappear over time. Indeed, as external stimuli, neuromodulators, and internal facilitating or adaptive processes cause the attractor landscape to shift, biasing neural activity toward new states, so the computational repertoire of such systems is revealed.

Taken in this light, attractor-state models do not preclude variability, even in a high-level variable such as the mean firing rate of a group of cells. Within an attractor state, variability of neural firing rate arises both from the slow drift of parameters and from inherent noise. Greater changes in neural activity arise from the switches between attractor states, which provide slow timescales for activity variation, that can span orders of magnitude [102]. Finally, attractor models can explain the observed reduction in trial-to-trial variability of neural activity upon stimulus presentation [103] when an external stimulus forces the activity into a particular state, following a period of itinerancy between many states during spontaneous activity [45,104–106].

Firing-rate jumps between discrete attractor states can be small

Early implementations of discrete attractor models suggested the firing rates of neurons comprising a unit would jump between zero and a maximal level as the unit switched between “on” and “off” values in different attractor states. However, differences in neural firing rate of much more than 10Hz are rarely seen in persistent activity or during the non-transient period of a constant stimulus [107,108]. Thus discrete attractor models appeared to be at odds with the data.

In most models the firing rate of the “on” state is limited only by saturating feedback. This can be kept in check via synapses dominated by NMDA receptors with a slow time constant [109] or when there is partial overlap of neurons representing the states [8]. Alternative models demonstrate that low firing rates of cells in the “on” state are possible if the network’s stable states do not depend on the two stable fixed points of excitatory cells with strong feedback [110–112]. In one case, high levels of noise shift activity away from deterministic fixed points [110], while in the other case, the inhibition-stabilized (IS) regime is used [111,112].

The IS network is intriguing (Fig. 3), as strong negative feedback stabilizes activity of excitatory cells at what would otherwise be an unstable fixed point (like a pencil balancing on its tip). The IS network is deemed “paradoxical” in that a reduction of external excitatory drive to inhibitory cells leads to an increase in their activity due to a resulting increase of within-circuit excitatory input. The excitatory cells stabilize at a higher firing rate with increased inhibition because of the concurrent destabilizing positive excitatory feedback. Of note, if units of excitatory and inhibitory cells in an IS circuit are weakly connected with each other, they can generate multiple stable “on” states of arbitrarily low firing rates (Fig. 3) [111]. Moreover, unlike other attractor models that rely on cross-inhibition, inhibition can rise in phase with excitation in the IS regime (Fig. 1b), as observed in many experiments [113–115].

Figure 3. Low firing rates and in-phase inhibition in the inhibition-stabilized regime.

A)–C) Two bistable units (one blue, one red) with symmetric cross-inhibition (excitatory connections to inhibitory cells between units) switch between states in which one unit’s activity dominates. D–F) With alternative circuit parameters, each excitatory-inhibitory unit is in the inhibition-stabilized (IS) regime, allowing for bistability between states of low firing rates. In the IS regime (E–F), inhibitory input to a cell covaries with excitatory input—notice that cells from the same group are in phase—unlike standard multistable models (B–C) in which cross-inhibition dominates—notice that cells from the same group are out of phase, because activity of inhibitory cells is determines most by the other unit’s excitatory cell. A), D) Arrows denote excitatory connections, solid circles denote inhibitory connections. B–C), E–F) Blue and red traces denote the two cell-groups. See supplement for code that generated this figure.

Discussion

Attractor states are inevitably present in any coupled system, because the value of one variable constrains to some extent the possible values of other variables. An attractor state simply means that there is a restricted set of combined values of all the variables, toward which the system returns following any small deviation from that restricted set. Even in chaotic neural systems, the combinations of firing rates that the system moves through over time are restricted to a small set of the possible combinations, so chaotic systems possess strange attractors. In oscillators, the phase relationship between different variables is typically fixed and returned to following a deviation, so oscillators are also attractors, namely cyclic attractors. (See [53] for a more general review with examples of attractors in neural circuits.)

Given that the observable ongoing dynamics of neural activity is more compatible with other types of attractor state, a focus on point attractors may seem artificial. However, whenever neural activity persists in one distinguishable pattern for a time much longer than the switching time to the next pattern, a model of switching between point attractors is a valuable starting point for analysis of the system. Even if the distinct patterns are oscillatory in nature, point attractors can be used in a model, in which the transitions are induced by an additional oscillatory term rather than by noise alone.

A method of analysis like HMM can reveal abrupt state transitions in systems without point attractors, so long as the state-specific features of neural activity are included in the analysis. When the states are distinguished by significant coordinated changes in firing rate across many neurons, as is the case when the states are distinct point attractors, then the spike counts in small time-windows are the appropriate variables for HMM analysis. However, when states are distinguished by oscillation frequency, then the amplitudes of different frequency components of the Fourier transform of small time-windows are the appropriate variables for HMM analysis. Whether neural activity patterns are epitomized by punctate jumps between relatively stable patterns, or by a gradual meandering of firing rates without any semblance of stationariness, should be revealed in the coming years as larger datasets of simultaneously recorded neurons are analyzed. HMM will be a valuable tool in this enterprise.

Conclusions

Circuits with multiple discrete attractor states have many computational benefits, are easily formed by known mechanisms of synaptic plasticity, and are suggested by both behavioral and electrophysiological data when trials are analyzed separately from each other. Therefore single-trial analysis methods, such as hidden Markov modeling, are to be recommended for any investigation of the dynamics of neural activity, with a view to extracting any underlying attractor-state itinerancy.

Highlights.

• Multiple point-attractor states form in neural circuits with clustered connectivity.

• Transitions between attractor states can arise without a change in stimulus.

• Transitions and final state reached can be stimulus-history or context dependent.

• Variability of the timing of transitions across trials renders the PSTH inadequate.

• Hidden Markov modeling reveals such state transitions in multi-unit data.