Looking for cognition in the structure within the noise

Abstract

Neural activity in the mammalian CNS is determined by both observable processes, such as sensory stimuli or motor output and covert, internal cognitive processes that cannot be directly observed. We propose methods to identify these cognitive processes by examining the covert structure within the apparent “noise” in spike trains. Contemporary analyses of neural codes include encoding (tuning curves derived from spike trains and behavioral, sensory, or motor variables), decoding (reconstructing behavioral, sensory, or motor variables from spike trains and hypothesized tuning curves), and generative models (predicting the spikes trains from hypothesized encoding models and decoded variables). We review examples of each of these processes in hippocampal activity, and propose a general methodology to examine cognitive processes via the identification of dynamic changes in covert variables.

Introduction

The standard neurophysiological approach to understanding how neuronal activity encodes information is to examine neural activity while a subject is repeatedly presented with the same stimulus, or performs multiple trials of the same behavior. Typically, the relationship of a single neuron’s activity to an overt variable is described by the cell’s tuning curve, which is constructed under the assumption that noise sources are independent of the overt variable of interest and will thus average to zero in the tuning curve. In addition to this noise assumption, the standard approach also assumes that neurons independently contribute to the representation of the overt variable so that a neural representation can be understood by computing an average of the independent tuning curves.

Although the standard “encoding” approach has been used to successfully characterize neural representations of overt variables in several areas of neuroscience, it has been more difficult to apply the encoding approach to identify representations of covert cognitive variables like attention, decisions and planning. One difficulty is found at the level of experimental design because such covert variables may not be expressed repeatedly and reliably, even given an identical set of external conditions. However, even beyond this experimental difficulty, the noise and independence assumptions that underlie the definition of tuning curves are at odds with the theoretical framework of ensemble representations e.g., cell assemblies¹. Within an ensemble framework, information is represented by conjoint activity distributed across a functionally-defined neuronal subgroup or cell assembly composed of many neurons. Furthermore, each neuron in a cell assembly can participate in other cell assemblies that represent different overt and covert variables. According to this view, understanding a neural representation requires characterizing conjoint neural activity rather than just an average from individual neurons. It is also unlikely that a neuron will be tuned to a single variable. Both of these considerations are difficult to reconcile with the standard encoding approach based on tuning curves.

A growing body of evidence supports the cell assembly framework, especially in recent studies of the representation of location in hippocampal ensemble activity. In this review we consider evidence that what may appear to be noise in tuning curves actually results from operation of an experimentally unobserved (covert) variable or process. We focus on recent studies of hippocampal place cell representations of locations, which have gone beyond the encoding approach to provide compelling evidence of covert cognitive variables in hippocampal discharge. Based on these studies, we outline a general generative approach for finding covert cognitive variables in the apparent noise of tuning curves.

An early example: Decoding mental rotation in cortical discharge

An important study by Georgopoulos et al. showed that coherent, cell assembly-like dynamics within a neuronal population were related to cognitive function². Monkeys were trained to reach toward a target selected on each trial from one of eight potential directions around a circle . However, on a subset of probe trials, a bright target indicated that the rewarded direction would be rotated ninety degrees from the signaled direction. Georgopoulos et al.² characterized the correspondence between observed spiking activity in the motor cortex immediately before reaching and the direction of the subsequent reach by computing tuning curves for a population of cells during the unrotated trials. Prior to reaching on the rotated probe-trials, spiking within this population changed dynamically, such that the orientation represented by this population rotated through 90 degrees. This study showed that apparent noise activity in the motor cortex was actually organized as a coherent, dynamic representation of movements prior to reaching and demonstrated that a covert cognitive process could be observed by characterizing the dynamics of spiking activity in a population of neurons.

Georgopoulos et al.² is an early demonstration that explanations of spiking activity in the neocortex by the overt stimulus and response properties of static tuning curves could be substantially improved by the addition of experimentally covert, dynamic cognitive variables. Although several more recent studies have shown spiking activity that is most readily explained as cell assembly dynamics in the neocortex^3–5, these investigations into cognition have been hampered by the long training times required for primate experiments (which tend to lead to the use of non-cognitive processes⁶). In contrast, many behavioral tasks that model cognition are quite amenable to unit recordings in rodents; often allowing for recordings throughout both acquisition and performance of the task^{7, 8}. The combination of these tasks, the highly selective spiking activity of hippocampal pyramidal cells and the well-established use of chronic recording technologies make the hippocampus a model system for studying the representation of cognitive variables in neural discharge.

Dynamic spatial information in hippocampal ensembles

The location of the animal relative to its environment is the clearest signal in the spiking activity of hippocampal pyramidal cells recorded in behaving rodents, leading to the adoption of the term “place cell” ^7–9. Each pyramidal neuron primarily spikes when the animal is in a particular part of its environment (the cell’s place field), which suggests that the activity across the place cell population creates a map-like representation of the animal’s current position in an environment ^7–9. This idea led to the convention of constructing a spatial tuning curve for individual place cells as a time-averaged firing rate map as a function of position ^{10, 11}. This time-independent firing rate map can be interpreted as a tuning curve of firing to spatial position. The strength of the spatial signal, the remarkable stability of these firing rate maps in constant environmental conditions and their response to well-defined changes of the spatial environment ^{8, 10–13} led to the prevailing view that a given hippocampal pyramidal neuron is either a place cell or not active in a given environment ¹⁴. Momentary spiking activity that deviated from a cell’s tuning curve was taken as noise and ignored by averaging across several minutes of recording.

Changes in place field firing (under, for example, environmental changes^{12, 13}) were identified as changes between multiple maps^15–18. This hypothesis led to the suggestion that these transitory events in which spiking activity deviated from a cell’s tuning curve might reflect self-consistent information about other locations or other maps, presumably as the animal shifted its attention from its current position to other kinds of information. If this were true, then one should be able to detect these covert cognitive shifts between input streams by examining the “noise” for statistical structure even if the underlying parameters of that structure are unknown. Identifying cognitive function then entails finding the cognitive bases for the covert parameters: tying these often-fleeting sets of active neurons that coalesce and disappear to the psychological processes that they presumably underlie, such as memory, decision-making, and attention. We review current progress towards these admittedly ambitious goals in the hippocampus and describe a general computational approach towards finding covert cognitive processes in the firing patterns of neural ensembles.

Dynamics at long timescales

While we are mainly concerned with the dynamics of hippocampal neurons at the millisecond timescale, there is abundant evidence for the dynamic influence of cognitive processes on the firing of hippocampal neurons at longer timescales, notably with regard to the property of place cells called “remapping”^{8, 12,19}. While a place cell can indeed have the same place field for months in a constant environment ²⁰, unlike sensory neurons, place cells can also completely change their spatial tuning curves between experiences within an environment ^{13, 19, 21-23}. While this remapping most commonly happens in response to changes in environmental cues (e.g. putting the animal in a novel environment), remapping is not a straightforward sensory transformation of the changed cues ^{8, 13, 23–30}. Moreover, place cells can remap without any change whatsoever in the available spatial cues, for instance when an animal changes its behavioral strategy³¹, uses different coordinate systems ^32–35 or as information about a context is acquired ^{36, 37}. Often, the remapping can best be thought of as multiple stable states, with different environments or conditions associated with particular sets of place cells that are stably retrieved upon subsequent reintroductions to the appropriate environment, or indeed as the animal switches between tasks or coordinate systems.

What might govern these shifts in the information reflected by hippocampal neurons, and how might they be tied to cognition? Kentros et al.²² found that place fields in mice spontaneously remapped far more often than those of rats under the same simple behavioral conditions (simply chasing after randomly dropped food pellets in a familiar environment). However, mice recorded while accurately performing a spatial task in the same familiar environment had place fields that were as stable as those of rats. Kentros et al. suggested that the key difference was that the animals doing the spatial task had reason to pay attention to the available spatial cues, so they remembered them, which was reflected in the stability of their place fields. The studies described above support the hypothesis that covert cognitive processes such as what the animal is paying attention to influence the long-term dynamics of place fields. However, attention typically operates at much shorter timescales, on the order of seconds to milliseconds. This suggests the possibility of finding task-related shifts in network activity in hippocampal neurons at these timescales as well, as we discuss in the following section.

Dynamics at short timescales

Might the covert variables that control map retrieval upon entering an environment also operate on short timescales to repeatedly switch between multiple maps in constant conditions? If the tuning curve maps were sufficiently different, almost any repetitive map switching would appear as noise in place cell tuning curves³⁸. Indeed, in the simple foraging tasks in which stable place fields are typically observed, a place cell has unexpectedly high levels of spiking variability inside its place field ³⁹. In these tasks, a robust place cell may emit 20 or more action potentials on a single pass through a place field, but fail to emit any action potentials seconds later on a pass that is behaviorally indistinguishable. The statistics of these deviations are incompatible with the hypothesis that place cell activity simply varies randomly about a mean described by a single spatial tuning curve ^{38, 39} and instead support a hypothesis that place cell activity reflects a small number of spatial tuning curves that differ mainly in firing rate and are alternatively switched on and off with a period of about one second ^{38, 40, 41}.

This proposal is analogous to the suggestion that the hippocampus maintains multiple spatial maps of the environment and somehow switches between those maps very quickly. Support for this proposal comes from studies by Harris et al.⁴² and Jackson and Redish⁴¹. Harris et al. initially showed that predicting the spiking activity of hippocampal place cells using both position and the spiking activities of a set of simultaneously recorded place cells was significantly better than predicting hippocampal place cell activity using position information alone. Harris⁴³ argued that the covariation of place cell spiking activity was evidence for use of multiple cell assemblies within the hippocampus. Jackson and Redish⁴¹ showed that coherent fast switching between multiple hippocampal cell assemblies could explain the excess variability observed within place cell spiking activity observed by Fenton and Muller³⁹. Furthermore, Jackson and Redish⁴¹ showed that fast switching between cell assemblies was clearly aligned to specific behavioral phases in certain tasks and produced multiple and distinct tuning curve maps. The cell assemblies observed on the linear track, for instance, were generally aligned with the animal’s running direction and their projection onto spatial position was apparent as directional place fields. Although directional place fields have been previously explained as indicative of multiple reference frames (maps) in linear track tasks^{10, 17, 18, 32, 41, 44}, Jackson and Redish⁴¹ showed that reference frames are not specific to linear track tasks and can explain the excess variability first identified by Fenton and Muller³⁹.

The high levels of place cell spiking variability is unlikely to be noise, because it has an across cell organization that can be explained as coordinated activity^41–43. These observations further suggest that internal, unobservable or covert processes mediate the cell’s active tuning curve and, consequently, determines the discharge of the cell at that moment. Observations that these cell assembly dynamics are modulated by cognitive demands ^{22, 38} and aligned to specific task components ⁴¹ suggest that cell assembly dynamics are better described as a reflection of covert cognitive processes than of noise.

Extra-field spikes during sleep, rest, and directed behavior

While place fields described pyramidal cell firing during awake behavior, these same pyramidal cells fired during specific sleeps states (e.g. during sharp-waves occurring within slow-wave sleep and during REM sleep^{7, 45}). This sleep-related firing was difficult to explain from a traditional tuning-curve perspective. Similarly, place fields fire extra spikes outside their place fields during rest, grooming, eating, and other, non-attentive pausing behaviors ⁷. Subsequent studies showed that these extra-field spikes entail a reactivation of place cell firing sequences during sleep states that is both reliable and coherent. Cell pairs, ensembles, and the temporal order therein that were active during awake behavior are reactivated during subsequent sleep states ^46–52 - hippocampal pyramidal cell spiking activity continues to be organized with respect to the representation of space during sleep. The firing of hippocampal cells during sleep is better described as replay than as noise. Decoding algorithms applied to neural ensembles found that the decoded location during these rest states deviated from the rat’s observed location⁵³, but Jensen and Lisman were not able to identify any structure in these processes and concluded that they were noise. Recent studies, however, have found that these extra-field spikes occurring during rest can be understood as reactivation of recently experienced behaviors ^54–59. These results suggest that the noise identified by Jensen and Lisman⁵³ actually contains structure and reflects a covert (cognitive) event.

Johnson and Redish⁶⁰ examined place cell firing at a decision point. At difficult decision-points rats pause, but remain attentive to their surroundings: they turn back and forth, orienting down potential choices in a process termed vicarious trial-and-error (VTE⁶¹). During these behaviors, the hippocampus remains in an active theta state (similar to running behavior^{7, 60, 62}). Johnson and Redish found that during these behaviors, place cells fired spikes even if the animal was outside of the cell’s place field, at a location where the tuning curve predicted zero spikes⁶⁰. Decoding activity during these behaviors revealed a sequential sweep of positions from the animal’s current position to potential future positions on each arm. The non-local forward representations contained sequential structure, were predominantly ahead of the animal, and were related to the orientation of the animal during the VTE behavior. These data suggest that place cell activity that occurs outside of a cell’s place field signals a covert process related to the prediction of potential spatial positions available to the animal rather that simple noise.

Discussion: Encoding, decoding, and generative approaches

The studies reviewed above suggest a correspondence between covert, cognitive variables and deviation of neural activity from a cell’s tuning curve. The conclusions from these studies result from at least one of three distinct approaches to analysis, encoding, decoding, and generative. Each approach is a distinct way to evaluate the validity of a hypothesized neural code and determine how well a variable can be related to neural activity⁵⁹ (see Box 1.)

Box 1.

Current technology allows the simultaneous recording of large neural ensembles from awake, behaving animals. The simultaneity of the large neural ensembles allows the identification of both decoded behavioral variables (reconstruction) and the prediction of spike trains (generative models) from hypothesized processes.

Encoding

Tuning curves are generated from the correlations of behavioral variables and simultaneously observed spike trains. Behavioral variables can include sensory inputs, motor outputs, or task-related behavioral variables (as shown here).

Decoding

By definition, a tuning curve encodes a description of an observed variable, such as a behavior as a function of spike trains. Through standard methods, it is possible to invert this description to predict the behavioral variable from an observation of spike trains.

Generative models

Because the tuning curve is a description of the spike train as a function of the behavioral variable, it is possible to predict the spike trains from the observed behavior and/or the value of a hypothesized or decoded covert variable.

Comparisons

The decoded behavioral variable can be compared with the actual behavioral variable. These differences (and similarities) can be examined for structure which provides evidence for cognitive processes. Similarly, the predicted spike trains can be compared with the actual spike trains for differences and similarities which can be examined for underlying structure which provides evidence for cognitive processes.

Figure B.1. Example of the encoding/decoding/generative models cycle.

Encoding– Tuning curves are generated from behavior variables (here the location of the rat during foraging within a 1m cylinder) and from ensemble spike trains (here showing diagrammatic action potentials from nine of 93 cells; spikes from each cell are indicated by a different color). Decoding–Combining tuning curves and observed spike trains produces decoded variables. This example shows a Bayesian probability distribution of the location of the rat given the tuning curves and spike trains at a given time. White star indicates the observed location of the animal. Generative Models– Combining decoded behavioral variables with tuning curves produces predicted spike trains.

Encoding approaches

If a cell reliably changes its firing as a function of different stimulus conditions or differences in a behavioral parameter, then one can say that the cell is tuned to the parameter in question⁶³. The tuning curve thus describes how activity represents information about the parameter⁶⁴. Within the encoding approach, validity of a neural code is based on how much information the neural code provides about the variable in question^63–65. The amount of information can be measured using either Shannon information⁶⁴ or Fisher information⁶³. However, information measures depend on the concept of a signal separated from noise, and they all effectively measure the extent to which the signal can be differentiated from an unstructured, unexplainable noise component. Identifying covert, cognitive processes using encoding approaches depends on making covert processes overt by aligning cognitive events to behavioral tasks so that they always occur at the same time on each trial or by creating tasks that differ only in the hypothesized covert processes. The encoding approach has enjoyed some success within tightly controlled behavioral experiments on attention²², categorization⁶⁶, and various other aspects of decision-making⁶⁷, including complex higher-order transformations of complex variables^{68, 69}. However, encoding approaches offer only limited and indirect methods for assessing the dynamic organization of neural populations that, according to cell assembly concepts, support cognitive function by coordinated changes across many cells within a single trial.

Decoding approaches

Decoding compares a sample of neural activity to an established tuning curve in an attempt to predict the value of the encoded behavioral, environmental or cognitive parameter^{53, 64, 70, 71}. Decoded results that deviate from the observation are interpreted as errors ^{53, 70–72}. Average errors are used to distinguish between multiple decoding algorithms^70–72, but individual decoding errors are typically thought to indicate noise or the inadequacy of the neural code used for decoding.

Analysis of covert, cognitive processes present a distinct problem for the decoding approach to neural codes because the cognitive variables implied by these processes are not generally experimentally observable. In decoding approaches, these cognitive processes will likely appear as decoding errors. To avoid the problem of calling these deviations ‘errors’, decoding approaches to covert, cognitive processes have either highlighted unexpected structure within the distribution of decoding errors or compared the time series of predicted (decoded) cognitive variables with a hypothetical time series that is derived from subjective expectations about the cognitive processing. The application of the decoding approach to cognitive processing has been successful in a number of studies on population vector rotation in motor cortex^{2, 4} and route replay in hippocampus ^{49, 53, 54, 57–59}. However, the decoding approach to cognitive processing provides a weak form of statistical testing because it depends entirely on how the hypothesized cognitive time series has been defined. As a result, decoding approaches to covert cognitive processes are usually reducible to an extended cross correlation analysis that is informed by each cell’s tuning curve, and this requires the potentially dubious assumption that the covert cognitive variable is constant across many trials^{2, 4, 48, 49, 51, 55, 73}.

Generative approaches

The studies of within session multi-stability and extra-field spiking used what we call the generative approach (see Box 1). The generative approach exploits the fact that a tuning curve is not merely descriptive; it can also be used as a basis for generating a prediction of neural activity from a given set of behavioral, environmental or cognitive parameters^{39, 41, 42, 59, 64}. The generative approach enables direct comparisons between the predictions that a spike should occur and actual observations. The generative approach can be used in flexible tasks that are more likely to involve cognition. As a result, the generative approach provides a framework for detecting and quantifying the multi-neuron cell assembly dynamics that Hebb (1949) originally proposed.

Support for the validity of a proposed neural code within the generative approach is determined by how well an observed set of neural activity can be predicted from behavioral, environmental or cognitive variables^{41, 42, 59, 64}. At a superficial level, the application of the generative approach to covert, cognitive processes faces the same problem encountered within decoding approaches (that a hypothetical time series of a cognitive variable must be proposed for evaluating validity). However, several applications of the generative approach have circumvented this problem by inferring the time series of a covert variable’s value on the basis of neural ensemble activity ^{41, 42, 59}. Instead of relying on assumed psychological processes, these variants of the generative approach either implicitly⁴² or explicitly^{41, 59} used decoding algorithms to infer the time series of the covert variable. These studies showed that apparent noise at the single unit level was actually self-consistent within the set of simultaneously recorded neurons. The ensemble activity was coordinated even when it deviated from what the experimenter expected given the animal’s position in the maze and presumed spatial map; these experimentally unexpected deviations were consistent with switching between spatial maps⁴¹, route replay⁵⁹ and other non-local spatial representations⁶⁰. The success of the generative approach in identification of covert processes illustrates the power of basing validity tests of the neural code on a biological signal (ensemble activity) instead of relying on more indirect expectations that arbitrarily delimit cognition.

Conclusion

Cognitive processes fundamentally and often subtly change how information is represented in the brain. As a result, understanding the neural bases of cognition requires one to distinguish between noise and covert, internally generated changes in neuronal firing patterns. The generative approach provides a distinct advantage over approaches that stop at encoding or decoding because generative approaches can quantitatively differentiate between random noise and structured variability that is by definition not noise, but cannot be accommodated by the tuning curve of the neuron. As techniques for recording large neural ensembles become a more standard tool for investigating the neural basis of cognition across the brain, the theoretical problem of how to analyze neural data under conditions where constancy of the cognitive variable cannot be safely assumed becomes critically important. Analysis of this ”excess” variability provides a point of departure for understanding how attention, memory consolidation, and decision-making contribute to and refine traditional ideas of spatial representations present in hippocampal activity. Further technological advances in the quality and density of recording neural ensembles will offer unprecedented opportunities to understand the dynamics of neural population activity with the promise of defining many of the largely covert, subjective processes we call cognition in terms of objective biological mechanisms.

Fig. 1. Decoded direction from a population of motor cortex neurons shows mental rotation.

Motor cortex neurons were recorded during direct reaches to a target or during reaches to a location offset by 90 degrees. In standard trials, decoding yielded a constant direction. In contrast, decoding during reaches 90 degrees counterclockwise from the target produced a decoded direction of reaching that rotated systematically from the trained direction to the rotated target. This covert, mental rotation of the planned movement could only be observed through the decoding process. The population in this study were recorded sequentially, not simultaneously, which makes the assumption of a constant cognitive representational dynamic from trial to trial. Modern technology enabling the simultaneous recording of large neural ensembles has alleviated this constraint.

Fig. 2. Remapping depends on covert, cognitive variables.

(A) Observations of place field distributions between two experiences on a task show remapping under some conditions but not others. (B) These effects are significantly modulated by cognitive variables, such as attention to the presence of a spatial task. (C) Diagram of the identification of structure in the place cell remapping. Tuning curves (green) are generated from observations of spiking data and behavioral variables (orange). The likelihood of remapping is dependent on covert processes of attention, task, and behavioral relevance.

Fig. 3. During simple tasks, there is noise in place fields that can be explained as changes in covert, cognitive variables.

(A) The noise in place field firing differs between attentive and goal-related conditions. In general foraging tasks, there is a high variability. In tasks with a goal, the variability decreases. Taking only firing during goal-approach produces variability consistent with the predicted tuning curves. (B) Using generative models, it is possible to split place cell firing rate maps apart using hypothesized covert variables. [S = split by covert variable; R = random shuffled control]. (C) Diagram of the identification of structure as covert variables. Cell assembly dynamics suggest the presence of covert variables (purple). Tuning curves (green) are generated by combining observations of spiking data and behavioral variables (orange) with those hypothesized covert variables (purple). The hypothesized covert variables explain large portions of the residual “noise” (blue).

Fig 4. Replay after behavior is an example of a covert, cognitive event identified by clear structure within extraneous spiking.

(A) Direct examination of cell firing shows that cells fire in the same sequence during post-behavior sleep [bottom] as the place fields during behavior [top]. (B) Direct observation of firing during sharp-waves during awake states shows sequential replay akin to that seen during sleep. (C) Diagram of the identification of structure in the noise. Tuning curves (green) are generated from observations of spiking data and behavioral variables (orange, top). When compared to observations of spiking data from sleep states (orange, bottom), these tuning curves can be used to decode represented position (blue). Structure in the decoding leads to the conclusion that there are covert variables (replay). (D) Diagram of the identification of structure in the noise during awake states. Tuning curves (green) are generated from observations of spiking data and behavioral variables (orange, top). During sharp-waves, there is a mismatch between the spiking predicted by the tuning curves and the observed spiking. When decoding represented position (blue), structure in the decoding leads to the conclusion that there are covert variables (replay).

Fig 5. Extraneous spiking during awake behaviors can be explained by covert, cognitive “planning-like” events.

(A) Decoding extra-field spikes at the choice point shows non-local sweeps of locations ahead of the animal down potential paths. (B) These decoded variables are significantly different at the choice point relative to a matched-time approach to that choice point. (C) Diagram of the identification of structure in the noise during awake states. Tuning curves (green) are generated from observations of spiking data and behavioral variables (orange, top). At the choice-point, there is a mismatch between the spiking predicted by the tuning curves and the observed spiking. When decoding represented position (blue), structure in the decoding leads to the conclusion that there are covert variables (forward sweeps).