Decoding the cognitive map: ensemble hippocampal sequences and decision making

Abstract

Tolman proposed that complex animal behavior is mediated by the cognitive map, an integrative learning system that allows animals to reconfigure previous experience in order to compute predictions about the future. The discovery of place cells in the rodent hippocampus immediately suggested a plausible neural mechanism to fulfill the “map” component of Tolman’s theory. Recent work examining hippocampal representations occurring at fast time scales suggests that these sequences might be important for supporting the inferential mental operations associated with cognitive map function. New findings that hippocampal sequences play an important causal role in mediating adaptive behavior on a moment-by-moment basis suggest specific neural processes that may underlie Tolman’s cognitive map framework.

Introduction

A long-standing conjecture in the study of animal learning, memory, and decision making is that adaptive behavior is supported by a cognitive map, a flexible, generative learning system that allows animals to reconfigure previous experience to make inferences about the future and plan forthcoming behavior [1–3]. While the spatially-tuned firing patterns of hippocampal pyramidal neurons (place cells) provide an intuitive neural substrate for the mapping component of the cognitive map [4–6], understanding how hippocampal computations relate to the more cognitive aspects of Tolman’s construct has proven challenging. Work in human and non-human species suggests that the hippocampus underlies both retrospective, mnemonic processes as well as prospective, future-oriented mental abilities [7–11], raising an interesting question: how can place cell representations, which undoubtedly form a reliable representation of the animal’s actual location in an environment (e.g. [12]), also support the more complicated cognitive processes associated with the cognitive map? Recent work suggests that the answer lies in the sequential activation patterns of hippocampal pyramidal neurons.

Hippocampal network states

The information represented within hippocampal sequences differs as a function of the hippocampal network state. The computations performed by hippocampus change in the presence and absence of neuromodulators [13–15] and state-input processes [4]. These network states are reflected in patterns of local field potential (LFP) oscillations, which are often divided into theta and non-theta states [4, 16–19]. The theta state accompanies active behavior or attentive processes, during which the LFP exhibits prominent oscillations in the theta frequency band (6–12 Hz). Theta oscillations organize the spiking of place cells. Within each theta cycle, place cells fire in a sequential order: cells with place fields behind the animal fire first and cells with place fields farther ahead of the animal fire later. Consequently, over the course of the theta cycle, place cells trace out an ensemble representation of spatial trajectories near the animal. During slow wave sleep and awake quiescence (e.g. grooming, food consumption), the hippocampal LFP is less orderly; instead of regular oscillations, broad band voltage fluctuations typify the large, irregular activity (LIA) state. In LIA, place cells activate in fast sequences during sharp-wave ripple (SWR) complexes, so named for the characteristic high-frequency ripple waveforms that punctuate the otherwise irregular LFP. Place cell spiking during SWRs does not necessarily represent the animal’s current location in space. Instead, ensemble firing sequences trace out trajectories that may traverse regions of space the animal does not currently occupy [20, 21].

Ensemble sequence representations

Hippocampal sequences were first identified through changes in the firing of place cells relative to the theta rhythm (fig. 1a). When running through a place field, place cells were observed to precess through phases of theta, so that on entry, spikes occurred late in the cycle, but, on exit, they occurred early in the cycle [22, 23]. It was immediately recognized that this phase precession implied a sequential firing representation along the path of the animal [23, 24]; it has since become clear that the phase precession is a consequence of sequences changing as an animal progresses through a task [25, 26]. Although hippocampal sequences were originally characterized by assessing pairwise correlations in spiking of place cells [27, 28] (fig. 1b), sequences are fundamentally an ensemble phenomenon, and the recent advances in our ability to observe, measure, and detect sequences derive from the ability to record many place cells simultaneously and look for higher-order structure in their firing patterns (fig. 1, c&d). The ensemble approach has several advantages over analyses predicated on single cells or pairs of neurons.

Figure 1. Methods of sequence extraction.

A variety of analytical approaches have been developed to identify and quantify temporally-structured spiking in hippocampal ensembles, ranging from techniques that examine one or a few neurons (a&b) to approaches that leverage ensemble activity (c&d). (a) Early work [22, 23] identified that hippocampal place cell spiking phase precesses against the theta cycle. This phase precession implies the existence of orderly, temporal patterns within theta cycles; however, certain types of sequences might not result in phase precession [26]. (b) Other early work (e.g. [27]) noted that correlations that were established between place cell spiking during exploration persisted during subsequent sleep, suggesting that place cell firing patterns recurred during offline states. (c) Sequence scoring methods like the one developed by Lee and Wilson [76] or Gupta and colleagues [47, 58] quantify the temporal structure of neural activity in a given time window. Pairs of spikes with an order of activation matching the order in which the animal passes through their fields during behavior are scored +1, while pairs of spikes whose activation order is opposite their ordering in space are scored -1. Summing across all spike pairs results in a metric of both the direction of the sequence (sign of the sequence score) and the temporal precision of the sequence (magnitude of the sequence score). The activity in the right panel exhibits temporal structure characteristic of theta sequences; the ordering of many spike pairs is consistent with a forward-directed representation. Unpatterned ensemble spiking (right panel) is consistent with both forward- and backward-ordered representations equally often, resulting in a net sequence score near zero. (d) Bayesian decoding methods [77] use the tuning curves of hippocampal neurons measured during active behavior (i.e. place fields) to estimate which positions in space are most likely being represented by ensemble spiking activity. Decoding hippocampal sequence representations reveals the spatial trajectories that sequence events encode.

First, ensemble analyses offer the ability to interpret neural activity on single trials. Instead of recording the activity of one neuron over many instances of the same behavior and averaging its response, ensemble analyses allow for meaningful interpretation of neural activity on single trials and at fast time scales, both of which are advantageous for understanding cognition, as mental operations are often transient, can unfold quickly, and are likely to vary on a trial-by-trial basis [29]. Single trial analyses avoid uncertainty over how multiple events should be averaged or aligned, as cognitive processes are unlikely to unfold consistently over a fixed duration and may not be reliably timed relative to either behavior or external features of the environment.

Ensemble approaches also offer a means of explicitly testing models of how representations are encoded by neural systems [30, 31]. With a defined tuning model, it is possible to test how well representations like hippocampal sequences conform to the expected patterns of activity predicted from those observed tuning functions. Thus, ensemble approaches have allowed researchers to assess how hippocampal representations of space mentally simulate trajectories that represent previous behaviors [32–34], possible future actions [33, 35, 36**], and even imagined experiences, such as paths that animals have never actually traversed [37, 38*] or within environments they do not currently occupy [39, 40].

LIA sequences for memory, planning, and inference

LIA sequences were first studied during sleep, and quickly rose to prominence as a potential mechanism of memory consolidation [41–43]. Consistent with a role for LIA sequences occurring during sleep in memory consolidation, disrupting SWR-associated sequences with electrical stimulation following performance of memory-guided behaviors impairs task learning [44–46]. Recent work examining LIA sequences during awake states, however, hints at a broader functional role for these representations.

Gupta and colleagues [47] recorded LIA sequences as rats performed a multiple-T decision making task, and observed that the content of representations did not necessarily favor paths leading to reward. In fact, in some cases the authors observed an over-representation of paths traversing the non-rewarded arm of the T-maze, a finding that is difficult to reconcile with a consolidation function, as the represented trajectories did not match the animals’ cumulative behavioral experience. The authors further observed that in some cases, LIA sequences synthesized de novo paths never traversed by the animal. These constructed trajectories plotted “shortcut” paths between reward sites (fig. 2a). In a similar vein, recent reports of “pre-play”, LIA sequences representing regions of space that animals could view, but not directly experience, further suggest that LIA sequences actively synthesize spatial representations, rather than simply consolidating actual experience [37, 38*]. In light of neuropsychological and functional imaging evidence suggesting that the human hippocampus is involved in similar synthetic, generative processes [48–51], these data suggest that the rodent hippocampus might manipulate spatial representations during LIA sequences to underpin similar mental operations.

Figure 2. Diverse functional roles for LIA sequences.

(a) Gupta and colleagues [47] showed that sequence representations of paths actually traversed by rats (e.g. left and right loops on the T-shaped decision maze) were recombined in a novel fashion to represent shortcut paths between potential food delivery sites. (b) LIA sequences expressed prior to initiation of a new behavior could represent the set of possible actions available to the animal in a given situation. Sequence representations simulating potential future actions (in this case left or right turns at the choice point) could be evaluated by other neural structures to guide behavior. Figure constructed after Singer and colleagues [52]. (c) Pfeiffer and Foster [36] found that as rats performed a goal-directed navigation task in a large, open area, LIA sequences depicted paths between the rat’s current location and its eventual spatial goal.

Two recent studies establish a tight correlation between awake LIA sequences and rats’ immediately forthcoming behavior. Singer and colleagues [52**] recorded awake LIA sequences from rats performing a memory-guided W-maze task. The authors found that the degree of ensemble coordination within sequence representations predicted the successful completion of the next task trial, suggesting that LIA sequences facilitate accurate task performance. Interestingly, the content of representations (i.e. which parts of the maze the sequences represented) did not differ between sequences that occurred before correct and incorrect trials. Instead, it appears that the quality of sequences, regardless of where they represented, was the principle determinant of successful choice. This finding favors a model in which sequences represent the space of potential actions available to the animal (fig. 2b), and are then evaluated by downstream reward processing structures [53, 54].

In work that offers an interesting contrast, Pfeiffer and Foster [36**] observed that awake LIA sequence content clearly reflected rats’ future behavior as they performed a goal directed decision making task. Prior to navigating to a goal location in a large, two-dimensional environment, sequences represented paths that began near the animal and ended in the region of space the rat would next visit (fig. 2c). An interesting possibility is that the nature of sequence expression may depend on the precise demands of the behavioral task. In Pfeiffer and Foster’s [36**] task, representations of paths leading to non-rewarded locations would have less utility than in Singer et al.’s [52**] binary choice task, where knowing with certainty either the correct or incorrect choice would suffice for accurate decisions. Future work comparing SWR representations in animals trained to perform both binary and multi-alternative decision making tasks could elucidate how awake LIA sequences vary to support different behavioral challenges.

In elegant work suggesting a causal role for LIA sequences in decision making, Jadhav and colleagues [55**] showed that electrical disruption of awake SWRs impaired rats’ decision making performance on a memory-guided behavioral task. This manipulation affected behavior during the initial acquisition of the hippocampus-dependent task, and also degraded choice accuracy in animals pre-trained to asymptotic performance, implicating awake LIA sequences in both decision making during early learning phases and the long-term maintenance of stable behavioral performance.

The different effects of SWR disruption during awake states [55**] and during sleep states [44–46] suggests that there are functional differences between these SWR components. Differences have also been found in representations during wake and sleep states [56], with wake states showing flexible sequences [32, 34, 47] more likely to be involved in some sort of analytical processing than consolidation, but with sleep states being more consistent with a consolidation hypothesis.

Theta sequences and online planning

Theta sequences have historically been the focus of less research than LIA sequences, as it was originally thought that they arose passively from the independent phase precession of individual place cells. Recent findings, however, challenge that notion [25, 26]. For example, spike time correlations between place cells are more precise than the correlation between the animal’s position and the theta phase of spikes that results from phase precession [28], and theta sequences are patterned with greater precision than would be expected if phase precession alone structured place cell spiking [57]. Furthermore, phase precession depends on motion, while theta sequences do not [26].

These data suggest that sequences are the primary organizing structure of spikes within theta cycles, and that theta sequence representations are coordinated at an ensemble level and imbued with richer structure than previously appreciated [25]. Gupta and colleagues [58**] examined theta sequences on a cycle-by-cycle basis, and found that sequences within theta cycles actively parsed the environment in a way that could not result from phase precession alone. This processing by theta sequences resulted in a cognitively “chunked” representation of space (fig. 3), effectively achieving a sort of information compression that might be useful behaviorally.

Figure 3. Spatial chunking by theta sequences.

Gupta and colleagues [58] found that the content of theta sequences varies around salient landmarks in the environment in a manner that gives rise to a partitioned representation of space. On rounding the first maze corner, sequences are forward-shifted, preferentially representing space in front of the animal. Between the two turns, sequences tend to be centered around the animal’s actual location. As the rat draws near to the second maze corner, sequences lag behind the animal, extending asymmetrically backward. Once past the final corner, representations are again forward-shifted.

Work by Johnson and Redish [35*] showed that theta-state spiking might contribute to decision making. As rats paused at the choice point of a multiple-T decision making task, hippocampal ensembles traced out forward-directed paths corresponding to possible future actions, suggesting a mechanism for deliberation between concurrently-available choices. These forward-directed paths remained structured within theta cycles, suggesting that they were a similar form of theta sequences. Interestingly, much like the Singer and Frank [52**] LIA result reviewed above, Johnson and Redish [35*] were unable to find a relationship between the sequences and the actual decision made, suggesting that the hippocampus may be playing a similar constructive role, identifying the space of possibilities. Johnson and Redish [35*] did find that these extended theta sequences became stereotyped and then vanished as behavior became more automated. Like Singer and Frank’s [52**] W-task, Johnson and Redish’s [35*] T-task was also a binary choice. It remains unknown whether these results would change in the light of a more open task such as used by Pfeiffer and Foster [36**].

Do theta sequence representations play a causal role in decision making that extends beyond the more general tuning properties of place cells? Cannabinoid agonists offer an intriguing manipulation for dissociating the influence of ensemble spiking sequences and other place cell properties on decision making by abolishing ensemble coordination of place cell spiking while minimally affecting cells’ spatial tuning or firing rates. Robbe and colleagues [59, 60] found that administration of cannabinoid agonists disrupted accurate decision making in rats performing a memory-guided spatial task. Importantly, cannabinoid agonism caused task performance to fall to chance levels even in well trained animals, suggesting that temporal coordination within theta cycles likely plays a role in the moment-to-moment selection of behavior. Together, these data suggest that theta sequences might be an important brain mechanism for deliberative decision making.

Conclusions and future directions

It is increasingly apparent that sequences play a more active and complex role in information processing than encoding veridical experience. Their role in flexibly manipulating and permuting representations of space to generate novel paths that might aid action selection meshes well with the cognitive map envisioned by Tolman [2, 3]; however, important questions about the function of hippocampal sequences remain unanswered.

Foremost, the interaction between LIA representations and sequences during the theta network state remains largely untested. Although there is good evidence that both are causally involved in planning and decision making [55**, 59, 60], it is unclear precisely which cognitive functions each are responsible for, whether they are interdependent, or whether the absence of one type of sequence might induce compensatory changes in the expression of the other. Future work that manipulates theta and LIA sequences independently could help reveal the relationship between hippocampal sequential representations that occur during different network states.

We know little of how hippocampal sequence representations affect other brain regions, although several lines of evidence hint at interplay between the hippocampus and extra-hippocampal structures during both theta [61–66] and non-theta [67–72] states. Simultaneously measuring neural activity in multiple brain regions as animals are engaged in complex behaviors will further our understanding of how the hippocampus and other brain regions interact to support cognition.

Finally, it is unclear whether the mnemonic and planning functions of LIA sequences are separable. Interestingly, causal manipulations clearly implicate sleep [44, 45] and awake [55**] LIA representations in both memory consolidation and decision making, respectively. One possibility is that the animal’s behavioral state (asleep vs. awake) determines which function prevails. The reduced sensory input associated with sleep states might allow internal hippocampal dynamics to dominate information processing, favoring memory consolidation processes, while waking states and their accompanying stream of information about the external world might shift hippocampal processing to memory recall or planning functions [73]. This arrangement would suggest, however, that consolidation of learning could not take place online, in waking states, as experience occurs. If awake consolidation is indeed possible, sequences dedicated to mnemonic and planning processes might be distinguishable, either by the content of representations or by some other neural signal, such as the LFP, components of which have been shown to vary with the quality of sequence representations [74], or even by the activity of other brain structures (such as pre-limbic cortex, [75]). Selective interruption of different functional classes of LIA sequences could lead to more precise understanding of how hippocampal sequences contribute to cognitive processes.

Highlights.

• Sequences of spiking are the dominant organization principle of hippocampal activity.

• New ensemble techniques allow observation and detection of hippocampal sequences.

• Sharp-wave sequences are involved in both consolidation and choosing future behavior.

• Theta sequences support online planning to guide behavior in real time.