Hippocampal sharp-wave ripples in cognitive map maintenance versus episodic simulation

Abstract

Hippocampal sharp-wave ripples (SWRs) have been proposed to support memory-based decision-making. A new study by Gillespie et al. (in this issue of Neuron) provides important new insights on how past experiences and future choices are reflected in neuronal activity during SWRs.

Memories can be among our most prized possessions, providing detailed tapestries of people, places, and sequences of events from the past that we can retrieve and relive—best with old friends, but also alone. But memories do not just tell us about the past—they also deeply affect our future. How we interpret the past guides our major decisions, such as who to marry and where to live, and our minor ones, like which road to work to take during traffic and which dishes to order at a local restaurant. How do we use memories to navigate the decision landscapes we regularly encounter? Our deliberation can be facilitated by mental simulation of different scenarios featuring sequences of possible events (“episodes”) and potential outcomes in a process that cognitive neuroscientists refer to as “episodic simulation” (Schacter et al., 2008). During this process, networks of neurons in our brains are believed to simulate episodes based on the options available to us using a combination of imagination and memory from past experiences to help decide on the most favorable outcome. Such episodic simulations can potentially take place outside of conscious awareness, nevertheless guiding our choices.

Much remains unknown about the neuronal mechanisms that enable memory-based simulations and the extent to which they overlap with the mechanisms of memory retrieval. However, a particularly promising candidate mechanism for episodic simulation appears to be provided by sharp-wave ripples (SWRs) that originate in the CA3 subregion of the hippocampus and are observed when animals are quiescent or asleep. CA3 neurons are highly inter-connected, so that when they are seeded under appropriate conditions, they can generate and amplify bursts of activity that resonate throughout the hippocampal formation and spread to regions throughout the brain over a fraction of a second. Strong evidence suggests that these activity bursts play an important role in the formation and long-term storage of complex spatial memories, but their potential contribution to episodic simulation is only recently being examined. There are several features of neuronal sequences during SWRs that make them potentially suitable for directing hippocampus-generated episodic simulations. First, the recurrent inter-connections between CA3 neurons are ideally suited for the storage and retrieval of learned episodic associations that would need to be accessed to generate realistic episodic simulations. Second, as SWRs powerfully activate neurons throughout the brain, these events can potentially direct hippocampus-generated simulations to other brain structures in a coordinated manner to support decision-making. Third, SWRs are seen during pauses in active behavior, such as during deliberation, often exactly at the junctions where decisions between different options need to be made.

About 14 years ago, when investigating the firing order of neurons in CA3 and its output hippocampal region CA1 during SWRs in rodents running on linear tracks, Diba and Buzsáki (2007) observed that the neurons fired rapidly in the same sequence as they typically did on the upcoming run. As these animals had previously run down the same track, we conjectured at the time that such prospective neuronal firing sequences during SWRs could represent the neuronal basis for planning future trajectories in the sense of episodic simulation. This idea received subsequent support from multiple studies in rodents. These studies typically measure spiking activities from populations of “place cells” (neurons that fire in the vicinity of specific locations, collectively forming a “cognitive map” of space [O’Keefe and Nadel, 1978]) recorded extracellularly from the CA1 and/or CA3 regions of the hippocampus in rodents running in mazes for rewards. Using the place-selective activities of neurons during active behavior, researchers can then examine whether the spiking activities of these neurons during SWRs can be decoded to locations in the past or potential futures of the animal. Using such an approach, Singer et al. (2013) found that neuronal activation during SWRs was biased toward outbound trajectories that the animal was choosing between, rather than paths the animal had recently completed. Furthermore, this activation was greater on trials when the animals chose correctly to obtain rewards. In another notable study, Pfeiffer and Foster (2013) recorded from rats during goal-directed behavior in an open-field environment in which their trajectories were minimally constrained. These authors demonstrated that spiking patterns during SWRs traced trajectories through the open field that were not replays of the animal’s exact behavior but instead provided possible paths, consistent with episodic simulation. Furthermore, by comparing the decoded paths during SWRs with those of the animals just prior to and immediately after the SWRs, the authors found that the 2D paths during SWRs were closer to the animal’s subsequent choices than to the paths prior to the choice, thus indicating that episodic simulations during SWRs could play an important role in planning.

However, other research has painted a more complex picture. Multiple studies have demonstrated that SWRs play a key role in the formation and consolidation of memories and frequently “replay” past experience; this makes it unclear whether and how SWRs could both support memory storage and decision-making. Other studies report SWR patterns that don’t fit squarely with either scenario. Gupta et al. (2010) found that when animals ran down only one loop of a figure eight maze, SWRs frequently simulated trajectories down the opposite loop, indicating that SWRs do not simply replay recent experience or predict upcoming choices. In another elegant study, Carey et. al (2019) gave animals a choice between two maze arms that contained food and water rewards, respectively, and analyzed decoded trajectories during SWRs before and during behavior. They adjusted the animals’ diets to increase motivation for food or water on alternate days. While it might be expected from either the memory storage or the episodic simulation hypotheses that SWR trajectories would match the animals’ choices, these authors found that the majority of SWR trajectories were instead directed toward sites not visited that day; SWRs reflected the paths to water rewards when animals went for the food and the paths to food rewards when animals were thirstier. Overall, these findings are not readily reconcilable with the hypotheses that SWRs simulate the paths animals intend to take, nor that they simply replay recent experience on a trial-by-trial basis.

In this edition of Neuron, a new study by Gillespie et al. (2021) sheds significant light on these questions and provides important insights on the trajectories observed during SWRs. In this study, rats were given water rewards at multiple points: for maintaining a nose-poke in one of two ports in a holding station, for correctly choosing the rewarded arm among eight arms of the maze, and for returning to the start segment of the task. The rewarded arm was changed for each block of trials. In each block, the animals first identified the rewarded maze arm by sampling the different maze arms during a “search” phase, and then began to repeatedly visit that arm during a “repeat” phase to amass reward. Animals received extensive training on this task. The task structure allowed for a comparison of SWR trajectories between eight discrete maze arm choices all initiated from the same holding station. Gillespie et al. found that SWRs in the holding station did not typically represent locations on the arms that the rats subsequently chose, neither in the search phase nor in the repeat phase of the task, when the animals reliably visited the same maze arm. Instead, SWR representations were highly variable, but the most consistently simulated locations were from the arm that was rewarded in the previous block of trials. Most other SWRs simulated the maze arms that animals actually chose not to visit. Interestingly, while a generalized linear modeling analysis revealed a significant, albeit weak, enhanced simulation of the prospective choice in most animals, a more consistent effect was that the last visited arm was discounted and less frequently replayed.

There are several important implications of these observations. Based on these and earlier studies, it is now clear that SWR trajectories comprise many of the potential options available to the animal, not only the path to be chosen or the path just completed. This might indicate that waking SWRs are not involved in episodic simulation after all and that their primary function is the maintenance of a balanced cognitive map, an idea first proposed by Gupta et al. (2010). This cognitive map is assumed to be established in the hippocampus during early exploration of an environment and learning of task and reward contingencies but would undergo refinement as those contingencies change. The cognitive map maintenance hypothesis can also be considered a type of memory storage, as suggested by Gillespie et al. (2021), as it is based largely on past experiences. In support of this conjecture, studies that either enhance or disrupt neuronal firing patterns during SWRs only affect the rate of learning in working memory tasks—they don’t alter performance on a trial-by-trial basis. This hypothesis can also explain why non-visited locations are replayed during SWRs in this study and others; perhaps activation of a subset of place cells from a map during behavior necessitates reconsolidation of the remaining map components, triggering their reactivation during SWRs. However, it’s not readily evident how this view can be reconciled with significant prospective SWR trajectories reported in previous studies (e.g., Xu et al., 2019) as well as this one, or the preferential replay of a previous goal location well after it is no longer rewarded (Gillespie et al., 2021). A purely memory-storage function for SWRs would also preclude their role in memory retrieval, which appears to contradict emerging evidence from both human and animal studies.

Alternately, some of these observations can still be reconciled with the episodic simulation hypothesis. SWRs could promote consideration of alternatives besides the most compelling choice, to better support flexibility in decisions. A remarkable finding by Gillespie et al. is that SWR simulations of the most recently visited arm were suppressed at the choice point, as though the recent past might be of lesser priority compared to options not recently sampled. Notably, this recently visited arm could have been either correct and rewarded or incorrect and not rewarded: when it was incorrect, few SWRs were elicited on that maze arm, and those locations were also discounted for subsequent replay, but when the visited arm was correct, robust SWRs at the reward site replayed this location. Thus, the subsequent suppression of rewarded arms during SWRs at the choice point could reflect the fact that these maze locations were recently activated in SWRs that took place on the arms themselves, obviating the need to simulate them yet again. Consistent with this view, Igata et. al (2021) recently reported SWR trajectories that preceded and predicted animals’ behavior, but these were observed near the reward locations rather than at the choice point of the maze. Additionally, SWR simulations may reflect hippocampal biases based on memories of the past, which can vary across individuals and shift over time, perhaps to enhance alternatives that are remote and/or not currently desirable (e.g., food location when animals prefer water and vice versa [Carey et al., 2019], and even locations that animals are actively trying to avoid). Other brain regions involved in flexible planning, such as the prefrontal cortex, that receive the hippocampal signal can overrule these biases and choose a different path. In this view, however, episodic simulations generated by the hippocampus can remain useful by positing worthy options, even in the absence of a trial-by-trial correspondence to choices taken.

Nevertheless, it is important to note that these ideas are not mutually exclusive, as the maintenance of a cognitive map is critical for episodic simulation, and episodic simulation and rehearsal may in turn help stabilize the cognitive map (Gupta et al., 2010). Overall, Gillespie et al. provide valuable information regarding how maze locations are prioritized for either simulation or replay during SWRs. These findings suggest refinement of the hypotheses surrounding SWRs, whose resolution will be important for our understanding of how memories influence cognition and decision-making.