Value representation in the monkey hippocampus

Abstract

The hippocampus is thought to form cognitive maps across different domains of experience, including space and time. A recent paper by Knudsen and Wallis [1] identifies a map of abstract value space in the monkey hippocampus We consider how these abstract variables might contribute to a comprehensive hippocampal representation of ongoing experience.

The hippocampus contributes to our ability to form memories of our experiences [2]. However, it has also been proposed that a primary role of the hippocampus is to act as a cognitive map of space [3]. In recent years, the mapping abilities of the hippocampus have been shown to represent a wide variety of non-spatial content, including elapsed time [4] and sound frequency [5], consistent with the idea that the hippocampus forms a “memory space” [6] in which all modalities of experiences are linked within a relational network. In a recent article, Knudsen and Wallis [1] extend this evidence by showing that primate hippocampal neurons are tuned to different positions in an abstract value space.

To study such abstract representations, Knudsen and Wallis used a reward-based learning task [7], which allowed them to formalize and quantify how hippocampal neurons encode value relationships. In this task, monkeys learned the relative probability of reward of three pictures. Reward contingencies changed over time, and monkeys learned to track picture values, choosing the highest rewarded picture most of the time. Importantly, when the value of one picture changed, the value of the other two other pictures changed in a correlated manner. These correlated changes can thus be envisioned as paths through an abstract value landscape, just like paths one might take through a physical space.

Hippocampal neurons were selectively active in distinct regions of the value space, representing the value relational code as place fields within this space. These value place cells share four key properties with canonical rodent spatial place cells. First, each neuron fired selectively in one region of the value space, collectively mapping the extent of the experienced value space. Second, like spatial place cells in rodents navigating linear tracks, value place cells yielded different activation profiles when the same location was experienced in opposite directions. Third, over time, value place cells shifted their firing location to earlier in the trajectory, reflecting an effect of experience. Finally, the firing location of some of these cells changed when different pictures were used for the same task, i.e.,they remapped with a change in context.

These data suggest that the hippocampus constructs a map of an abstract value space, potentially relaying the relative value of outcomes to other areas involved in reward learning, such as the orbitofrontal cortex [7].

Ongoing experience and flexible use of cognitive maps

Value place cells may serve as part of a universal positional code that captures value relationships between stimuli. Alternatively, the hippocampus may encode value relationships because of their intrinsic temporal contiguity in the behavioral task [8]. In this framework, neurons in the hippocampus respond to value in the same way as they respond to any other variable relevant to the ongoing experience [4,5]. Consistent with this proposal, value place cells initially remapped with context changes but became increasingly correlated across contexts with additional experience. In their remapping experiment, Knudsen and Wallis introduced new pictures, but used the same trajectory through value space. Thus, the task structure remained virtually unchanged. Accordingly, the observed experience-dependent correlations across contexts could reflect a generalization of the task structure, which enables rapid behavioral flexibility to different contexts by repeatedly traversing the same path through value space [9].

In a different experiment, the authors show that opposite directions of travel in value space produce largely uncorrelated value place fields. This result suggests that, rather than representing a precise position in value space, hippocampal neurons may represent the changing picture values as one of the salient features of the task. Future experiments with different trajectories through value space could assess this further: would hippocampal activity reflect the amount of change in the trajectory, or would additional place cells get recruited to represent new value positions? Moreover, if different trajectories share overlapping positions, would value place cells have a similar response in each location independently of the trajectory used to get there?

Importantly, in order to enable flexible behavior when encountering novel or uncertain situations, cognitive maps need to be updated and structured to reflect experiences. If, for example, reward contingencies unexpectedly flip, subjects need to detect the change and adapt their choice preferences. Spatial place cells in CA1 and in the entorhinal cortex show changes in activity that align with different reward locations [8], and regions of behavioral significance are often over-represented in the hippocampal spatial map. To maintain subjects’ engagement, Knudsen and Wallis included stable reward contingencies that were interspersed between periods of drifting values. Accordingly, in future experiments, it would be interesting to determine representations around these stable reward contingencies. An over-representation of value place cells around the stable periods would point towards a role of the hippocampus in encoding variables that are relevant to the subject, independently of the construction of a comprehensive cognitive space for value.

Knudsen and Wallis extend our understanding of reward as an abstract and relational parameter that is represented in the hippocampus even when monkeys passively experience transitions in value space. In rodents, active movement is a critical factor in generating place cells, which fail to emerge or become very large during passive motion [10]. As the authors acknowledge, an interesting question for future studies would be to determine whether the activity of value place cells is fundamentally different when the abstract value space is actively explored.

In summary, Knudsen and Wallis provide an innovative and elegant demonstration of hippocampal activity related to reward-based learning, bringing attention to how abstract variables can be represented within this network. This work paves the way for future studies of how even such abstract variables coalesce into a comprehensive representation of ongoing experience.