Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex

Abstract

The hippocampus and the orbitofrontal cortex (OFC) both have important roles in cognitive processes such as learning, memory and decision making. Nevertheless, research on the OFC and hippocampus has proceeded largely independently, and little consideration has been given to the importance of interactions between these structures. Here, evidence is reviewed that the hippocampus and OFC encode parallel, but interactive, cognitive ‘maps’ that capture complex relationships between cues, actions, outcomes and other features of the environment. A better understanding of the interactions between the OFC and hippocampus is important for understanding the neural bases of flexible, goal-directed decision making.

Despite possessing distinct neurochemical, anatomical and physiological properties, the hippocampal formation and the orbitofrontal cortex (OFC) have been ascribed broadly similar functional roles. For instance, a core feature of OFC function has been identified as “predicting the specific outcomes that should follow either sensory events or behavioral choices” (REF. 1), whereas others have conceptualized the hippocampal formation as a “system that facilitates predictions about upcoming events” (REF. 2). Although these statements are, of course, meant to provide very general descriptions of how each structure functions, the similarities are striking. Both structures are implicated in forming predictions about the future to support flexible behaviour and in leveraging general knowledge about the world rather than relying exclusively on specific previous experiences^1,2.

More formal theories have also arrived at similar functional specifications for these two structures. For decades, the hippocampus has been synonymous with mapping. Indeed, the cognitive map framework of hippocampal function — first proposed more than 40 years ago in the now classic 1978 work by O’Keefe and Nadel³ — is the most enduring theory of hippocampal information processing. Although this mapping function is frequently thought of in terms of spatial mapping, the cognitive map framework revisited the older concept of a cognitive map, as proposed by Tolman⁴. This referred not to a literal map of space, but rather to an abstract map of causal relationships in the world: that is, a set of mental representations that binds external sensory features with internal motivational or emotional factors to form an integrated relational ‘database’ (REFS 4–6; BOX 1). Subsequent work has shown that, beyond the spatial tuning of its principal neurons, many aspects of hippocampal physiology are consistent with the cognitive map function that Tolman envisaged, and it is well established that the hippocampus has a role in encoding information about the world in a way that facilitates flexible and inferential cognitive processes^7–12.

Box 1. A modern twist on Tolman’s cognitive map.

Tolman’s conception of learning was developed in reaction to the prominent stimulus–response theorizing that dominated in his day. In contrast to other researchers of the time, Tolman framed learning as an active process of extracting information from the world, rather than as a passive accumulation of associations imposed on the animal by the environment^5,143–145. Instead of learning individual action–outcome or cue–outcome relationships by storing specific instances of events, Tolman posited that animals track the underlying structure of the world in a map-like representation of causal associations. Tolman named this mental construct the cognitive map, an evocative description of the mental architecture he was proposing⁴. Just as a physical map allows one to plan novel or unique routes to a previously visited destination, a cognitive map would allow one to combine knowledge about causal relationships in the world in a manner that would enable one to derive novel and unique means of achieving outcomes. Importantly, although Tolman tested his ideas using spatial paradigms^146,147, he did not intend for cognitive maps to explain spatial planning alone. Instead, he envisioned a much more general system for creating schemas that encapsulate how the world works by tracking latent causal relationships between stimuli, actions, and outcomes.

Although much of Tolman’s thinking is now accepted¹⁴⁸, his ideas met resistance when they were first introduced, partially because of his difficulty in developing a theoretical framework that articulated his rather complex perspective¹⁴³. Although a mathematical description of the cognitive map eluded Tolman, subsequent advances in computational modelling of cognition and behaviour have expanded the range of processes than can be described mathematically. Many of the ideas that Tolman expressed have been subsumed by current models of learning and decision making. In particular, reinforcement-learning models⁶¹ and their progenitors from psychological learning theory^149–151 have proved to be useful for quantitatively describing different forms of value learning and decision making. Such models distinguish between two fundamental forms of learning and decision making, often labelled ‘model-free’ and ‘model-based’. Model-free algorithms learn the value of actions but do not learn specific information about the sensory properties, identity or other features of outcomes. Model-based systems store a richer set of associations and capitalize on a world model that tracks how different states of the world are linked together and the specific identity of the outcomes those states contain^152–154.

Many behaviours that Tolman attributed to cognitive map function can be solved using model-based reasoning^{143,152,154,155}. The cognitive map can thus be defined as an associative structure that facilitates model-based learning and behaviour. This definition remains true to Tolman’s thinking but also takes advantage of recent advances in computational modelling of complex behaviours to understand neural function. In line with this definition, we suggest that the cognitive map entails a number of components. First, there must be a mechanism for recognizing and categorizing the world into discrete states based on features that are relevant to current behavioural demands. Second, the cognitive map requires a means of learning and storing the relationships between world states (that is, how states are connected and how they are arranged relative to one another in the broader space of possible states). Third, the map must encode rich representations of outcomes that are associated with states (including their sensory features and identities) that can be used both to predict specific outcomes and, more generally, to estimate how ‘good’ or ‘bad’ states are expected to be in a way that reflects fluctuations in motivation, changes in the outcomes themselves and animals’ current needs or goals. Finally, there must be a mechanism to use all of this information prospectively to construct novel plans for reaching goals and to predict the outcomes that are likely to follow from combinations of cues or states that have never previously been experienced. The consilience of these modern constructs with Tolman’s older ideas suggests that the neural instantiation of the cognitive map might be distributed across the constellation of structures that support model-based, goal-directed behaviour, including the orbitofrontal complex and the hippocampus.

By contrast, the OFC has historically been associated with reward- and value-based behaviour^13–20. Recent thinking, however, has suggested that the OFC also has a cognitive-map-like role. Drawing on ideas from computational reinforcement-learning models, it has been proposed that a fundamental function of the OFC is to form and to maintain neural representations of task state: that is, a representation of all the relevant internal and external stimuli or features that define a particular situation in the world^21,22. Because this function requires the OFC to encode both features of the environment (including observable sensory properties and unobservable, implicit variables that must be inferred) and how relationships between those features might change in different situations, the OFC has been described as a cognitive map of task state²². Viewed from this perspective, the OFC and hippocampus appear to be involved in very similar cognitive processes.

Here, we examine similarities and differences in OFC and hippocampal contributions to cognitive mapping and flexible behaviour. We propose an updated definition of the cognitive map that is faithful to Tolman’s conception but that is grounded in contemporary computational models of reinforcement learning. We review evidence that the hippocampus and OFC have separate roles in cognitive mapping and examine in detail a handful of studies that have directly compared hippocampal and OFC processing in similar behavioural paradigms. Finally, we consider how information might be passed between the OFC and hippocampus and how such a cross-structural dialogue might contribute to behaviour that is dependent on cognitive mapping. Our discussion focuses on studies of the rodent dorsal hippocampus and lateral OFC, although anatomical diversity in both regions^18,23 and potentially across species^24,25 is an important consideration. The overarching message that we hope to convey is that the cognitive map perspective offers a productive unifying direction for future studies of hippocampal and orbitofrontal function.

The road to a cognitive map

Hippocampus

The discovery of place cells (FIG. 1a–c) quickly transformed our understanding of the hippocampus²⁶. In a masterful synthesis, O’Keefe and Nadel³ proposed the hippocampus as the seat of a Tolmanian cognitive map (BOX 1). However, the parallels between hippocampal function and Tolman’s ideas run far deeper than the existence of place cells. Beyond the intuitive correspondence between Tolman’s use of the term ‘map’ and the spatial tuning of place cells, other properties of hippocampal representations fulfil Tolman’s cognitive criteria.

Figure 1. Hippocampal and orbitofrontal cognitive mapping.

a | In spatial tasks, many hippocampal neurons exhibit spatially specific firing. The firing fields (place fields, represented in the image by coloured ellipsoids) of these ‘place cells’ tile the environment. At an ensemble level, the firing of place cells encodes the animal’s position in the environment. b | In a reinforcement-learning framework, the ensemble firing of spatially tuned place cells could be thought of as encoding an environmental state space: that is, it would represent both individual states (the circumscribed portions of the environment within which each individual place cell is most active) and how they connect to one another. States in this example environmental state space are coloured to correspond to the place fields shown in part a. c | As animals traverse the environment, the activity of hippocampel neurons could be thought of as representing trajectories through the environmental state space. The figure shows raster plots that illustrate the firing of seven individual hippocampal cells representing two different state space trajectories (coloured to match parts a. and b.). The trajectories overlap as the animal travels along the central arm of a T-maze but diverge for left and right turns. Different sets of place cells represent positions to the left and right of the choice point. In a similar manner, the activity of non-spatially tuned hippocampal neurons could represent position in a more abstract, non-spatial state space. d | Similarly, as animals are engaged in decision- making tasks, orbitofrontal cortex (OFC) neurons that are active during the performance of actions or other task events (such as the presentation of cues and outcomes) could encode the current task state. e In contrast to the example state space shown in part b, which was defined entirely by position in the environment, the state space for an operant decision- making task might be structured around important task events, include ing actions (such as making an initial nose poke in an odour sample port). cues (such as the presentation of an odour) and outcomes (such as the delivery of a liquid reward). In these examples, the states represented might be the ‘odour-sampling state’ or the ‘reward-delivery state’. f | Because different OFC neurons are activated by specific actions and events (for example, they fire in response to particular odour cues rather than to general odour presentation), the trajectories through state space encoded by OFC ensembles vary depending on the animal’s actions (in this case, a left or right response) and on the information received from the environment.

For instance, ensembles of hippocampal neurons quickly and obligatorily adopt place-specific firing patterns without explicit reinforcement, and place-cell activity holistically reflects the topology of the environment, as Tolman’s cognitive map was proposed to do^3,12. However, just as Tolman’s cognitive map extended beyond the spatial domain, place-cell activity encodes more than an objective, allocentric map of space. The spatial firing patterns of hippocampal neurons are often modulated by non-spatial factors, such as the presence or absence of objects^27,28, attention²⁹, conditioned stimuli³⁰, novelty^31,32, perceptual features of the environment³³ and an animal’s internal state³⁴. Place cells sometimes cluster their firing fields around — or show ancillary firing fields near — goal locations³⁵ or places that animals receive reward³⁶, indicating that motivational information can be tied to hippocampal spatial representations.

Moreover, the moment-to-moment dynamics of hippocampal activity suggest a mechanism by which animals might prospectively explore their mental models of the world to aid action selection. For example, in rats carrying out a goal-directed navigation task, hippocampal representations recorded just before an animal initiated a journey encoded paths that led to the next location the animal would visit³⁷. In another study, the extent to which pre-trial hippocampal activity was coordinated at an ensemble level predicted future correct choices³⁸. Even more surprisingly, the hippocampus synthesizes representations that include information beyond the animal’s direct experience. For example, in rats that had been trained on an apparatus with visible but inaccessible corridors, hippocampal representations encoded paths through the environment that crossed into corridors that the animals had never physically entered^39,40.

Lest it be thought that the hippocampal cognitive map is solely spatial, we would further emphasize that explicitly non-spatial information is also encoded by hippocampal neurons. This is frequently shown in studies of human hippocampal function. For instance, in a task that challenged participants to estimate the value of never-experienced outcomes composed of unusual combinations of familiar foods, the hippocampus repurposed existing representations of the component familiar foods to construct a representation of the composite snacks⁴¹. The hippocampus was also shown to map social relationships when participants were forced to use this information in order to perform well on a role-playing video game that required interaction with multiple virtual characters⁴². Hippocampal activity has also been shown to reflect the learning of predictive relationships embedded in sequences of visual cues, and the hippocampus is engaged when participants use these implicit stimulus–stimulus associations to guide their decisions^43–46.

Such non-spatial representations are not limited to humans. In rats that had been trained to expect sequences of odour cues in a consistent temporal order, hippocampal neurons showed sequence-dependent selectivity for individual odours: that is, they responded to a given odour only when it was a member of one sequence and not when the odour was presented in a different sequence^47,48. This sort of contextual modulation is also observed in directional place cells⁴⁹, which fire only when an animal passes through a place field in one direction of travel. Just as place cells encode sequences of positions in a way that reflects the order in which they are traversed, the activity of these odour-selective neurons preserved the temporal structure that was present in the environment.

In all, these data suggest that the hippocampus provides a supremely flexible system for rapidly encoding complex features of the world (both those that are directly experienced and those that are inferred), allowing important information about experience to be encoded in a way that preserves higher-order spatial and relational information.

Orbitofrontal cortex

Although it has long been known that the OFC is important for selecting behaviour that is suited to the current context, agreeing on a more precise specification of OFC function has been difficult^1,18,50,51. Early theories of OFC function centred on its role in suppressing actions that are inappropriate to the current context. However, more recent experimental work has argued against response inhibition as the primary function of the OFC^50,52, and contemporary theories have arisen from studies of OFC function in associative learning and decision making. Interestingly, OFC activity is not necessary for simple conditioning or even for certain forms of more complex learning⁵². However, tasks that force subjects to adjust their behaviour in light of new learning generally depend on OFC function. For instance, in outcome devaluation studies, an intact OFC is necessary for animals to respond to previously acquired cue–outcome and response–outcome associations in a manner that accords with recent changes in outcome value^13,53–56. Similarly, tasks that hinge on subjects’ knowledge of the specific sensory features of outcomes rather than their more general hedonic properties require OFC function^57–59. Such diverse findings have proved to be difficult for any single theoretical model to explain⁵⁰. These results might reflect a multiplicity of function within OFC circuits, or it might show that the OFC may have a general underlying function that accounts for its involvement in such a range of behaviours.

A potentially unifying account suggests that OFC represents task state²² (FIG. 1d–f). Reinforcement-learning models hinge on the ability of decision-making agents to parse the complexity of the world into discrete, well-defined states^60–63. According to these models, by segmenting the world in this way and tracking information about the collection of possible world states (that is, the ‘state space’), individuals can assign values to different states, depending on how good or bad those states themselves are and whether they predict future reward or punishment. A recent paper²², drawing on previous work that modelled how animals learn deeper world structure when solving conditioning paradigms^62,64,65, tested reinforcement learning models on a number of behavioural tasks, including classic benchmarks of OFC function, such as reversal learning and devaluation. The results showed that reinforcement-learning models in which the capacity to represent task states was absent or impoverished behaved remarkably like animals with OFC lesions. This suggested a role for the OFC in representing task state, particularly in cases in which states are abstract or not directly observable and must be derived from experience.

Patterns of activity in the OFC are broadly consistent with task state representations. For example, several results suggest that OFC neurons track variables that are crucial for task performance but that are not directly observable^50,66,67. In a task in which correct responses depended on the identity of odour cues presented in both the current and previous trials, OFC neurons encoded cue matches and mismatches⁶⁶. Moreover, recent work suggests that OFC ensembles might even encode counterfactual outcomes^68–70 — that is, consequences that would have occurred had the animal behaved differently. Animals could use this sort of ‘what if’ representation to improve future behavioural performance. In agreement with this, it was found that when animals are given incomplete information about a change in task contingencies, OFC neurons update their outcome expectancies before experiencing the complete set of new action–outcome pairings⁷¹. The OFC also tracks rule or strategy cues that define correct task performance^72,73.

Although the OFC is proposed to be especially important for tracking unobservable states, this does not preclude a role in processing sensory-bound state information. Indeed, evidence for OFC encoding of sensory information related to task performance is abundant. For instance, OFC neurons encode odour identity when odour cues predict a subsequent outcome and also prospectively signal features of impending rewards predicted by cues⁶⁷. However, just as hippocampal place cells show spatial tuning on tasks that do not explicitly tax spatial abilities, the OFC may encode observable state information even when such encoding is not necessary for the behaviour. This proposal is consistent with recent experimental work⁷⁴ showing that rats lacking medial OFC function behaved normally when task state was explicitly signalled but showed deficits when task state was unobservable. This work also highlights a challenge in matching OFC neural activity to animals’ state space representations: even well-controlled behavioural tasks are somewhat under-constrained and could be solved in different ways by individual subjects. Going forward, it will be important to construct paradigms that provide a behavioural readout of the state space that subjects are using⁷⁵ or that bias participants towards using particular state spaces⁷⁶. These approaches — combined with ensemble analyses that measure and interpret high-dimensional state space representations⁷⁷ — will be necessary to rigorously test the idea that OFC maps task state space.

These challenges aside, the state space perspective unifies a broad range of data on OFC function. For instance, economic value representations figure prominently in some models of OFC function^17,51. Such value coding might emerge from the representation of task state, particularly when state value most directly drives behaviour. Similarly, behaviours in which other factors dominate task structure would be expected to elicit more diverse OFC representations. For instance, tasks that depend heavily on action–outcome associations drive strong OFC encoding of action information that is absent in other situations^78,79.

It should also be noted that other brain regions, particularly other portions of frontal cortex, may also have an important role in cognitive mapping. Parts of the frontal cortex beyond the OFC have been associated with working memory, representation of task rules and top-down control of behaviour¹⁸. Such processes are clearly related to the cognitive map proposal, although they may, in some situations, be dissociable (this issue is explored in more detail in REF. 22). More work is necessary to determine how each of these brain regions fits within the framework described here.

Cognitive maps in action

Anatomy suggests various pathways by which cognitive maps in the hippocampus and OFC might interact to influence behaviour (BOX 2). Studies that have recorded or manipulated activity in these structures under the same behavioural conditions are of particular value in understanding the nature of this interaction. Here, we discuss four parallel data sets that are particularly amenable to direct comparison of OFC and hippocampal function.

Box 2. Hippocampus and orbitofrontal cortex: pathways for interaction.

There are at least three (not mutually exclusive) pathways by which the hippocampus and orbitofrontal cortex (OFC) might interact to influence behaviour: direct projections between the two structures, indirect projections and convergence on a common target (see the figure). We highlight several candidate pathways that could foster cross-structural communication.

Direct projections from the hippocampus reach multiple targets in the frontal cortex, including orbitofrontal regions^26,156–158. These projections are densest from the ventral hippocampus and become gradually scarcer towards the dorsal hippocampus. Projections that arise from the subiculum (which receives a strong, direct projection from CA1) largely mirror the CA1 projection to the cortex, contacting a similar constellation of frontal cortical targets, including the OFC¹⁵⁹. The OFC does not appear to return a direct projection to the hippocampus. Instead, there are several indirect channels through which orbitofrontal output might reach the hippocampal area. Recent anatomical work has mapped two polysynaptic pathways that link the frontal cortex with dorsal and ventral regions of the hippocampus by thalamic nuclei¹⁶⁰. In fact, both OFC and hippocampus send projections to and receive projections from the thalamic nucleus reuniens, which appears to have a particularly important role in coordinating bidirectional interactions between frontal cortex and hippocampal nuclei^161–165. In addition, OFC projections to parahippocampal structures, including the entorhinal, perirhinal and postrhinal cortices, are another path that links OFC and hippocampal processing streams¹⁶⁶.

Additionally, many brain regions receive broadly overlapping projections from the OFC and hippocampus. Several in particular seem striking for their role in learning and decision making. For instance, the ventral tegmental area (VTA) receives indirect input from both the hippocampus and the OFC. Dopamine-containing neurons in the VTA are thought to have a crucial role in associative learning¹⁶⁷, signalling reward prediction errors that drive learning. Lesions of the OFC alter the encoding of prediction errors in VTA neurons^21,168. Disrupting hippocampal outflow to VTA also has behavioural consequences, preventing context-induced reinstatement of drug seeking¹⁶⁹. In addition, electrophysiological experiments have found that hippocampus, VTA and frontal cortical regions are coupled by coherent, low-frequency local field potential oscillations¹⁷⁰.

The ventral striatum (vStr), which is another structure that has long been implicated in reward and motivational processes^171–173, also receives robust inputs from the OFC and the hippocampus. One subset of ventral striatal neurons encodes proximity to reinforcement with graded increases in firing rate^174–176 (‘ramp cells’), and the temporal patterning of ramp cell spiking is organized with respect to oscillations in the hippocampus¹⁷⁷, suggesting that hippocampal information processing may ‘clock’ reward-related representations in the ventral striatum¹³¹. Interestingly, OFC lesions interfere with the coding of reward magnitude in ventral striatal neurons¹⁷⁸, suggesting that OFC and hippocampal inputs converging in the ventral striatum may support complex, multi-attribute representations of predicted outcomes.

PFC, prefrontal cortex.

Goal-directed spatial decision making

A series of studies have compared hippocampal and OFC neural representations in rats that perform a T-maze decision-making task^69,80–83 (FIG. 2a,b). In this task, rats made decisions by turning left or right at the choice point of the maze. At the beginning of each session, rats relied on trial and error to determine which of three behavioural patterns (always turn left, always turn right or alternate left and right decisions) would be rewarded. Once animals found the correct strategy, a non-signalled switch in reward contingency occurred, and rats were again forced to identify which strategy would be reinforced. At the beginning of task sessions, when reward contingencies were unknown, rats often paused at the choice point before committing to a left or right turn. Tolman called this behaviour vicarious trial and error (VTE) and suggested that animals, when confronted with difficult decisions, were mentally simulating the consequences of potential actions before deciding which course of action to undertake^84–86. In agreement with this, VTE behaviour in the T-maze was found to follow rats’ performance: it decreased as they acquired the correct strategy but then re-emerged following the contingency switch later in the session.

Figure 2. Cognitive maps in action.

a | In the T-maze decision-making task, animals learn by trial and error that a particular pattern of choices (for example, always turn left at the choice point) will be rewarded. b | When animals deliberate over their options at the choice point, hippocampal ensembles simulate spatial trajectories towards potential reward sites. At the same time, reward-sensitive neurons in the orbitofrontal cortex (OFC) become active. It has been proposed that these neurons are engaged in outcome simulation, potentially providing a substrate for the evaluation of action plans represented by hippocampal ensembles. c | In the sensory-preconditioning task (as depicted on the top row), animals learn a predictive relationship between two neutral stimuli, such as tones, during the preconditioning phase. During the conditioning phase of this task, one of these stimuli is paired with reward. In test sessions, animals responded to the preconditioned cue that, although never directly paired with reward, predicts the occurrence of the conditioned cue. This behaviour has been likened to inference, as animals seem to correctly derive the implicit causal structure of the task (that is, tone A is followed by tone B, which is followed by a reward) despite never having directly experienced this arrangement. Lesion and inactivation data suggest a potential neural model of this task (as depicted on the bottom row) in which hippocampal ensembles encode the relationships between elemental stimuli during the preconditioning and conditioning phases of the task, and this information is accessed by the OFC to support responding during the test session.

Hippocampal ensemble activity was recorded as rats performed this T-maze task, and representations at the choice point were examined⁸³. During VTE, hippocampal place-cell ensembles encoded paths ahead of the animal, along the potential left- and right-turn options, suggesting that the hippocampus simulated each of the available options to aid decision making. Similar ‘look ahead’ representations in the hippocampus have been observed in rats navigating towards goal locations^87,88 and in human participants carrying out virtual navigation tasks^89,90. In computational terms, place-cell representations of locations removed from the rats’ actual positions could be thought of as a neural mechanism for exploring the environmental state space encoded in the hippocampus. However, for these sorts of state space searches to inform decisions in a useful way, information about the potential outcomes associated with environmental states must also be retrieved. One possibility is that the evaluation of trajectories represented by the hippocampus might take place in structures that receive hippocampal inputs, such as the OFC⁸³.

Subsequent work tested this directly by recording ensembles of OFC neurons as rats performed the T-maze task⁶⁹. OFC cells responded to reward receipt on correct trials, which is consistent with previous work. Surprisingly, however, these same reward-responsive OFC neurons were also active when rats paused at the choice point during VTE. Thus, prospective OFC reward-cell activity occurs approximately when an evaluation of hippocampal VTE representations would be expected to arise. Simultaneous recordings of hippocampal and OFC ensembles could clarify the precise temporal relationship between representations in each structure. Nevertheless, these data suggest that the hippocampus aids action planning by searching through the space of previously learned (in this case, spatial) associations, whereas the OFC evaluates candidate actions that are represented by the hippocampus to determine which is best. Note that such an evaluative role for the OFC is not inconsistent with its proposed function of state representation, because expected state value is likely to be an important feature of the trajectories that are proposed by the hippocampus.

The foregoing studies examined hippocampal and OFC activity under behavioural uncertainty and specifically during punctate moments of deliberation. An interesting counterpoint to these data comes from work that examined OFC and hippocampal coding throughout the course of entire trials^91–95. These experiments tested rats on a plus-shaped maze. Animals began each trial in one of the north or south start arms and had to travel to either the east or west goal arm, where food was delivered. In the ‘place’ version of this task, the reward was reliably located in one of the goal arms, and animals needed to approach this arm regardless of which start arm they began from. In the ‘response’ version of the task, animals learned to make a single response (turn left, for example), regardless of which arm they began from. The structure of the task allowed neural representations to be probed by several challenges, such as reversals of reward contingency or changes between response- and place-based task variants.

Hippocampal neurons recorded on the task showed spatial responses that were strongly context- or state-dependent^91,94,96. For instance, a place field located on the south start arm might only be active as the animal ran towards the west goal arm, showing no firing at all when the animal passed in the same direction through the identical location en route to the east goal arm. Similarly, a place field located on the west goal arm might be active only when the animal began its journey from the north start arm, showing no response when the animal began from the south. Thus, these prospective and retrospective place cells signalled spatial information only in the context of where the animal began from, or where it was going to, respectively. These data show that the hippocampus differentially parses identical instances of behaviour depending on non-spatial factors.

OFC neurons recorded on the same plus-maze task shared common features with hippocampal responses but also showed telling differences⁹². Rather than forming discrete place fields, OFC neurons tended to fire evenly along entire paths between start and goal arms. These neurons did not encode space per se; instead, path-sensitive OFC neurons reflected the probability that a particular path would lead to reward delivery, and the firing patterns of these cells tracked behavioural performance following contingency reversals and switches between tasks. This suggests that OFC representations integrated information about responses and reward expectation. Like the context-sensitive place cells observed in this task, OFC responses discriminated journeys through the same physical space depending on where the animal was travelling to and where the journey began. Interestingly, because OFC firing was spread over entire journeys rather than concentrated within discrete locations, as is the case for hippocampal responses, single OFC cells could reflect prospective information before reaching the choice point and retrospective information on final approach to the goal arm. This perhaps suggests a more integrated, large-scale representation of the task in single OFC neurons and a more granular, distributed encoding scheme by hippocampal ensembles^92,97,98.

Intriguingly, analyses of local field potentials (LFPs) that were recorded simultaneously from the OFC and hippocampus as rats carried out the plus-maze task hinted at interactions between these structures. During stable task performance, LFPs in the OFC and hippocampus oscillated coherently at theta (5–12 Hz) frequency. However, coherence fell following reversals of reward contingency or switches in task type only to rise again slowly as rats acquired the new behaviour⁹². Thus, learning resulted in a transient decoupling of activity in the hippocampus and the OFC, whereas consistent performance on the task was accompanied by stable interactions between these structures.

Taken together, the single-unit recordings suggest that both OFC and hippocampal firing patterns are influenced by contextual information related to task performance, such as the start and end locations of journeys across the maze. However, only in OFC ensembles was this contextual modulation dependent on the presence or absence of a food outcome at the end of trajectories. This suggests that the general spatial context modulation observed in the hippocampus gives way to a more elaborate representation in the OFC that is coloured by biological meaning — that is, factors that are closely related to the animals’ current needs or goals. The field potential data provide intriguing evidence that information flow between the hippocampus and OFC is a dynamic process that varies with learning.

One way to conceptualize OFC and hippocampal responses in the plus-maze task is to assume that both structures encoded somewhat overlapping information but with different emphases. Whereas neurons in the hippocampus encoded context information about rats’ journeys while preserving single-cell representations of position, neurons in the OFC seemed to average across large swaths of space, placing a premium on outcome encoding^91,97. This suggests that cognitive maps in the hippocampus and OFC might contain similar types of information but that they might format this information in different ways. This is consistent with recent work that assesses the hierarchy of spatial, contextual and outcome representations in the OFC and hippocampus^99,100. In these studies, rats were trained to choose between objects presented in the corners of a testing chamber to earn food reward. However, the objects were not consistently presented in the same corner, and the association between objects and reward was context-dependent: an object that was rewarded in one chamber was unrewarded when presented in the context of a second testing chamber. In this way, the task dissociated the location at which objects were presented from the outcomes that they predicted and also dissociated objects’ reward contingency from their sensory properties, making it possible to test how these task variables are encoded relative to one another. The authors used representational similarity analysis to determine which combinations of task variables evoked the greatest divergence in ensemble representations and which trial types were coded most similarly.

In hippocampal ensembles, spatial information fractionated neural representations: that is, trials that occurred in different contexts were encoded by anti-correlated patterns of activity, and the location of objects within a context drove the next-greatest divergence in ensemble representations¹⁰⁰. Nevertheless, non-spatial information was also encoded: representations of object–reward associations were more strongly segregated than representations of the individual objects’ identities. In OFC ensembles, a different hierarchy of representations emerged. Neurons in the OFC distinguished most sharply between rewarded and non-rewarded objects, suggesting that reward contingency is the major dimension along which OFC representations are constructed in this task⁹⁹. The next greatest separation in ensemble representations reflected the location of items within each context. Finally, the absolute position at which trials occurred was most similarly represented by ensemble activity. These data suggest that, unlike hippocampal representations, which strongly reflected where events occurred, OFC activity principally favoured reward contingency over location. Thus, hippocampal and OFC ensembles both encode a variety of task-relevant information, but each is specialized to emphasize different aspects of the resultant cognitive map.

Inferring implicit value

The OFC and hippocampus are both known to be involved in situations in which previous learning is essential — but in itself insufficient — to support behaviour. A particularly salient example of this is sensory preconditioning¹⁰¹ (FIG. 2c). In this task, subjects are first exposed to an arbitrary pairing of two neutral stimuli (for example, tone A is followed by tone B). Neither stimulus is paired with reward: subjects are simply presented with the cues in a reliable order. Next, in the conditioning phase of the experiment, one of the previously unrewarded cues is paired with a valuable outcome (for example, tone B is followed by a reward). Finally, in the test session, animals are presented with the preconditioned tone A. Across a range of species^102–110, subjects’ response to the preconditioned cue is found to be similar to that evoked by the directly conditioned cue (that is, tone B). The interpretation is that subjects inferred the complete causal chain that was made implicit in a piecemeal manner over the different phases of the experiment (tone A is followed by tone B, which is followed by a reward). Model-free cue values are insufficient to explain subjects’ responding at test, suggesting that model-based inference must be at work.

Parahippocampal structures have long been implicated in supporting inferential behaviour in studies of sensory preconditioning. Early work suggested that hippocampal lesions prevented value inference from the directly conditioned stimulus to its preconditioned partner^104,111. Although subsequent work has reported that the hippocampus is not necessary for sensory preconditioning, when cues are presented simultaneously as compounds¹¹², rather than serially as described above¹⁰⁶, other structures within the hippocampal network have also been implicated. For instance, lesions of the perirhinal¹¹³ or retrosplenial¹¹⁴ cortex abolish sensory preconditioning. Although these studies leave open what function is supported by processing through the circuit, recent work using a chemogenetic approach found that specifically silencing the retrosplenial cortex during the preconditioning phase prevented value inference at test without influencing first-order conditioning¹¹⁵. Thus, hippocampal outflow is potentially crucial to the establishment of the associative scaffolding that is later used to infer the value of the preconditioned cue at test. Consistent with this idea, humans tested on a task that was similar to the sensory preconditioning paradigm used in animals¹⁰⁸ showed enhanced preference for the preconditioned cue at test, and the strength of this preference correlated with hippocampal activity during the learning phase of the task.

Notably, these data dovetail with evidence that the OFC is necessary for this inference process¹¹⁶. Pharmacological inactivation of the rat OFC during the test session abolished responding to the preconditioned cue without affecting responding to the cue that underwent first-order conditioning by being paired directly with reward. These data indicate that the OFC is necessary for using the hippocampus-dependent associative scaffolding acquired in the first phase to predict reward at test.

Overall, these data suggest that both regions are crucial for value inference in sensory preconditioning but that the dynamics of hippocampal and OFC involvement differ in important ways. For example, whereas OFC function was found to be important for inference at the test stage¹¹⁶, hippocampal involvement was confined to the conditioning portion of the experiment in both the human¹⁰⁸ and animal work¹¹⁵. These data suggest that the hippocampus is perhaps crucial for encoding a world model that links preconditioned cues with reward, whereas the OFC is more important for accessing this information in the test session to drive responding.

Putting it all together

The evidence reviewed here suggests that the OFC and hippocampus each contribute to cognitive mapping and the resultant behaviour. We have emphasized experiments that demonstrate the striking similarity in OFC and hippocampal function. This is not to say, however, that the hippocampus and OFC operate synergistically in every situation. For instance, hippocampal lesions selectively alter rats’ preferences for delayed outcomes^117–119 (at least in some testing paradigms; also see REF. 120) without affecting decisions between probabilistic outcomes, whereas OFC lesions produce an inverted pattern of impairment¹¹⁷. Similarly, tests of outcome devaluation paint a somewhat divergent picture in which the OFC appears to be involved in behavioural changes that are induced by specific satiety manipulations^121–123, whereas the hippocampus is not^124–129. Differences in experimental design often make direct comparisons difficult, but these discrepancies point to potential functional differences.

It also seems clear that OFC and hippocampus, although perhaps contributing to similar types of behaviours, show some level of domain specificity in their information processing. This arises both from the unique anatomical organization within each structure and the pattern of inputs they receive from other brain regions. The hippocampus is remarkably adept at linking information into temporally patterned sequences that span large ensembles of neurons^130,131 and seems particularly concerned with organizing experience along the axes of space and time. This ability to connect elemental representations and to flexibly produce sequences that reflect learned connections makes the hippocampus well suited to encoding, retrieving and exploring mental models of state spaces^132–134. By contrast, representations of similar information in the OFC are more rooted in biological importance^20,135,136. Although it is clear that the OFC has an important role in cognitive processes such as learning and decision making, the behaviours that depend on the OFC are motivated by biological needs. This includes, for instance, learning how to respond to obtain food or liquid reward or, in the case of devaluation, updating associations to direct behaviour away from food that had previously been linked with illness, as well as behaving appropriately in social situations^137–139. These data argue that, whereas the hippocampus is a flexible and promiscuous processer of abstract associations, OFC information processing may be more grounded in items of immediate biological relevance.

One way to view these distinctions is to consider that there are multiple components to the cognitive map. In other words, just as real-world maps often consist of multiple overlays that describe different aspects of the environment, so too must our global cognitive map be constructed of multiple informational layers that can be turned on and off as necessary. One efficient way of doing this is to task different modules in the brain with representing different informational layers of the global map, thereby distributing cognitive maps across multiple neural structures. If cognitive maps were to be distributed across multiple neural structures, coordination between brain regions would be important for supporting adaptive behaviour. What precise form might this cross-structural dialogue take? One intuitive idea is that OFC input to hippocampus contributes to the extra-spatial modulation of place-selective hippocampal neurons, giving rise to spatial representations that are sensitive to reward, goals or motivational states. The OFC might be thought of as imbuing the hippocampal map with information about expected outcomes to facilitate goal-dependent navigation. Conversely, spatial or relational information conveyed by the hippocampus to the OFC might allow OFC outcome expectancies to become bound with information about spatial positions or more abstract relationships between potential outcomes and ‘paths’ to obtain those outcomes, whether spatial, as in a maze, or non-spatial, as in sensory preconditioning. Organizing OFC representations in this way could facilitate the development of integrated action–outcome associations by tying abstract outcome predictions (such as a cherry- or banana-flavoured sucrose pellet) to locations that are reached by particular responses (for example, go left at a maze choice point to get the banana pellet, or press the right lever in an operant box to get the cherry pellet). Time-sensitive hippocampal representations^140–142 might also confer temporal specificity to OFC expectancy representations, either directly or through interactions in some downstream areas, such as the ventral striatum.

Although we are in the early stages of understanding the interplay between brain regions, the increasing sophistication of techniques for measuring and manipulating the activity of neural circuits in projection- and cell-type-specific ways draws the issue of cross-structural interactions to the fore. We suggest that these approaches, when applied to interactions between hippocampus and OFC, might be particularly fruitful in improving our understanding of how cognitive-map-dependent behaviour is learned and deployed.

Glossary

Economic value

An integrative measure of how good an outcome is to a decision maker that distils the many multidimensional features of that outcome into a unidimensional measure of worth.

Outcome devaluation

The process of rendering a normally appetitive outcome aversive, typically by pairing it with illness.

Place cells

Pyramidal neurons in the hippocampus that fire action potentials when an animal occupies or passes through particular portions of the environment.

Reinforcement-learning models

A collection of machine-learning models that are inspired by psychological learning theory and that are aimed at solving the problem of using experience of the world to guide future behaviour.

Representational similarity analysis

An analysis approach that quantifies the similarity (or dissimilarity) of neural ensemble representations evoked by different conditions.

Response inhibition

The active suppression of actions that are not adaptive in the current setting.

Specific satiety

A means of devaluing a particular outcome by allowing an animal unrestricted access to it before a test session.

Stimulus–stimulus associations

Associations that are formed between neutral stimuli in the environment in the absence of explicit reinforcement.

Vicarious trial and error (VTE)

A pause and orient pattern of behaviour that decision makers often show when deliberating over potential choices. This is thought to be an overt marker of covert, mental processes that simulate potential outcomes of each course of action.