Reconciling neuronal representations of schema, abstract task structure, and categorization under cognitive maps in the entorhinal-hippocampal-frontal circuits

Abstract

Learning leads to a neuronal representation of acquired knowledge. This idea of knowledge representation was traditionally developed as a “cognitive map” of spatial memory represented in the hippocampus. The framework of cognitive mapping has been extended in the past decade to include not only spatial memory, but also non-spatial factual and temporal memory. Following this conceptual advancement, a line of recent neurophysiological research discovered such knowledge representations not only in the hippocampus, but also in the entorhinal cortex and frontal cortex. Although the distinct terms of “cognitive map,” “schema,” “abstract task structure” or “categorization” were used in these studies, it is likely that these terms can be reconciled as a common mechanism of learned knowledge representations. Future experimental work will be required to differentiate the parametric nature of knowledge representations across brain areas.

Introduction

Understanding how neural circuits develop spike representations of learned knowledge and how this representation is used for future behaviors is one of the major goals of learning and memory research. In the past few years, there has been an explosion of discoveries about neuronal representations of learned knowledge using neurophysiological recording techniques [1–7]. These studies report the neuronal representations during or after animals learn behavioral tasks in the entorhinal cortex [1], hippocampus [2–5] and frontal cortex [6, 7] (Fig. 1A). Although these studies distinctly refer to these representations using the conceptual terms of cognitive map, schema, abstract task structure, or categorization, what they found is likely a common mechanism of knowledge representation. Indeed, circuits of entorhinal cortex, hippocampus and frontal cortex are reciprocally connected (Fig. 1B) and shown to be indispensable for learning [8–10]. Here, we briefly review historical background of these psychological terms and suggest a unified interpretation of the recent exciting findings. Readers are also suggested to refer another review on memory flexibility by Takehara-Nishiuchi in the present issue.

Figure 1.

(a) Table of recent studies highlighted in this review. They report cognitive map-like representations across the hippocampus, entorhinal cortex, and frontal cortex.

(b) Schematic showing connections of the hippocampus (HPC) with the lateral entorhinal cortex (LEC), medial entorhinal cortex (MEC), and medial prefrontal cortex (mPFC). The mPFC receives hippocampal output from the ventral hippocampus, and output from mPFC reaches the dorsal hippocampus via output from nucleus reuniens of midline thalamus (nRe). Recently, direct hippocampus-projecting interneurons were found in the mPFC (Malik et al., Cell 2022). mPFC is reciprocally connected with orbitofrontal cortex (OFC).

Classical cognitive map for spatial memory

In the learning and memory research field, the idea of cognitive map has been the dominant framework to understand the representation of spatial memory. The concept of cognitive map was coined by Tolman, based on observing the behavior of rats in complex mazes [11]. After learning, rats appeared to possess a high-level, “overhead” understanding of the maze, as they would infer optimal routes and adapt to obstructions rather than learning individual paths one at a time [12]. He proposed the concept of the cognitive map, a mental model of the world in which the relationships between different objects and events are expressed. Such an organization of knowledge would account for animals’ ability to make inferences and flexible decisions in their environments, rather than having to learn every possible permutation through experience, which avoids significant computational expense.

Over the past decades, cognitive maps have become synonymous with the spatial navigation functions of the hippocampal-entorhinal network. The discovery of place cells in the hippocampus - neurons firing at single restricted locations in 2D space - provided a stark example of map-like representations in the brain [13]. In the emergent spatial theory of hippocampal function, place cells are fundamental units signaling the animal’s current position within an allocentric representation of space. O’Keefe and Nadel proposed that place cells in the hippocampus provide a neuronal basis for a cognitive map [14]. The subsequent discovery of grid cells in the medial entorhinal cortex (MEC) strengthened the role of the circuit between the MEC and hippocampus in the representation of cognitive maps for spatial memory [15, 16].

In contrast to the theory of hippocampus as a specialized spatial system, the medial temporal lobe including the hippocampus and entorhinal cortex is also widely recognized as indispensable for declarative memory - the ability to learn and recall not only spatial memory but also non-spatial facts, events, and time [17, 18]. Solving this discrepancy between the spatial cognitive map and non-spatial memory has been a driving force for the line of work by Eichenbaum [19–21]. In the late 90s and early 2000s, his group discovered that individual place cells show distinct firing rates depending on the odor types animals sniff at a particular location [22]. Another study showed that when rats are performing an alternating figure-8 maze, individual place cells showed distinct firing rates between left and right upcoming turns (Fig. 2a) [23]. With these findings, Eichenbaum concluded that place cells represent non-spatial information on top of spatial place information [24]. Additional works from the group and other labs further reported that place cells also respond to non-spatial aspects of experience, such as floor texture [25], odors [26–29], tone [30], passage of time [31–33] and motivational states [34,35], though behavioral impact of these representations needs further examination in future studies. Following this line of work, the learning and memory field congruently extended the framework of cognitive maps as internal knowledge representations for place, context, and time, rather than merely a spatial representation [19, 36–39]. It seems the field has caught up with Tolman’s original idea, which referred to a general mental model of cause-and-effect relationships in the environment [11]. The Wallis group recently showed that hippocampal neurons encode position in an imaginary space of reward values, suggesting the cognitive map can also include reward values [4]. In human imaging studies, the framework of cognitive map has been further extended from spatial representations to social relationships [40–42].

Figure 2.

(a) Hippocampal CA1 cells conjunctively encode spatial position and planned direction. Top, schematic of T-maze. Rats alternated turning left and right at the T. Middle, example hippocampal cell which fired along the maze stem selectively for upcoming left-turn trials. Bottom, another example hippocampal cell which fired at different areas of the maze stem depending on an upcoming left vs. right turn. Adapted with permission from Wood et al. Neuron 2000.

(b) Hippocampal CA1 population activity encodes spatial position and an abstract task-relevant variable (accumulated visual cues, referred to as “evidence”). Top, schematic of virtual-reality T-maze with two-photon imaging. Mice counted the accumulation of visual cues to make a left-or-right-turn decision. Bottom, organized conjunctive representation of position and evidence is visualized in a dimensionality-reduced representational space. Each dot represents a population representation at one timepoint in each trial, colored according to position (left) or evidence (right). Arrows represent neural trajectories through position- and evidence-space as mice traverse the maze. Adapted with permission from Nieh et al. Nature 2021.

Schema and assimilation

Closely related to the cognitive map concept, schema are mental frameworks built from experience. Schema guide the acquisition of subsequently learned knowledge, which is referred to as “assimilation.” Formulated by Piaget and expanded upon by Bartlett from psychological observation of human learning, schema describe how prior experiences shape expectations to assimilate new experiences [43,44]. Morris introduced schema learning in experimental configurations with rodents mastering a flavor-place association task, and found that schema learning is hippocampus-dependent [45]. Using immediate early gene expressions and pharmacological inhibition, the Morris group also proposed that schema is stored in the prefrontal cortex [46]. An electrophysiological recording study by the Wirth group showed schema-like neuronal representation existed in the macaque hippocampus [47]. Because the task used in the above studies were spatial tasks, it remained unclear if the hippocampus was involved in non-spatial schema learning. A study by the Eichenbaum group further reported schema representation in the hippocampus when olfactory task was used [48]. Recently, Schoenbaum’s group explored whether frontal cortical neurons represent schema [7]. In rats performing a nonspatial odor-sequence task, orbitofrontal cortex activity encoded sequence position and expected value, which they had previously referred to as “task structure representation” ([49], see next section for task structure representation). A subsequent study using the same task showed that this orbitofrontal cortex activity decreased in dimensionality while the sequence task structure was learned, suggesting an overall process of abstraction, which was referred to as “schema” development [7].

Do schema representations exist also in the entorhinal cortex? Although grid cells in the MEC support the classical framework of a spatial cognitive map, it remained unclear if the entorhinal cortex represents non-spatial acquired knowledge with learning. Using an odor-place association task, we previously demonstrated that LEC neurons develop learning-related changes of odor cue responses during learning [28]. Following up on this study, we recently asked if the LEC is involved in the representation of schema and memory assimilation [1] (Fig. 3). After mice learned an initial odor-reward association rule (schema), they were introduced with novel odor cues. As mice already acquired a schema, they quickly learned the associative rule for novel odor cues (assimilation) (Fig. 3A). Using an optogenetics-assisted cell-type specific spike recording method [50], we found that LEC layer 2a fan cells start to represent newly learned rewarded cues overlapped with that of a pre-learned rewarded cue, but these representations were separated from that of a pre-learned unrewarded cue (Fig. 3B). The observed overlapping representation of the newly learned cue in LEC fan cells, represented on top of the pre-learned cue, can be interpreted as a signature of “memory assimilation.” Optogenetic inhibition of fan cells impaired the learning of new associations, while sparing the retrieval of pre-learned memory. It is thus likely that LEC is primary involved in memory assimilation, rather than schema storage.

Figure 3.

(a) Schematic of odor-reward associative memory task from Lee et al. 2021. We propose that an odor discrimination schema is developed during initial training of odors A and B (familiar go- and no-go cue). After initial training, odors 1 and 2 (novel go- and no-go cues) are introduced and “assimilated” to the existing schema.

(b) Time-resolved trajectories of population representations by LEC layer 2a neurons during the odor-reward associative memory task. In principal components space, population activity for newly learned rewarded cue (Odor-1) becomes similar to previously-learned rewarded cue (Odor-A), whereas they are separated from previously-learned unrewarded cue (Odor-B). Adapted with permission from Lee et al. Nature 2021.

The above grouping representation found in the LEC can also be interpreted as “categorization” between rewarded cues and unrewarded cues. Because the rewarded cue – lick and unrewarded cue – no lick association is a fundamental rule in this task for mice, the representation can also be interpreted as an “abstract task rule” representation. Furthermore, the grouping representation of go and no-go cues can also be interpreted as a non-spatial cognitive map of learned knowledge. It is likely that schema, categorization, abstract task rule, and cognitive map are interchangeable terms, at least for interpreting our findings in the LEC. With the perspective of non-spatial cognitive map in the LEC, we proposed a unified framework for the roles of entorhinal cortex as a “cognitive map builder:” the MEC builds cognitive maps for spatial memory, whereas the LEC builds cognitive maps of items and rewards [1].

Abstract task structure

In the field of reinforcement learning, animals are referred to as ‘agents’ which learn optimal actions to maximize reward given a particular situation, or ‘task state’ [51]. These computational-field concepts have inspired approaches to studying learning in the brain. There is now a search for how the brain represents task states for reinforcement learning problems. It is thought that the brain may represent state in a low-dimensional, or abstract, manner which reflects the underlying ‘structure’ of the task at hand [52–55]. Such a representation of task structure could provide a substrate for model-based reinforcement learning [56]. Brain lesioning and fMRI studies suggest that task structure representations depend critically on the hippocampus and orbitofrontal cortex [57,58].

The Tank group recently asked if this representation of abstract task structure exists in the hippocampus, using two-photon imaging (Fig. 2b). Mice performed a virtual reality-based evidence accumulation task, where they counted numbers of visual landmarks appearing on left and right sides of the linear track to decide turning to left or right reward ports [3]. As a population, CA1 neurons encoded a joint representation of both spatial and non-spatial landmark counts, both variables relevant to represent the task structure. Thus the activity of CA1 at a given moment represents the current task state on top of the spatial information, consistent with the contextual modulation of place cell firing found previously [23]. Interestingly, the shape of the representation, or “geometry” of the cognitive map, was similar across mice.

Understanding the shape of neural representation, or neural geometry analysis, is a trending approach to unveil cognitive maps from high-dimensional recording data [59]. The Salzman and Fusi group reported orderly geometry of task state representations in the hippocampus and prefrontal cortex of monkeys [5]. Hippocampal spike activity dynamically encoded abstract, hidden task variables in a low-dimensional representational space. A different study from the Dupret group used graph theory to show that place-reward memory was represented in an abstract format, which they proposed as the backbone of memory schema [2]. These studies provide evidence that hippocampal neurons encode multiplexed representations of task variables, which can be dimension-deducted by experimenters as an abstract format of cognitive map.

Categorization

Here we attempt to extend the framework of knowledge representation into the neural representation of prefrontal cortex. Earlier studies from primates discovered the role of PFC in adaptive decision making, based on theories that PFC encodes abstract representations of goals or task rules [60, 61]. Categorization of these abstract representations by PFC neurons is key to serving as a flexible conduit between multiple circuit pathways [10]. Recently, the Bonhoeffer group developed a visual categorization task for mice and discovered PFC neurons that develop selectivity to categories during the rule-learning process [6]. We suggest that this categorical representation of learned knowledge can be interpreted as a cognitive map. Now that a path for rodent study is open, future work is expected to further integrate (or delineate) between categorization coding and cognitive map representation in the frontal cortical areas.

Reconciling cognitive maps, schema, abstract task structure, and categorization

The conceptual advancement in the representation of cognitive map, schema, abstract task structure, and categorization paved the way to the recent flourish in understanding how neuronal populations establish parametric representations of learned knowledge [1–7]. Although the individual works interpret their results with slightly distinct terminology, we suggest that they have touched upon a shared mechanism of learned knowledge. Because each terminology was independently introduced as conceptual ideas, there are currently no clearly defined links or distinctions between them. It is unknown if the brain has neuronal correlates of each of these conceptual terms. Rather than searching for neuronal correlates that fit these terms, it would be a logical strategy to estimate brain functions from observed neuronal representations (“inside-out approach”, [62]). In future works, neural geometry analysis will be a key first step for understanding representations of learned knowledge, as it provides parametric comparisons of multi-dimensional knowledge representations (Fig. 4). Comparing geometries of knowledge representation between brain regions and tasks is expected to provide further understanding of how individual brain areas support distinct types of learning. Until such a data-driven approach provides a complete picture of knowledge representations in the brain, using “cognitive map” as an umbrella term, or simply “knowledge representation” would be an unbiased approach for describing results.

Figure 4. Conjunctive cognitive maps in the CA1 and LEC.

(a) Diagram of a conjunctive cognitive map of space and context for an animal performing the alternating T-maze task (Wood et al., Neuron 2000; Nieh et al., Nature 2021). Individual CA1 neurons (circles) code information of context (x-axis; left vs right turn) and position (y-axis). When the animal is preparing a left turn, primarily left-turn neurons fire to inform the animal for the left-turn context. When the context is for right turn, neurons on the right side of the map will be firing primarily. As a result, trajectories of population representation become separated between left and right turns (arrows), presumably informing animals for correct decisions.

(b) Diagram of a cognitive map of object and reward contingency found in the LEC (Lee et al., Nature 2021). When each odor object is presented, the population representation transitions in the 2-dimensional space, presumably representing reward contingency (x-axis) and object amplitude (y-axis). Solid arrows indicate the time trajectories of neuronal representations when odors are presented, and dashed arrows show the returning of representations to zero-point during inter-trial intervals. Because discriminating the reward contingency of individual odor stimuli is the primary challenge for animals in this task, this representation may serve as a cognitive map of the task rule. Although 2-dimensional maps are shown here, CA1 and LEC neurons are likely to represent further information in higher dimensions.

Current challenges and future directions

Although we have begun to have a clear understanding of representations of learned knowledge, there remain several challenges. First, although some of the above studies found developing representations during learning of tasks, some of them recorded neural activity after learning. To specifically identify the role of learned knowledge representations, future studies need to involve tasks that allow recording of neural activity during learning. Second, most of these works observe learned knowledge representations during or after learning, but it remains unclear if the knowledge representations found are directly required for learning the tasks. Experiments are needed to test if perturbation of these representation impacts learning of animals. Third, it is unclear if the seemingly distinct results from different brain areas in the above highlighted studies derive from species and/or task differences in each study. It would be a logical strategy to compare representations using the same task in same species. Anatomically, the entorhinal cortex makes reciprocal connections with the hippocampus [63] whereas the hippocampus has direct output (ventral hippocampus→PFC) and indirect input (PFC→nucleus reuniens→hippocampus) with PFC (Fig. 1). A long-range direct projection of PFC interneurons to hippocampus was recently found [64]. These regions may exchange cognitive map representations during learning. Also, cognitive map representations in these regions would support executive functions including goal-directed behaviors and motor movements, although interdependence between these areas in executive functions remains unclear. There is an abundance of works showing goal-directed functions in the hippocampus and frontal cortical areas [65]. Understanding the interdependence of these areas in establishing and executing cognitive maps using circuit intervention techniques would be an exciting focus for future studies.