Entorhinal-hippocampal interactions lead to globally coherent representations of space

Abstract

The firing maps of grid cells in the entorhinal cortex are thought to provide an efficient metric system capable of supporting spatial inference in all environments. However, whether spatial representations of grid cells are determined by local environment cues or are organized into globally coherent patterns remains undetermined. We propose a navigation model containing a path integration system in the entorhinal cortex and a cognitive map system in the hippocampus. In the path integration system, grid cell network and head direction (HD) cell network integrate movement and visual information, and form attractor states to represent the positions and head directions of the animal. In the cognitive map system, a topological map is constructed capturing the attractor states of the path integration system as nodes and the transitions between attractor states as links. On loop closure, when the animal revisits a familiar place, the topological map is calibrated to minimize odometry errors. The change of the topological map is mapped back to the path integration system, to correct the states of the grid cells and the HD cells. The proposed model was tested on iRat, a rat-like miniature robot, in a realistic maze. Experimental results showed that, after familiarization of the environment, both grid cells and HD cells develop globally coherent firing maps by map calibration and activity correction. These results demonstrate that the hippocampus and the entorhinal cortex work together to form globally coherent metric representations of the environment. The underlying mechanisms of the hippocampal-entorhinal circuit in capturing the structure of the environment from sequences of experience are critical for understanding episodic memory.

Graphical abstract

Highlights

• •A computational model contains a path integration system in entorhinal cortex and a cognitive map system in hippocampus.
• •During first exposure to an environment, neural codes of grid cells and head direction cells are anchored to local cues.
• •After revisiting familiar places, grid cells and head direction cells form globally coherent representations.
• •Entorhinal-hippocampal interactions allow grid cells to act as a spatial metric.

1. Introduction

The cognitive map, a map-like internal representation of the spatial environment, allows an animal to navigate efficiently in the space (Tolman, 1948). Grid cells in the medial entorhinal cortex (MEC) of mammalian brains form regular grid-like firing patterns spanning the whole explored environment (Hafting et al., 2005; Yartsev et al., 2011). The periodicity of grid maps allows highly effective multi-resolutional spatial representations of large environments (Sreenivasan and Fiete, 2011; Mathis et al., 2012; Vágó and Ujfalussy, 2018). Grid cells are thought to be the core of an intrinsic positioning system for mammals performing spatial navigation tasks in the environments (McNaughton et al., 2006; Fiete et al., 2008; Buzsáki and Moser, 2013). Grid cell activity is likely to provide a universal spatial metric system across all environments (Fyhn et al., 2007; Moser et al., 2014; Bush et al., 2015; Yan et al., 2016).

Recent experimental investigations however discovered that grid firing patterns are distorted and fragmented by various environmental features, revealing a more complex picture deviating from the ideal notion of rigid metric representation. Grid cell firing patterns rescale in response to the shrink or stretch of the environment (Barry et al., 2007). The hexagonal firing patterns of grid cells are rotated and deformed by the salient geometric borders of the environment (Stensola et al., 2015). Environmental geometry strongly alters the symmetry, scale, and homogeneity of grid firing in highly polarized environments, such as trapezoids (Krupic et al., 2015). In addition, complex environments are not represented holistically by one global map in the hippocampus and entorhinal cortex, but rather by multiple submaps that are anchored to the environment's local space (Derdikman et al., 2009). These experimental results indicate that grid firing patterns represent spatial environment by local submaps that are elastic and adaptive to local sensory cues (Rosay et al., 2019).

Regular and global coherent grid firing patterns support spatial navigation (Bush et al., 2015), as demonstrated by robot experiments (Yuan et al., 2015; Zeng and Si, 2017; Yu et al., 2019). A question needed to be addressed is how to form globally coherent grid maps in order to provide accurate and universal metrics uniformly covering the entire environment explored.

Recent exciting experiments show that grid patterns self-correct with exploration experience (Carpenter et al., 2015). Local sensory cues in multiple compartments of the environment initially dominated grid firing patterns, and after sufficient familiarization of local space, the distortions and discontinuities in grid firing patterns were reduced, resulting in globally continuous grids (Carpenter et al., 2015). If the partition between the multiple compartments was removed, the spatial periodicity of grids was established quickly (Wernle et al., 2018). These results imply that the formation of globally coherent grid pattern is contingent on the traverse of the environment in long-scale trajectories.

A globally coherent representation is necessary to establish the distance relationship between locations in the environment. The transition from a local to a global representation has not been computationally addressed by previous models (McNaughton et al., 2006; Fuhs and Touretzky, 2006; Yoram Burak and Ila R Fiete, 2009). Some computational models of grid cells however showed that the learning of spatial firing rate map from sensory-motor inputs during exploration contributes to the formation of coherent grid maps (Si and Treves, 2013; Mulas et al., 2016).

Stable grid codes in MEC require the inputs from the hippocampus (Bonnevie et al., 2013). In general, the hippocampal outflow is necessary for maintaining spatial representations in a wide range of neocortical regions, including retrosplenial cortex and posterior parietal cortex (Esteves et al., 2021). Anatomical studies have shown that the hippocampus projects strongly to the entorhinal cortex (Rozov et al., 2020). The feedback projection from the hippocampus as well as the interlaminar connections within the entorhinal cortex are critical links to mediate information from the hippocampus to grid cells in MEC (Kloosterman et al., 2003; Si and Treves, 2013). The feedback information from the hippocampus could be relayed to the entorhinal cortex through oscillations such like sharp wave ripples (SPW-Rs) (Chrobak and Buzsáki, 1996; Leonard et al., 2015; Zheng et al., 2021). Yet it remains an open question how the hippocampal feedback supports the formation of globally coherent grid codes in MEC.

In this paper, we address the question of forming globally coherent patterns to represent spatial metrics for navigation. We propose that the feedback from hippocampus plays an important role in the formation of globally coherent firing maps. The feedback is in the form of map calibration error, and possibly realized by the firing activity correction mechanism from the hippocampus to the entorhinal cortex. We demonstrated, by map learning task in an unfamiliar realistic environment, that globally coherent firing maps of grid cells and HD cells develop during long-term exploration using the map calibration error propagated from the hippocampus (Fig. 1). Previous models without the feedback mechanism from hippocampus however express locally anchored firing maps (Fig. 4B vs. C). During the formation of globally coherent maps, sensory cues play an important role in calibrating local representations, especially during loop closures where motion cues and visual cues are combined to minimize the mismatch. Our model therefore predicts that loop closure or revisit of familiar places are important for spatial learning.

Fig. 1.

Globally coherent encoding of space requires long-term exploration. (A) Spatial representations in a complex maze. Initially, the grid cell firing pattern is determined by local sensory cues (bottom left). After experience-dependent correction during the traverse of the environment, the grid cell firing pattern becomes globally coherent (bottom right). (B) Spatial representations on a road. Before loop closure by revisiting familiar places, the grid pattern is locally anchored due to path integration errors (bottom left). After loop closures, globally coherent pattern is formed (bottom right).

Fig. 4.

Map calibration in hippocampus and activity correction in MEC during loop closures are necessary for the formation of globally coherent firing maps. (A) Inconsistent representations without map calibration. Top: The summed firing map of the HD cells with north or south preference in the ring manifold. Bottom: The firing map of an example grid cell at (0,0) in the torus manifold. (B) Locally anchored representations. Top: Without firing activity correction, the summed firing rate map of the HD cells at 0 and π in the ring manifold anchors to local space, preferring particular directions at different part of the environment. Bottom: The firing map of the grid cell at (0, 0) in the torus manifold has multiple firing fields, which are anchored to a local region on a rectangular grid. (C) Globally coherent representations. Top: With firing activity correction, the summed firing rate map of the HD cells at 0 and π in the ring manifold has strong activity along the trajectories orienting towards east and west, showing global coherency across the environment. Bottom: The globally coherent grid pattern of the grid cell at (0,0) in the torus manifold has regularly spaced firing field across the whole environment. In each panel, firing rate is plotted at the locations in the experience map. The colorbar shown to the right of each panel color-codes peak firing rate with red and silent activity with blue. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

We tested our model based on our previous simultaneous localization and mapping (SLAM) system (Zeng et al., 2020) on an iRat rodent-sized robot platform in a naturalistic maze (iRat 2011 Australia dataset) (Ball et al., 2010, 2013). In our robot experiments, grid and HD firing patterns anchor to local sensory cues before loop closure optimization. After correction with experience, periodic grid patterns tile two-dimensional environment coherently, and the neural codes of HD cells converge to their respective directions across the explored space. Our experimental results provide a viable explanation to global coherent representations of grid cells and HD cells for navigation in complex spatial environments (Carpenter et al., 2015).

In summary, the major contributions of this work are in the following. First, we proposed a model both for grid cells and HD cells that integrate local representations initially anchored to local environmental cues into globally coherent firing patterns. The globally coherent representations encode geometry metrics of spatial locations or head-direction angles. Second, we implemented the model on a SLAM system, and demonstrated, in robot navigation experiments, that global representations of grid cells and HD cells emerge after longer-term exploration. The study in this paper provides theoretical supports that globally coherent patterns of grid cells and HD cells can emerge through exploration experience to function as an accurate and universal metric required by large-scale and long-term spatial navigation (Carpenter et al., 2015; Wernle et al., 2018).

2. Methods

In our model, the positions and head directions of an animal are represented by neural activities of Bayesian attractor neural networks. The metric relationships of spatial locations and head directions are mapped into the corresponding neural space. The discrepancy of the metric relationship between locations and head directions is minimized by solving a non-linear least-squares problem.

2.1. Bayesian attractor neural network model

The Bayesian attractor neural network model takes the form of probabilistic distributions to encode head directions or spatial locations (Zeng et al., 2020). The model includes integrator cells and calibration cells, which integrate vestibular inputs and visual inputs respectively. The conflicts between vestibular cues and visual cues are solved by mutual inhibition between integrator cells and calibration cells. The activities of the two populations are stabilized by global inhibition.

2.1.1. Head direction cell model

HD cell model represents the rotation of the animal, whose velocity input is the angular velocity of the animal in the physical environment. The neural activity of the HD cell model is updated by attractor dynamics, vestibular cues integration, and visual cues calibration.

2.1.1.1. Attractor dynamics

The head direction of the animal is represented by a normal distribution on a ring manifold [0, 2π). The firing rates of integrator cells and calibration cells in the head direction network are concisely described by the respective means and variances of the distributions

$$f(\theta )=\frac{1}{\sigma \sqrt{2\pi }}{e}^{-|\theta -{\mu }_{\theta }{|}^2/2{\sigma }^2},$$ (1)

where θ is the coordinate of a HD cell in the ring manifold, and |⋅| computes the distance between two angles on a circle. μ_θ, the mean of the probability distribution, represents the center of the firing activity profile of the HD cells. The Normal distribution in Eq. (1) has the bump shape as the attractor states in continuous attractor networks (Tsodyks and Sejnowski, 1995; Zhang, 1996).

The attractor dynamics of the head direction network are achieved by global inhibition and mutual inhibition between the integrator cell and calibration cell.

The global inhibition is defined by a normalization mechanism

$$\begin{array}{lll}\frac{1}{{\sigma _{\text{cc}}^{t-1}}^2}& =\frac{1}{{\sigma _{\text{inte}}^{t-1}}^2}+\frac{1}{{\sigma _{\text{cali}}^{t-1}}^2}& \\ \frac{1}{{\sigma _{\text{inte}}^t}^2}& =E\frac{\frac{1}{{\sigma _{\text{inte}}^{t-1}}^2}}{\frac{1}{{\sigma _{\text{cc}}^{t-1}}^2}}& \\ \frac{1}{{\sigma _{\text{cali}}^t}^2}& =E\frac{\frac{1}{{\sigma _{\text{cali}}^{t-1}}^2}}{\frac{1}{{\sigma _{\text{cc}}^{t-1}}^2}},& \end{array}$$ (2)

where $\frac{1}{{\sigma _{\text{inte}}^{t-1}}^2}$ and $\frac{1}{{\sigma _{\text{cali}}^{t-1}}^2}$ are the confidence of the activity of integrator cells and calibration cells at the previous time step, respectively. $\frac{1}{{\sigma _{\text{cc}}^{t-1}}^2}$ is the total confidence. E is a constant, which is the total neural activity energy.

The mutual inhibition is modeled by

$$\begin{array}{lll}\frac{1}{{\sigma _{\text{inte}}^t}^2}& =\frac{1}{{\sigma _{\text{inte}}^{t-1}}^2}-{\mathrm{\Delta }}_{\text{inte}}\frac{1}{{\sigma _{\text{cali}}^{t-1}}^2}& \\ \frac{1}{{\sigma _{\text{cali}}^t}^2}& =\frac{1}{{\sigma _{\text{cali}}^{t-1}}^2}-{\mathrm{\Delta }}_{\text{cali}}\frac{1}{{\sigma _{\text{inte}}^{t-1}}^2},& \end{array}$$ (3)

where Δ_inte and Δ_cali are the inhibition intensities. The confidence of each population is guarded by a lower bound U, allowing the weak population to recover at the time when reliable cues are available.

2.1.1.2. Vestibular cues integration

Path integration is performed by shifting the mean of the normal distribution without bump deformation. The integrator cells integrate vestibular cues, and drive the calibration cells to update the bump activity as well

$$\begin{array}{c}{\mu }_{\text{inte}}^t=\mathrm{mod}({\mu }_{\text{inte}}^{t-1}+{\nu }^t\mathrm{\Delta }t,2\pi ),\\ {\mu }_{\text{cali}}^t=\mathrm{mod}({\mu }_{\text{cali}}^{t-1}+{\nu }^t\mathrm{\Delta }t,2\pi ),\end{array}$$ (4)

where ${\mu }_{\text{inte}}^t$ and ${\mu }_{\text{cali}}^t$ are the centers of the bump attractors of the integrator cells and the calibration cells, ν^t is the rotation velocity, Δt is the time interval between time steps t and t − 1. mod (x, L) is the modulus operation in the domain of real number, and returns the remainder in the range [0, L).

2.1.1.3. Visual cues calibration

The neural activity of HD cells is calibrated by familiar sensory cues. When a new view is perceived, this new view is associated with a new local view cell corresponding to the current HD activity pattern by a strong link. When a familiar view is encountered, the local view cell is reactivated by template matching mechanism. Energy is injected into HD cells network through the learned link in a Bayesian fashion

$$\begin{array}{lll}& & \frac{1}{{\sigma _{\text{cali}}^t}^2}=\frac{1}{{\sigma _{\text{cali}}^{t-1}}^2}+\frac{1}{{\sigma _{\text{inject}}^t}^2}\\ & & {\mu }_{\text{cali}}^t=\mathrm{mod}\left(\left(\frac{{\mu }_{\text{cali}}^{t-1}}{{\sigma _{\text{cali}}^{t-1}}^2}+\frac{{\mu }_{\text{inject}}^t}{{\sigma _{\text{inject}}^t}^2}\right){\sigma _{\text{cali}}^t}^2,2\pi \right),\end{array}$$ (5)

where $\frac{1}{{\sigma _{\text{inject}}^t}^2}$ is the confidence of the injected energy, ${\mu }_{\text{inject}}^t$ is the injected location on the one dimensional neural manifold of HD cells.

2.1.1.4. Cue combination

The activities of the integrator cells and the calibration cells are combined according to the Bayesian rule, resulting in the combined confidence and HD phase

$$\begin{array}{lll}& & \frac{1}{{\sigma _{\text{cc}}^t}^2}=\frac{1}{{\sigma _{\text{inte}}^t}^2}+\frac{1}{{\sigma _{\text{cali}}^t}^2}\\ & & {\mu }_{\text{cc}}^t=\mathrm{mod}\left(\left(\frac{{\mu }_{\text{inte}}^t}{{\sigma _{\text{inte}}^t}^2}+\frac{{\mu }_{\text{cali}}}{{\sigma _{\text{cali}}^t}^2}\right){\sigma _{\text{cc}}^t}^2,2\pi \right).\end{array}$$ (6)

If the combined HD phase is similar to the HD phase of the calibration cells, i.e. $|{\mu }_{\text{cc}}^t-{\mu }_{\text{cali}}^t|$ is smaller than a threshold δ, the decision that the animal revisits a familiar place is made, and the calibration cells inherit the combined distribution of HD phase. Otherwise, the calibration cells keep their own estimation and neglect the distribution calibrated by visual cues.

2.1.2. Grid cell model

To model grid cells, we expand the ring manifold of the HD cell model to the torus manifold. The grid cell model adopts the same mechanism as the HD cell model to encode positions in two-dimensional environments. The firing activities of integrator cells and calibration cells in the torus manifold are described by

$$f({\theta }_x,{\theta }_y)=\frac{1}{2\pi {\sigma }_x{\sigma }_y}{e}^{-({\left(|{\theta }_x-{\mu }_x|\right)}^2/2{\sigma }_x^2+{\left(|{\theta }_y-{\mu }_y|\right)}^2/2{\sigma }_y^2)},$$ (7)

where (θ_x, θ_y) is the coordinate of a cell in the torus neural manifold, with θ_x, θ_y ∈ [0, 2π). (μ_x, μ_y) is the center of the bump attractor of the grid cell network.

To model the modular organization of grid cells along the dorsal ventral axis, the grid cells in different modules integrate vestibular cues with different parameters

$$\begin{array}{c}{\mu }_{\text{inte}}^t=\mathrm{mod}({\mu }_{\text{inte}}^{t-1}+\rho R(\alpha ){\nu }^t\mathrm{\Delta }t,2\pi ),\\ {\mu }_{\text{cali}}^t=\mathrm{mod}({\mu }_{\text{cali}}^{t-1}+\rho R(\alpha ){\nu }^t\mathrm{\Delta }t,2\pi ),\end{array}$$ (8)

where ${\mu }_{\text{inte}}^t$ and ${\mu }_{\text{cali}}^t$ are two dimensional vectors representing the centers of the bump activities of the integrator cells and the calibration cells. ν^t is the running velocity. R(α) is the rotation matrix to offset the movement direction by α. ρ > 0 is a gain factor scaling the shift speed of the bump in neural space.

The grid cell model uses the same mechanisms, including attractor dynamics, visual cues calibration, and cue combination, like those of the HD cell model to integrate linear velocity and sensory cues. We will not go further into the details here.

2.2. Map calibration by odometry error minimization

Mammals are able to explore long distances for forage, and rightly return to their home nests. The familiar views near their home nests can calibrate the cognitive maps in their brains to form globally coherent maps to ensure that they do not get lost.

In our model, local view cells are used to encode distinct visual views in the environment. When a novel view is perceived, a new local view cell is recruited and the visual features of the view are associated with this new local view cell. At the same time, the new local view cell is linked to the activity states of the grid cells and the HD cells at that moment. A topological map is constructed to represent the experience of the animal. An experience is defined as the activated local view cell, and the linked neural activity states of the grid cells and the HD cells. Each experience is added to the graph as a node. The spatial constraint between two nodes is modeled as a link. During exploration, new links can be easily added to the graph to impose new spatial constraints.

During loop closure, the animal sees a familiar view again. The matched view cell activates, and injects energy into the grid cells and the HD cells through the excitatory link. Given that each view is associated with a different discrete local view cell, for the long sequence of familiar views, the familiar views are successively recalled in correct order (Rolls, 2017), and the firing sequences of the grid cells and the HD cells are retrieved over time in the neural space. The retrieval of firing states from experiences brings mismatch as compared to the current firing states of the grid cells and the HD cells.

To minimize the mismatch in the firing activities, a robust constrained non-linear least squares approach is employed to calibrate the map representation by optimizing the consistency of the experiences in the topological map (Agarwal and Mierle, 2012; Zeng and Si, 2021). The consistency of the map is optimized by finding a solution to the following problem using the Ceres solver (Agarwal and Mierle, 2012)

$${\min }_{\mathbf{e}}\frac{1}{2}\sum _{i,j}{\rho }_i\left(\left|\right|{\left(f_i\left(e_i,e_j,e_{ij}\right)\left|\right|\right)}^2\right),$$ (9)

where e is the set of nodes. Each node e_i = (x_i, y_i, θ_i) contains the spatial phase of the grid cell network and the HD phase of the HD network. e_ij = (x_ij, y_ij, θ_ij) describes the constraint from e_i to e_j. ${\rho }_i\left({\left(\left|\right|f_i\left(e_i,e_j,e_{ij}\right)\left|\right|\right)}^2\right)$ is a residual block, where f_i (⋅) is a cost function and ‖ ⋅‖ is the L₂ norm. ρ_i is a loss function, and here Huber Loss is used due to its insensitive to the influence of outliers in the optimization of the topological map. More specifically, cost function f_i (⋅) for a pair of vertices e_i and e_j connected by an edge e_ij is computed by

$$\begin{array}{lll}{f}_i\left(e_i,e_j,e_{ij}\right)& =& \left[\begin{array}{c}{e}_j-e_i-e_{ij}\end{array}\right]=\left[\begin{array}{c}{x}_j-x_i-x_{ij}\\ y_j-y_i-y_{ij}\\ {\theta }_j-{\theta }_i-{\theta }_{ij}\end{array}\right]\\ & =& \left[\begin{array}{c}{x}_j-x_i-d_{ij}\cos ({\theta }_i+{\theta }_{ij}+{\theta }_{relative})\\ y_j-y_i-d_{ij}\sin ({\theta }_i+{\theta }_{ij}+{\theta }_{relative})\\ {\theta }_j-{\theta }_i-{\theta }_{ij}\end{array}\right],\end{array}$$ (10)

$$\begin{array}{lll}\text{s.t.}& & 0\le {\theta }_i<2\pi ,\\ & & 0\le {\theta }_j<2\pi ,\end{array}$$ (11)

where d_ij is the distance between e_i and e_j. θ_relative is the relative angle between matched visual templates and current visual scenes (Ball et al., 2013). θ_i and θ_j are in the range [0, 2π).

2.3. Firing activity correction by experience

The firing activities of the grid cells and the HD cells are corrected by the change of the topological map during the map optimization. This is achieved by mapping the changes in the physical space back to the neural manifold.

2.3.1. Head direction cells

In the ring attractor network of HD cells, as the neural manifold of head directions has the same scale as the directional space in the physical environment, the corrected HD phase can be directly calculated from head direction changes during map optimization in the physical space. The corrected HD phase is given by

$${\mu }_{\theta }={\mu }_{\theta }^{\prime }+\mathrm{mod}(\theta -{\theta }^{\prime },2\pi ),$$ (11)

where θ′ and θ are the head directions in the physical space before and after optimization, respectively. ${\mu }_{\theta }^{\prime }$ is the HD phase before correction. μ_θ is the HD phase after correction.

2.3.2. Grid cells

In the torus attractor network of grid cells, the wrapping of network edges allows the representation of infinite large environments by periodic firing activities. Usually, the neural space of the grid cell network has a different coordinate system from the reference frame in the physical environment. The spatial wavelength of grid fields expressed by grid cells is determined by the gain of the velocity input (McNaughton et al., 2006). The position changes before and after optimization in the physical environment are used to update the grid cells network in the neural space. The corrected spatial phase of grid cells is given by

$$\begin{array}{cc}\left[\begin{array}{c}{\mu }_x\\ {\mu }_y\end{array}\right]=& \left[\begin{array}{cc}\cos (\phi )& -\sin (\phi )\\ \sin (\phi )& \cos (\phi )\end{array}\right]\left[\begin{array}{c}{\mu }_x^{\prime }\\ {\mu }_y^{\prime }\end{array}\right]\\ & +\mathrm{mod}(\left[\begin{array}{c}x-x^{\prime }\\ y-y^{\prime }\end{array}\right]v_{scale},2\pi ),\end{array}$$ (12)

where (x′, y′) and (x, y) are the positions of the animal before and after optimization, respectively. v_scale is the gain of velocity input. φ is the angle of the rotation transformation from the reference frame in the environment to the coordinate system of the neural space. $({\mu }_x^{\prime },{\mu }_y^{\prime })$ is the phase of the grid cell network before correction. (μ_x, μ_y) is the corrected phase of grid cells. This equation also shows the connection between physical distance and grid phase distance.

2.4. Implementation of SLAM system

We implemented a SLAM system based on the proposed model in Robot Operating System (ROS) Indigo on Ubuntu 14.04 LTS (Trusty) using C++ language. The software architecture of the SLAM system is organized into four nodes as shown in Fig. 2. We reuse the local view matching algorithm in OpenRatSLAM system (Ball et al., 2013).

Fig. 2.

The software architecture of the SLAM system. Images and odometry information are provided by the sensor/bagfile node. Whether the current view is familiar or not is determined by the local view cells node. The grid cell model and the HD cell model are implemented in the Bayesian attractor network node, which performs path integration and makes decisions of loop closures. The experience map node achieves the global graph optimization of the experience map.

Visual images and odometry information are provided to our SLAM system as inputs by the sensor/bagfile node. The local view cell node compares the current image with view templates to determine whether the current view is familiar or not. If a familiar view is detected, the local view cell node injects calibration currents to the Bayesian attractor network node. The Bayesian attractor network node integrates the movement and sensory information by simulating the responses of grid cell network and the HD cell network. This node also makes decisions about the creation of new nodes and links in the experience map. The experience map node builds the topological map and optimizes this map by graph-based non-linear least-squares approaches.

2.5. Robot experiment setup

We tested our method on a publicly available open-source dataset, iRat Australia dataset (Ball et al., 2013). The data was collected by a miniature mobile robot, called Intelligent Rat animat technology (iRat), a tool to investigate spatial navigation and cognition for interdisciplinary robotics and neuroscience studies. iRat is similar to a rat in size and shape. The dataset was obtained while the iRat robot explored a maze of Australian geography that contains prominent Australian landmarks. The views from the webcam on the iRat robot were captured into a sequence of images (Fig. 3c). The robot explored for approximately sixteen minutes, and traversed the paths of the maze several times. The proposed model gets the image sequence as visual inputs, localizes the robot in the maze, and constructs a map of the maze.

Fig. 3.

Screenshots of SLAM system for iRat Australia dataset. (A) Neural activities of HD cells; (B) Overhead image; (C) Input visual scene (top); the local view template and the matched template (bottom); (D) Neural activities of grid cells; (E) Experience map.

3. Results

To investigate the role of the hippocampal feedback in the formation of globally coherent spatial representations in MEC, we propose a navigation model in which the changes of the hippocampal representation during map learning is mapped back to the path integration system in MEC. In the path integration system, grid cells and HD cells are modeled by Bayesian attractor networks which integrate multisensory motion and vision cues by Bayesian inference. The cells in the grid cell network and HD network form attractor states to represent the positions and head directions of the animal. In the hippocampal cognitive map system, a topological map is constructed capturing the attractor states of the path integration system as nodes and the transitions between attractor states as links. When the animal revisits a familiar place, the detection of the familiar scene by view cells allows the establishment of a link to a previous node in the map and thus the minimization of the odometry errors accumulated along the loop just explored. The change of the topological map is mapped back to the path integration system in MEC and corrects the activities of the grid cells and HD cells therein.

3.1. Disruption of consistent map without map calibration

On loop closure, the topological map is calibrated to reduce the odometry error accumulated during exploration (ref. Section 2.2). However, if the map calibration is switched off, the constructed topological map will not be consistent with the structure of the environment (Fig. 4A vs. B). Without map calibration, the nodes in the map are not correctly registered relative to each other. The map, therefore, expands in space since the spatial relationship between the nodes could not be bounded in time. To give a quantitative comparison, we calculate the size of the experience maps by counting the number of pixels occupied by the maps. Without loop closure, the experience map occupied 19.72 square meters. As a comparison, the map learned with map calibration was of 5.23 square meters. Without map calibration, the map was 3.77 times as the size of the map with calibration during loop closures. Although the topological map is not consistent, the firing fields of HD cells are globally coherent (Fig. 4A top). The grid cells show coherent square grid maps, with firing fields regularly spaced (Fig. 4A bottom). Without map calibration, the topological map is not stable due to the missing establishment of spatial relationship from the experience. Although grid cells and HD cells are able to maintain attractor states, the attractor states are subject to odometry errors in new experience until the correction from local view cells during loop closures. Map calibration therefore is critical in the formation of map representations that match the topology of the environment.

3.2. Firing activity correction leads to globally coherent HD maps and grid maps

To investigate the effect of firing activity correction (ref. Section 2.3), we compare the firing maps of HD cells and grid cells with the maps formed when the firing activity correction is switched off with intact map calibration (Fig. 4B vs. C).

Without firing activity correction, HD cells are selective to different orientations at different parts of the environment. As shown in the top panel of Fig. 4B, the single firing field is continuously expressed in one direction until the trajectory is oriented towards different directions. However, at the global level, the preferred firing directions of HD cells drift, showing different firing directions in different areas (top panel in Fig. 4B, central region vs. northeast and southwest regions). As a result, the peak firing activities of HD cells scatter in a wide distribution (Fig. 5A top). During the whole exploration, HD cells do not show selectivity to particular directions (Fig. 5A bottom). Within short time windows, however, the peak firing activities are concentrated in a few clusters, demonstrating that the anchoring of HD cell activity is only locally stable instead of globally coherent (Fig. 5B).

Fig. 5.

HD cells show globally coherent directional selectivity with firing activity correction. (A) Without firing activity correction, the distributions of the HD cell activity are not localized. Two cells with opposite preferred directions are shown in columns. Top row depicts firing rate as a function of HD. Bottom row plots firing rate histograms. (B) Within each quarter of the exploration time, The activity of the HD cell at 0 concentrates on multiple clusters. Each panel shows one fourth of the total firing activity. Red color encodes peak activity and blue encodes zero activity. Some clusters can be fitted by Gaussian functions (red solid lines). (C) With firing activity correction, the distributions of the HD cell activity follow bell-shaped distributions during the whole exploration (top). The firing rate histograms in polar coordinates reveal strong directional selectivity (bottom). (D) Within each quarter of the exploration time, the HD cell at 0 keeps its preferred direction stable. (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article.)

The local anchoring of the firing pattern can also be observed in the firing maps of grid cells when firing activity correction is inactivated (Fig. 4B bottom). In local areas, grid cells show multiple firing fields that are arranged with respect to each other in a rectangular grid, a structure determined by the periodic boundary conditions of the torus manifold of grid cells. In each field of the grid map, the firing rate always increases first and then decreases gradually. On a global scale, grid fields are scattered in the experience map, and do not align into a coherent grid. The local anchoring of grid patterns is determined by local sensory cues.

As a contrast, HD cells and grid cells develop globally coherent firing maps with the help of firing activity correction during map calibration (Section 2.3). HD cells that are selective to east or west fire along all trajectories that are oriented in horizontal directions, thus showing globally coherent HD tuning (Fig. 4C top). Grid cells maintain periodic square grid across the whole environment. Each firing field is spaced by about 0.5 m, the grid-scale determined by the gain of velocity input (Fig. 4C bottom). Note that the spacing of grid patterns is preserved during firing activity correction (Fig. 4C bottom). The firing activity of HD cells follows bell-shaped distributions, showing strong directional selectivity. The difference between the distribution centers matches the difference between the preferred directions of the cells (Fig. 5C). The firing rate maps do not change during exploration, therefore with firing activity correction HD cells maintain their directional selectivity stably (Fig. 5D).

To quantify the degree of global coherence of grid maps in space, we follow the method in Sargolini et al. (2006) and define rectangular gridness score to measure the four-fold symmetry in the autocorrelograms of the grid maps. Rate maps are first binned into two-dimensional matrixes (Fig. 6A), and then shifted with respect to themselves to compute the autocorrelograms in 2D (Fig. 6B). The total correlation value along the radial directions in a ring region covering the peaks close to the center are computed (Fig. 6C). The 90-degree periodicity in the radial correlation value is defined as the rectangular gridness score. Without map calibration, the grid firing map has high gridness score (Fig. 6 top). This is determined by the attractor dynamics of the grid cell network. Without activity correction, the gridness score of firing map is low, demonstrating the loss of periodic arrangement of grid fields (Fig. 6 middle). With activity correction, the grid map shows high gridness score, demonstrating global coherence in firing map (Fig. 6 bottom). To quantify the degree of global coherence of grid maps in time, we split the trajectory into two halves and compute the crosscorrelograms between the firing maps of the first half and the second half (Fig. 7). The central peak of the crosscorrelogram is evident and strong only with firing activity correction, manifesting stable and global coherence of grid maps. The process of firing activity correction, therefore, shoves the state of grid cells and HD cells in the attractor network to match the topology of the environment.

Fig. 6.

Globally coherent grid maps have high degree of symmetry. (A) Firing rate maps; (B) Autocorrelograms of the firing maps; (C) Rectangular gridness score estimation. Each row shows the computation of gridness score for grid maps without map calibration (top), without activity correction (middle) and with activity correction (bottom).

Fig. 7.

With firing activity correction, grid cells maintain globally coherent grid maps. Each row shows one example grid cell for the three conditions respectively. Left column: firing rate maps of the example grid cells during the first half of exploration. Middle column: firing rate maps of the example grid cells shown in the left during the second half of exploration. Right column: the crosscorrelograms between the firing rate maps of the first half and the second half of each cell. With firing activity correction (bottom row), the crosscorrelogram of the example grid cell has peek in the center and high degree of spatial symmetry, confirming stable grid codes over time. Without loop closure or firing activity correction (two rows on the top), the central peaks of the crosscorrelograms are weak, demonstating the loss of stable or globally coherent grid patterns.

Map calibration could be a possible computation performed by the replay of the hippocampal activity (Gupta et al., 2010; Roux et al., 2017). During replay, the activity of place cells is rehearsed in a faster time scale, and the structural relationship between the place cells could be reinforced to reduce the mismatch accumulated during long exploration. Firing activity correction could be achieved by the feedback connections from CA1 of the hippocampus to the layer V of MEC, where grid cells and HD cells coexist (Witter et al., 2017; Rozov et al., 2020). With this feedback input, the attractor states of the MEC network could be modified (Agmon and Burak, 2020). Firing activity correction constitutes a mechanism to eliminate incoherence of spatial representations in grid cells and HD cells. It requires reorganizing the representation of past experience. Therefore, the formation of globally coherent firing maps of grid cells entails the traverse of the whole environment, as observed experimentally (Carpenter et al., 2015).

3.3. Development of grid maps

Grid cells in the network express firing fields quickly. In each firing field, the activity of the cell increases first and then decreases while the animal enters and leaves the field (Fig. 8). After loop closure, large localization errors are abased by the mechanism of firing activity correction, and periodic grid structure is evident in the firing map (Fig. 8A). It takes time to traverse the whole environment, but for each interval of one-quarter of the dataset, the coherence in grid maps is maintained globally across the whole environment (Fig. 8A–D). The final experience map (Fig. 8D) captured the spatial layout of the maze (Fig. 3B), demonstrating the correct representation of the spatial relationship between the locations in the environment.

Fig. 8.

Development of the firing map of a grid cell.. (A), (B), (C) and (D) show the firing map of one example grid cell for each interval of one quarter of the exploration. During exploration, the grid map develops and is maintained coherent globally by correcting localization through the feedback from cognitive map.

3.4. Coverage of space with globally coherent firing maps

The HD cells in the HD cell network fire in different directions depending on their preferred HDs (Fig. 9A). The firing fields of each HD cell are coherently laid down along the same direction in the environment, demonstrating globally coherent HD tuning in the environment. The differences between the firing directions of the HD cells match the relationship between the preferred directions of the cells. The example HD cells, shown from top to bottom in Fig. 9A, express firing fields orienting towards east, north, west, and south respectively. The increasing firing directions in steps of 90° are in accordance with the increasing coordinates, i.e. 0, $\frac{\pi }{2}$, π, and $\frac{3\pi }{2}$, of the cells in the ring manifold.

Fig. 9.

Global coherent representations with various phases. (A) Global coherent HD representations. The firing maps of four example HD cells are shown. The cells are at 0, $\frac{\pi }{2}$, π, and $\frac{3\pi }{2}$ in the ring manifold from top to bottom. (B) Global coherent grid representations. Each panel shows the firing map of one example grid cells. From top to bottom, the grid cells are at (0, 0), $(\frac{\pi }{2},\frac{\pi }{2})$, $(\frac{3\pi }{2},\pi )$, and $(2\pi ,\frac{3\pi }{2})$ in the manifold.

We also select four different grid cells in the torus manifold, i.e. (0, 0), $(\frac{\pi }{2},\frac{\pi }{2})$, $(\frac{3\pi }{2},\pi )$, and $(2\pi ,\frac{3\pi }{2})$, and show their firing maps in Fig. 9B. Grid cells in the network have different firing locations, forming a full coverage of the environment. The firing maps of the example grid cells share the same spacing and orientation, since they are in the same module. The grid cells in the model encode the environment with various phases, which is consistent with experimental results (Hafting et al., 2005; Stensola et al., 2012).

3.5. Multiscale representation of space with globally coherent grid maps

In addition to various spatial phases (Fig. 9), grid cells in the model show different grid spacings if the gain of velocity inputs are chosen differently (ρ in Eq. (8)). The firing rate maps of three example grid cells from different modules are shown in Fig. 10. As the gain parameters ρ decreases, the spacings of the grid maps increase from 0.5m to 1.5m, covering the range of typical grid-scale observed in experiments (Brun et al., 2008; Stensola et al., 2012).

Fig. 10.

Global coherent grid maps with various spacings. (A), (B), and (C) show firing maps of three grid cells from three grid modules, with grid spacing 0.5m, 1.0m, and 1.5m, respectively.

4. Discussion

We proposed a navigation model in which grid cells and HD cells form globally coherent representations of space. The firing maps of grid cells and HD cells are easily anchored to visual features in the local space of the environment on the first visit. During the revisit of a place, the familiar visual features provide error correction inputs to calibrate the activity states of grid cells and HD cells by Bayesian integration through loop closures in the cognitive map. A topological map, containing the phases of network activities as nodes and the transition between phases as links, is constructed and calibrated by minimizing the odometry errors. Globally coherent spatial firing maps require the correction of activity state by matching the changes in map calibration. The globally coherent spatial codes serve as an accurate spatial metric for navigation. We tested the model on a rat-like miniature robot exploring a naturalistic environment (iRat 2011 Australia dataset) (Ball et al., 2013). Our results demonstrated that to form universal metric representations in complex environments, grid cells and HD cells require revisits of explored places. While revisiting the explored places, visual cues could minimize odometry errors of mental cognitive map and the network states of the model could be corrected to form globally coherent firing maps.

Due to de design of the maze, the movement of the robot was restricted in paths that connect the landmarks. Within the restricted paths explored by the robot, grid cells in the model show regular grid patterns. The setup of the robot experiment mimics the initial exploration patterns of mice in environments near their ecological habitants. In free exploration, mice often repeat familiar paths to the frontiers of exploration and start to acquaint unknow territory (Dong et al., 2021). After sufficient exploration, the trajectories of the agent cover the whole environment. With the mechanisms of map calibration and firing activity correction, grid cells and HD cells in the model are still able to anchor attractor states to external landmarks resulting in global coherent firing maps.

4.1. Related works

The computational mechanism underlying spatial navigation is an intriguing research topic both in neuroscience and machine intelligence (Widloski and Fiete, 2014). Kalman filters are widely used in artificial agents to estimate the positions of the agent and the landmarks it observed in the environment (Dissanayake et al., 2001; Aulinas et al., 2008). By assuming a joint Gaussian distribution of the positions, Kalman filter recursively update the mean of the distribution as the state of the system, as well as the covariance matrix of the state (Huang and Dissanayake, 2007). Our model is a neural approximation of Kalman filter, with the essential components of sate transition, uncertainty representation and error correction. Different from the Kalman filter framework, our model decouples the state of the agent and the state of the landmark. The state of the agent is tracked in MEC by grid cells and HD cells. The landmarks are processed in visual areas by local view cells. The relationship between the locations is updated in the hippocampus resulting in a topological map. Our model focuses on cognitive map learning in spatial navigation tasks. Recently, a more general model, the Tolman-Eichenbaum Machine, was proposed to learn relational maps both in spatial domains and in abstract conceptual spaces (Whittington et al., 2020). In this model, the computational mechanisms of the hippocampal-entorhinal system could efficiently extract the structural knowledge across environments and bind with environment-specific sensory information. The hippocampal-entorhinal system therefore supports the understanding of the environment by learning the transitional relationship between entities therein.

4.2. Hippocampal-entorhinal interaction

Vestibular inputs are prone to noise and would result in accumulated errors. Visual and boundary features provide strong anchoring cues for grid cells and HD cells, and would distort their firing maps (Barry et al., 2007; Derdikman et al., 2009; Knight et al., 2012; Krupic et al., 2015; Stensola et al., 2015). Previous models suggest the integration of vestibular cues (McNaughton et al., 2006; Fuhs and Touretzky, 2006; Yoram Burak and Ila R Fiete, 2009; Si et al., 2014) and visual cues (Franzius et al., 2007; Raudies et al., 2012) as an important mechanism of stable grid cell firing activity. In this study, we show that the interplay between the path integration system and the cognitive map system is important in forming globally coherent spatial representations (Fig. 11). The interaction between these two systems affects the dynamics of spatial representations, and could be one critical driving force for the update of the spatial codes (Carpenter et al., 2015; Ziv et al., 2013; Rubin et al., 2015). On the one hand, the firing patterns of grid cells and HD cells are stabilized by sensory cues in the environment. On the other hand, the firing activity of grid cells and HD cells is corrected by the changes of the cognitive map during the revisit of familiar places. The corrected HD and grid firing activities are consistently anchored to external reference frames. These globally coherent patterns provide necessary information for mammals to identify the relative positions of the places in the environment. The experience-dependent interaction is implemented by the recurrent circuit between the hippocampus and entorhinal cortex, and should be viewed as a whole system while we investigate the mechanisms of spatial memory, which is likely to be critical in understanding spatial memory, even episodic memory (Barry et al., 2007).

Fig. 11.

The interaction between the path integration system in the entorhinal cortex and the cognitive map system in the hippocampus.

To pin down the role of the interaction between the cognitive map system and the path integration system, we inactivated the map calibration mechanism (Section 2.2) and the activity correction mechanism (Section 2.3) selectively. If the map calibration was switched off, the activities of grid cells and HD cells drifted, and the cells did not show stable firing fields (Fig. 4). This explains the fact that grid codes require hippocampus (Bonnevie et al., 2013), functioning as associative anchoring codes (Mulas et al., 2016). If the activity correction was inactivated, grid cells and HD cells lost global coherence in their firing maps (Fig. 4), resulting in locally anchored representations. In our model, globally coherent firing maps develop very quickly (Fig. 8) as compared with the gradual and continuous build-up process found in rats (Carpenter et al., 2015). This might due to the fact that the environment in our experiments is composed of restricted paths on a two-dimensional plane. In open field environments, it would take a longer time for the model to form globally coherent firing maps.

4.3. Neural correlates of spatial learning

Our model is consistent with recent anatomical and neurophysiological findings. Hippocampus sends feedback projections to layer V of MEC, where grid cells and HD cells are abundant (Witter et al., 2017; Rozov et al., 2020). Hippocampal activity such as from spatial view cells in the CA3 region of the hippocampus (Rolls, 2017) may be relayed by this projection to MEC, and functions as activity correction signals for grid cells and HD cells. The replay of the hippocampal activity during exploration could support map calibration (Gupta et al., 2010).

In our model, map calibration happens when the animal detects a loop closure. Map calibration is naively modeled by optimizing the consistency of the spatial relationships between the nodes in the topological map. It could be implemented by more biological models (Evans and Burgess, 2019; Fukushima et al., 2021). Animals often explore environments with intermittent locomotion, which allows them to investigate environment features and possibly mental adjustment of internal representations in an offline manner (Monaco et al., 2014). Map calibration may correspond to SPW-Rs of the hippocampus when the animal is in awake rest. Experimental studies have shown that in spatial learning tasks longer-duration waking SPW-Rs occur more frequently in novel environments than in familiar environments (Fernández-Ruiz et al., 2019). In addition, artificial prolongation of ripples improves memory performance. It is very likely that waking SPW-R is an important mechanism to rehearse past experiences in order to reduce error in map learning and facilitate action planning in reinforcement learning. The activity correction mechanism in the model might correspond to the propagation of SPW-Rs to MEC. There are accumulating evidences showing that deep layers of MEC show coordinated oscillations triggered by hippocampal SPW-Rs (Chrobak and Buzsáki, 1996; Staresina et al., 2019). This feedback oscillatory inputs may play functional role in changing the population activities in MEC.

The cognitive map in the model is based on graph representation for simplicity. To better model the learning of cognitive map, neural networks of place cells should be adopted (Wagatsuma and Yamaguchi, 2007; Zhao et al., 2020, 2021). Place cells integrate spatial inputs from multiple grid cell modules as well as contextual inputs from lateral entorhinal cortex. Through the feedback connections from the hippocampus to the entorhinal cortex, place codes in the hippocampus provide modulatory inputs to shift the attractor states of grid cells gradually and continuously, so that globally coherent representations of the environment is reached.

The proposed model only considers path integration system in the MEC. Besides the inputs from MEC, the hippocampus also receives substantial inputs from subcortical structures, such as nucleus reuniens and anterior claustrum. These inputs contribute to the formation of cognitive map in hippocampus. In the maze composed of parallel compartments where salient local visual cues are missing, hippocampal cells form similar place fields in each of the parallel compartments due to the reliable inputs from boundary-encoding cells in the nucleus reuniens and anterior claustrum (Grieves et al., 2016; Harland et al., 2017). If the HD cell network is disrupted, MEC may not provide stable attractor states for hippocampus, and leads the hippocampal place cells to repeat stereotypical place fields in each compartment, even when the compartments are arranged radially (Harland et al., 2017).

5. Conclusion

The ability of determining the distance between places in any environment is critical for navigation. Grid cells in the entorhinal cortex fire at multiple locations that are regularly spaced on triangular grids in open fields. The firing maps of grid cells are anchored in environments coherently, keeping the relative spatial relationship of each grid map fixed. The population activity of grid cells may provide a universal metric for spatial navigation in large-scale space. Recent experimental results show that grid maps in complex environments are anchored to local space before they are transformed to globally coherent firing maps with long exploration. In this work, we propose a navigation model to underscore the mechanisms underlying the transformation from locally anchored maps to globally coherent representations. Our model suggests that the experience-dependent interactions between the entorhinal cortex and the hippocampus play a critical role in setting up the globally coherent firing maps in the entorhinal cortex. Grid cells indeed are able to provide a universal spatial metric for mammalian spatial navigation in complex environments, after familiarization of the environment.

CRediT authorship contribution statement

Taiping Zeng: Conceptualization, Methodology, Software, Writing – original draft. Bailu Si: Conceptualization, Methodology, Software, Writing – original draft. Xiaoli Li: Methodology, Writing – original draft.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following is the Supplementary data to this article: