Generalization of cognitive maps across space and time

Abstract

Prominent theories posit that associative memory structures, known as cognitive maps, support flexible generalization of knowledge across cognitive domains. Here, we evince a representational account of cognitive map flexibility by quantifying how spatial knowledge formed one day was used predictively in a temporal sequence task 24 hours later, biasing both behavior and neural response. Participants learned novel object locations in distinct virtual environments. After learning, hippocampus and ventromedial prefrontal cortex (vmPFC) represented a cognitive map, wherein neural patterns became more similar for same-environment objects and more discriminable for different-environment objects. Twenty-four hours later, participants rated their preference for objects from spatial learning; objects were presented in sequential triplets from either the same or different environments. We found that preference response times were slower when participants transitioned between same- and different-environment triplets. Furthermore, hippocampal spatial map coherence tracked behavioral slowing at the implicit sequence transitions. At transitions, predictive reinstatement of virtual environments decreased in anterior parahippocampal cortex. In the absence of such predictive reinstatement after sequence transitions, hippocampus and vmPFC responses increased, accompanied by hippocampal-vmPFC functional decoupling that predicted individuals’ behavioral slowing after a transition. Collectively, these findings reveal how expectations derived from spatial experience generalize to support temporal prediction.

 

Recent work has shown that through experience knowledge becomes structured into rich internal models known as cognitive maps (Tolman 1948; Schiller et al. 2015; Epstein et al. 2017; Peer et al. 2021). For instance, by navigating our world, we form representations not only of where individual items are located within our spatial environment (O’Keefe and Nadel 1979; Johnson and Redish 2007; Karlsson and Frank 2009), but we also derive knowledge about the spatial relationships among them (Brown et al. 2016; Deuker et al. 2016). Cognitive maps therefore go beyond our direct experience to build holistic representations of our spatial world (Behrens et al. 2018). However, various scales of representation have come to be encompassed in the term cognitive map. While the classic theories of cognitive maps assume these representations comprise a Euclidean coordinate system that encodes accurate distances and directions between locations in space (O’Keefe and Nadel, 1978), in some instances, cognitive maps may in fact code representational distortions to infer new knowledge based on task goals (Mack et al. 2016; Boccara et al. 2019; Butler et al. 2019; Morton et al. 2020). Here, we examine cognitive maps that may augment representation of spatial similarities and differences, by expanding or compressing perceived distances, which may influence how knowledge comes to bias behavior in new settings.

For example, on a visit to Austin, you may go out to several bars in East Austin, visit museums located north of downtown, and enjoy outdoor activities along Lady Bird Lake. As you form a cognitive map representing individual locations across Austin, it may be adaptive to emphasize the similarities among and differences between the people, places, and objects experienced in distinct neighborhoods to extract the unique character of each neighborhood. As evidence for such representational distortions, when individuals are asked to draw maps of familiar environments, the resulting sketches do not align with accurately scaled maps (Kuipers 1982; Jafarpour and Spiers 2017). Moreover, participants tend to underestimate travel time for familiar routes relative to their objective length, suggesting representations of locations along the route are compressed in memory (Jafarpour and Spiers 2017). In contrast, individuals’ subjective perception of time is expanded when traversing trajectories that cross spatial boundaries (Brunec et al. 2018, 2020). Collectively, these behavioral findings suggest that cognitive maps learned through spatial experience are schematized to reflect abstract, goal-relevant properties of the environment (Hirshhorn et al. 2012).

Thus, cognitive maps may be more cognitive than spatial to support prediction in a wide variety of settings, including those beyond the original learning environment. Within such schematized knowledge structures, spatial context becomes the organizing principle by which events experienced in each neighborhood, for example, become categorized. Recent evidence suggests that distortions within cognitive maps represent abstract knowledge that extends beyond direct experience to support enhanced prediction and inference (Baraduc et al. 2019; Mack et al. 2020; Morton et al. 2020; Park et al. 2020). For example, by augmenting differences within your cognitive map of Austin, boundaries may be established between neighborhoods that emphasize the abstract character of each (i.e. East Austin has live music; Lady Bird Lake is great for outdoor activities). These boundaries might also influence how knowledge subsequently biases behavior (Radvansky and Zacks 2017; Peer and Epstein 2021). When reflecting on your Austin trip, recall of particular events may be slower when mentally traversing across neighborhood boundaries because representational distortions may code two locations that cross a boundary as farther apart than their objective distance from one another. Moreover, cognitive map distortions may aid future decisions that rely on inference; if your goal is to hear live music on a subsequent visit to Austin, you might head to East Austin to find a new bar because you previously encoded East Austin as an entertainment district.

While several studies have revealed neural mechanisms supporting spatial cognitive map formation (Brown et al. 2016; Deuker et al. 2016), including representational distortions (Boccara et al. 2019; Butler et al. 2019), fewer studies have examined how distorted cognitive maps influence prediction in new contexts beyond the original learning environment. The flexibility to generalize knowledge across different domains of experience, such as space and time, is a function that has been theoretically proposed for decades (Eichenbaum et al. 1994; Eichenbaum 1997), yet it has little empirical support at the level of neural representation. Here, our goal was to quantify how cognitive maps formed via spatial experience generalize to support prediction and decision making during temporal sequences of objects. We predicted that exaggerated representation of the shared and distinct spatial properties of objects (i.e. their locations within virtual environments) would guide how individuals subsequently organized continuous object sequences into discrete “chunks” (Baldassano et al. 2017; Radvansky and Zacks 2017; Clewett et al. 2019). Prior work has demonstrated that behavior, specifically a reduction in response time measures across triplets of stimuli, was sensitive to shared learned statistical regularities (Hunt and Aslin 2001; Turk-Browne et al. 2005, 2010). Here, we tested how representation of spatial commonalities prime expectations during temporal processing as well as how representation of spatial differences may elicit behavioral slowing when participants “cross” a spatial boundary during continuous object sequences.

Interestingly, the brain regions that have been implicated in segmenting continuous experience into discrete episodes (Schapiro et al. 2012, 2013; Ezzyat and Davachi 2014; Baldassano et al. 2018) are highly overlapping with regions implicated in cognitive map formation, including hippocampus, parahippocampal cortex (PHC), ventromedial prefrontal cortex (vmPFC), and medial parietal cortex (Tavares et al. 2015; Constantinescu et al. 2016; Deuker et al. 2016; Aronov et al. 2017; Knudsen and Wallis 2021). Thus, one of our theoretical goals was to connect theories derived from the spatial navigation, cognitive map, and event segmentation literatures to demonstrate how hippocampus, PHC, vmPFC, and medial parietal cortex contribute to processing of both spatial and temporal features of our environment. While these literatures have parallels (Brunec et al. 2018), fewer studies have simultaneously examined the influence of knowledge across domains.

Prior work has shown that hippocampus and vmPFC form associative structures that represent the relationships between stimuli (Schlichting and Preston 2015; Morton et al. 2017; Varga et al. 2021), wherein vmPFC interacts with hippocampus to support additional abstraction that emphasizes goal-related similarities and differences (Mack et al. 2016, 2020). Hippocampal-vmPFC interactions further support knowledge expression during decision making in rodents and non-human primates (Navawongse and Eichenbaum 2013; Place et al. 2016; Wikenheiser et al. 2017; Yu et al. 2018; Park et al. 2021; Das and Menon 2022). For example, hippocampus transmits contextual information to vmPFC when animals re-enter a previously experienced environment (Place et al. 2016). Ventromedial PFC then uses that contextual information to bias reactivation of memories toward actions associated with rewarded outcomes through reciprocal interaction with hippocampus, driving optimal choices (Chan et al. 2016; Place et al. 2016; Schuck et al. 2016; Zhou et al. 2019). The central question in the present study extends this prior work to ask how hippocampus and vmPFC representations and interactions guide expression of spatial knowledge beyond the original task setting.

We also examined how PHC contextual representations, and anterior PHC representations in particular, support expression of spatial knowledge in a temporal sequence task. Together with hippocampus, PHC plays an important role in representing spatial experience (Mullally and Maguire 2011; Spiers and Barry 2015; Baumann and Mattingley 2016; Julian et al. 2018). PHC not only responds more so to spatial environments than other categorical stimuli (i.e. faces and objects) (Epstein and Kanwisher 1998; Polyn et al. 2005), but anterior PHC also shows enhanced response to objects that have strong associations with particular spatial locations (e.g. blenders are strongly associated with kitchens) (Bar and Aminoff 2003; Aminoff et al. 2007; Bar et al. 2008). More recent evidence indicates that PHC represents hierarchical cognitive maps that group items that share a context together in memory, while simultaneously distinguishing those from different contexts (Morton et al. 2020). Such adaptive representational distortions in PHC coded similarities and differences that went beyond direct observation and further supported knowledge expression during inference. Together these data suggest that when entering a context with familiar elements, predictive inference supported by anterior PHC might elicit expectations for what you expect to experience in that context. Here, we test this idea by examining how reactivation of anterior PHC context representations formed during spatial learning speeded decision making in a temporal sequence task.

Finally, we also considered how medial parietal cortex may be involved in cognitive map formation and expression across spatial and temporal domains. Medial parietal cortex function, including retrosplenial cortex (RSC), is important for spatial coding (Vann et al. 2009; Sherrill et al. 2013; Auger et al. 2015; Julian et al. 2018). However, there is a growing appreciation that the RSC plays a broader role in memory processing, through schematic abstraction (Marchette et al. 2014; Baldassano et al. 2018; Peer and Epstein 2021; Pudhiyidath et al. 2021). Both spatial and temporal associations are represented in the RSC (Aminoff et al. 2007; Bar et al. 2008; Pudhiyidath et al. 2021). Recent evidence further indicates that associative coding in RSC is in the form of context-dependent schemas that prime expectations for what actions are relevant in a given context (e.g. the sequence of actions one might expect to take when traveling by air) (Baldassano et al. 2016, 2018). Moreover, RSC compresses or expands object representations based on their temporal similarities and differences, which further biases novel inference decisions (Pudhiyidath et al. 2021). Here, we hypothesize that similar distortions in RSC based on spatial experience may guide how individuals process information in a subsequent temporal task.

We measured cognitive map formation as participants performed a spatial learning task in naturalistic virtual environments. Critically, we also scanned participants 24 hours later to assess how those cognitive maps bias temporal processing. We predicted that structured representations of spatial experience acquired on one day would persist in the long-term to impact processing on the next day. In the spatial learning task, participants learned the locations of novel objects in distinct virtual environments during active navigation (Fig. 1A and B). Before and after learning, participants viewed the novel objects in isolation during fMRI scanning (Fig. 1C). If a distinct cognitive map is formed for each environment, neural representations for objects should simultaneously: (i) become more similar when experienced in the same spatial environment and (ii) be neurally distinct from objects experienced in different environments (Fig. 2A).

Fig. 1.

Spatial learning task and behavior. A) Participants performed object location learning in four virtual environments. Related to Fig. S1A. B) An aerial perspective of the forest environment. Within each environment, objects were located in the four corners of the rectangular arena space but were not visible during navigation; schematic of object locations within the forest environment are shown here (not to scale). Object location assignments to environment and location within an environment were randomized across participants. Related to Fig. S1B. C) Participants viewed the novel objects in isolation both prior to (pre-learning) and following (post-learning) the spatial learning task (learning). During learning, participants were spawned within a virtual environment (learning, top left) then cued with the target object for that trial (learning, top right). Participants then actively navigated the environment (learning, bottom left) to the cued object’s location. When the participant reached a location within a virtual radius of the object’s exact location, the object appeared (learning, bottom right) as feedback during learning trials. Twenty-four hours after the spatial learning task (dashed line), participants completed an incidental sequence task and environment localizer task. In the incidental sequence task, still images of objects from the spatial learning task were presented in sequential triplets based on whether they were located in the same (light purple; same env.) or different environments (indigo; different env.) in the spatial learning task. During an environment localizer task, participants viewed still images of each of the four virtual environments. D) Path efficiency increased (decreased slope) and time (seconds) to object locations decreased across the six learning repetitions and test trials. Error bars indicate SEM.

Fig. 2.

Schematic depicting the analysis strategy for isolating hierarchical cognitive maps. A) Visual depiction of hypothesized representational similarity (RS) matrices reflecting the pairwise comparisons between object representations from the post- and pre-learning phases as well as the difference in representation across phases. For simplicity of visual presentation, the schematic depicts partial RS matrices, reflecting hypothesized patterns for objects learned in only two of the environments (desert objects, grouped with yellow bars; forest objects, grouped with green bars). For each task phase (post-learning, left panel; pre-learning, center panel), we performed a searchlight analysis within each of our a  priori ROIs, extracting voxel patterns elicited by each individual object. We then calculated the pairwise similarity between each object to construct RS matrices that could be compared between pre- and post-learning (RS change, right panel). The inset bar graphs for each panel reflect the hypothesized average pairwise similarity for same-environment objects (purple) and different-environment objects (indigo). We predicted that after spatial learning object representations would be more similar for same-environment objects and less similar for different-environment objects, reflecting formation of a hierarchical cognitive map of spatial experience. B) Significant clusters within the left anterior hippocampus (HPC; left), bilateral posterior vmPFC (center), and bilateral RSC (right) that represent the predicted hierarchical cognitive maps. C) Regression plots depicting the relationship between cognitive map representation in left HPC (left), bilateral vmPFC (center), and bilateral RSC (right) and decoding of virtual environment reactivation post-learning in PHC. The light gray within the plot indicates the 95% confidence interval. Related to Fig. S2B.

Twenty-four hours after learning, participants were scanned while performing an incidental sequence task in which they made preference judgments for objects from the spatial learning task. Unbeknownst to participants, the objects were presented in sequential triplets comprised of three objects from either the same or different environments (Fig. 1C). This organization of object presentations thus allowed us to quantify how knowledge about spatial boundaries (i.e. a change from one environmental context to another) impacted behavior and neural response. Prior work has shown that while individuals experience events in a continuous sequence, memory for those events is often segmented into discrete episodes (Baldassano et al. 2017; Radvansky and Zacks 2017; Clewett et al. 2019). We predicted that spatial knowledge would be used generatively to determine how individuals organize events in time. We hypothesized that when two items experienced in different spatial contexts were seen in a temporal sequence, augmented representation of spatial differences within the cognitive map may elicit formation of a boundary. Interestingly, the regions that have been implicated in segmenting continuous experience into discrete episodes include hippocampus, vmPFC, and medial parietal cortex (Schapiro et al. 2012, 2013; Ezzyat and Davachi 2014; Baldassano et al. 2018). Hippocampus and vmPFC activation increases at event boundaries is thought to be related to prediction error or uncertainty signaling (Kim et al. 2014, 2017; Schapiro et al. 2016; Baldassano et al. 2018).

At implicit boundaries in our sequence task, we predicted that decisions speeds for the preference task would increase after the transition between objects learned in different environments. Revisiting our real-world example, an analogous prediction would expect that when ranking venues visited during your trip to Austin, your decisions would slow when shifting between venues located in different neighborhoods (i.e. rating live music venues in East Austin to rating museums located near UT Austin). We also hypothesized that such behavioral slowing would be related to the fidelity of neural cognitive maps formed during navigation on the previous day (Fig. 3, see Results section “Cognitive maps support memory-based prediction in new contexts” for analytic approach). In other words, more exaggerated representations of spatial similarities among and differences between objects should result in slower decision times when transitioning from a same-environment to a different-environment triplet. We further predicted that reactivation of contextual predictions in anterior PHC would decrease at the boundaries between same and different environment triplets, resulting in further alterations of hippocampal-vmPFC interactions and response, consistent with increased uncertainty at these time points (Kim et al. 2014, 2017; Schapiro et al. 2016; Baldassano et al. 2018).

Fig. 3.

Schematic of the incidental sequence task and hypothesized neural framework guiding our analyses. A) In the incidental sequence task, objects from the spatial learning task 24 hours prior were presented in isolation. Unbeknownst to participants, sequence order alternated between three successive presentations of objects learned within the same-environment (light purple box) then three object presentations from three different environments (indigo box). The critical analysis period occurs at the implicit boundary between same- and different-environment triplets (as indicated by black box) when predictions derived from the cognitive map were hypothesized to influence both neural response and behavior. B) In the same-environment triplets, we predicted that as the sequence of object presentations progressed, there would be a corresponding buildup in reactivation of the target spatial context. We used a neural classifier trained on data from the localizer task to decode reactivation of the perceptual representation of the context associated with each object (here desert) relative to the other three non-target contexts. C) When transitioning across the implicit triplet boundary (black box) from the last object in a same-environment triplet (SE3) to the first object presentation in a different-environment triplet (DE1), we predicted that reactivation of environmental contexts would be reduced based on the violation of contextual expectations. D) We further predicted that vmPFC (teal) and hippocampal (HPC; orange) activation would increase after a transition to a different-environment triplet, reflecting a prediction error or uncertainty signal corresponding to the reduced reactivation of contextual information. E) Moreover, we predicted that reciprocal connections between vmPFC and HPC would decrease after an implicit boundary, given that contextual signals from hippocampus and medial temporal lobe to vmPFC would be reduced after a boundary. F) Finally, the neural signatures associated with memory-based prediction and uncertainty derived from cognitive maps were predicted to track individuals’ behavioral sensitivity to implicit sequence boundaries, as quantified by slowing of preference response times after a boundary.

Materials and methods

Participants

Thirty-eight right-handed individuals participated in the experiment after giving informed consent in accordance with a protocol approved by the Institutional Review Board at the University of Texas at Austin. Similar to previous neuroimaging studies using immersive open-world environments (Doeller et al. 2010; Sherrill et al. 2013), participants were pre-screened for video game experience and spatial reasoning to increase participant retention. All participants reported having first-person video game experience consistently for at least 5 years within their lifetime, scored less than a 40 on the Santa Barbara Sense of Direction Scale (Hegarty et al. 2002), and had at least a score of 20 on a Questionnaire of Spatial Representation (Pazzaglia and De Beni 2001). Participants received monetary compensation ($25/hour) for their participation in the study.

Data from eight participants were excluded from all reported analyses. Two participants were excluded due to excessive head motion, defined as having more than 15% of time points within a run exceeding 0.5 frame-wise displacement. Six participants were excluded due to poor spatial memory performance, defined as failure to reach above 80% accuracy on the final test of object-location memory. An additional three participants were excluded from the analysis of fMRI data collected on day 2: two participants due to excessive head motion and one participant due to technical difficulties with scanning. After exclusions, data from thirty participants were included in the analysis indexing cognitive map formation during the day 1 session (mean ± SD: 20.867 ± 3.235, 13 females), and data from twenty-seven of those thirty participants (mean ± SD: 20.889 ± 3.412, 11 females) were included in the analysis of the data from the day 2 scanning session.

Stimuli

Stimuli consisted of twenty multicolored 3D-rendered objects created using Blender (Hsu et al. 2014; Schlichting et al. 2015). The novel objects were created to appear physically realistic but distinct from real-world objects. The use of novel objects ensured that participants had no pre-experimental familiarity with the stimuli, allowing us to isolate the impact of spatial experience on the neural representation of the objects.

In addition to the object stimuli, five 3D environments were created using Unity (Unity 5.0.0, Unity Technologies, 2016). One environment was used for a practice phase, and the four remaining environments were used in the spatial learning task (two indoor environments, two outdoor environments). All environments had a similar physical organization but were unique in appearance (Fig. 1A and Fig. S1A). Environments had a centralized, navigable arena (81.74 × 70.84 virtual meters (vm) ± SEM 4.08 × 2.83 vm) with invisible walls to restrict participants’ movement beyond the rectangular arena space. Landmarks unique to each environment were placed along the four sides of the arena space (Fig. S1A). Four of the twenty objects were assigned to the practice arena and were consistently used as practice objects across participants. The remaining sixteen objects were randomly assigned to the learning environments, across participants. Within each environment, the four objects were randomly assigned to one of four possible locations positioned at the corners of the arena (Fig. 1B and Fig. S1B).

Overview of task phases

Participants completed two experimental sessions, separated by 24 hours (Fig. 1C). On the first day, participants were scanned during an initial pre-learning exposure phase to measure brain activation patterns evoked by the sixteen objects prior to spatial learning. After exposure to the individual objects, participants completed the spatial learning task outside of the scanner; participants navigated each individual environment to learn the locations of the four objects assigned to an environment. After spatial learning, participants then viewed the individual objects in isolation again during a scanned post-learning exposure phase to quantify brain activation patterns evoked by the objects as a function of spatial experience.

The participants then returned on the second day and were scanned while viewing the sixteen objects in the context of an incidental sequence task to assess how spatial experience influenced how individuals segmented events in time. After the sequence task, participants then viewed still images of each of the four virtual environments during an environment localizer task. Data from the localizer were used to decode reactivation of the perceptual features of the virtual environment during the pre- and post-learning exposure phases on day 1 and the incidental sequence task on day 2.

Scanned exposure to objects prior to learning

Participants viewed the sixteen objects used in the spatial learning task individually during four scanning runs. Using Psychtoolbox in MATLAB, each object was presented four times within a run (1 second (s) presentation with a 3 s intertrial interval (ITI) which consisted of a black fixation cross), totaling 64 trials. While each object was on the screen, participants completed a change-detection task by determining whether a black plus sign superimposed on the object changed color to either green or blue (Kriegeskorte et al. 2008; Schlichting et al. 2015). Participants were instructed to indicate the color change by button press. Detection accuracy was monitored to ensure that participants were paying attention to the task (see Supplemental Material for performance). Trial order was pseudorandom; objects assigned to the same environment never appeared in succession and were separated by presentation of at least two other objects assigned to different environments. Then, sixteen null trials were randomly distributed within a run and consisted of a black fixation cross (4 s duration). Each of the four runs was 5 minutes (min) and 20 s, resulting in a 21 min and 20 s duration for the pre-exposure phase. Prior to scanning, participants were familiarized with the novel objects in a practice run of the exposure phase to reduce potential neural signatures related to novelty upon first viewing the objects (32 object presentations with eight null trials randomly distributed, identical trial structure to exposure phase).

Spatial learning and memory test

Outside of the scanner, participants learned to locate objects within the four virtual environments across three learning blocks. Participants were instructed to learn which objects were located in the corners of each environment. On each learning trial, participants were spawned within an environment at the center position along one of the lengths of the arena space (Fig. S1A). Participants were then cued with an object (0.5 s) and instructed to navigate through the environment until reaching the unmarked location of the cued object (60 s time limit; Fig. 1C, learning). For further details on the object locations and participant spawn locations in each environment, see Supplemental Material and Fig. S1. Navigation occurred from the first-person perspective using a walking speed of 10 vm/s in all environments. A cued object was not visible to participants during navigation until they were within 8 vm of an object’s location, at which point the object appeared (0.5 s) providing feedback to participants about the object’s exact location. Participants were then spawned in one of the three other environments and cued with an object to find that was uniquely found within that environment. Environment sampling was interleaved; participants had to spawn in each environment before returning to an environment. Spawn position was randomized within environment; participants had to be spawned into each of the four spawn positions within an environment before revisiting a spawn position. We counterbalanced our spawn positions across trials within an environment to discourage learning that may lead to a consistent location mapping or response learning strategy (i.e. turn left, then turn right) and instead promote learning of a mental configuration or grouping of the object locations by environment.

Path distance was calculated as the total movement along the participant’s route; path distance traveled was calculated from positioning coordinates output from custom Unity scripts every 0.8 s during navigation. Path efficiency was normalized based on the Euclidean distance between the participant’s start location and the object location (e.g. total path distance/Euclidean distance to object from start location). Path efficiency of one indicates the path taken was the Euclidean distance. Response time was calculated as the time elapsed between spawning in the environment and when the participant was within 8 vm of the object’s location.

Learning the object locations in each environment occurred in an interleaved manner. Each object was cued twice within a learning block; all sixteen objects were cued once before the second repetition of any object could occur. Within a learning block, the trial order was constrained such that objects assigned to the same environment were not cued on consecutive trials. This constraint avoided any temporal associations between the object presentations from the same environment during the spatial learning phase, which was essential given the use of a temporal sequence learning manipulation on day 2 (see section “Incidental sequence task” below). Across the three learning blocks, each object was cued six times within its assigned environment. Environment assignment refers to the fixed relationship between environment, location, and object in the spatial learning task; each object only appeared in one environment and always in the same location. The object is assigned to that environment for the entirety of the scanning sessions.

After the final learning block, participants completed a test block, in which each object was cued once. At the beginning of the trial, participants were spawned within an environment and cued with an object to locate. The navigation phase was identical to the learning phase except the objects did not appear once participants arrived at the location. In other words, no feedback was given once participants moved within the selected corner of the environment. As during learning, the trial order was constrained so that objects assigned to the same environment were not cued on consecutive trials. The duration of the spatial learning task on average was approximately 40 min (± 5 min), varying due to the free roaming nature of navigation in the virtual environments.

Scanned exposure to objects after learning

Upon completion of the spatial learning and memory test, participants were again scanned while viewing individual presentations of the objects. This phase was identical to the pre-learning exposure phase (see Supplemental Material for performance).

Incidental sequence task

Twenty-four hours later, participants returned for the second experimental session. Across three separate scanning runs, participants viewed the sixteen objects from the spatial learning task in the context of an incidental sequence task. On each trial, an individual object was presented on the screen superimposed with a black plus sign (0.5 s), and participants rated their preference for each object presentation on a 4-point scale (Trapp et al. 2014) (1 = low preference, 4 = high preference; Fig. 3A). Response time was calculated as the elapsed time from the start of a trial until participants indicated their preference judgment by button press. Object presentations were separated by a pseudorandom ITI sampled from 3 s (40%), 4.5 s (40%), or 6 s (20%) (Turk-Browne et al. 2012).

Unbeknownst to participants, object presentation order was generated based on the spatial commonalities and differences derived from the spatial learning task one day prior. Objects were presented in sequential triplets comprising three objects that were assigned to either the same environment or three different environments (Fig. 3A). Critically, the objects were viewed in isolation on the screen with no reference to the environment assignment given. Within a run, there were 32 triplets, with triplet conditions alternating in presentation order (e.g. same, different, same).

For the same-environment triplet condition, the three objects presented sequentially were assigned to the same environment in the spatial learning task, with environment assignment equally represented within a run (four triplets per environment assignment). Each object from the spatial learning task occurred three times within a run as a part of a same-environment triplet. Importantly, having four objects per environment with this triplet structure allowed for object shuffling within the triplets in the incidental sequence task. Within a run, the alternating presentations of same-environment triplets was controlled such that triplets from the same environment did not occur in succession. For the different-environment triplet condition, the three objects presented sequentially were assigned to three different environments in the spatial learning task. Objects in the different-environment condition could not share the same environment assignment as the object either ahead or behind it in presentation order.

Our analyses, both in terms of behavior and neural response, focus on the transition between same- and different-environment triplets. We hypothesized that predictions about the objects and their associated environmental context would build up across successive presentations of objects from the same environment. When transitioning to a different-environment triplet, prediction would be uncertain, providing a signal about temporal boundaries that emerge from learned information about spatial statistics. Given this hypothesis testing framework, the difference in response times at the transition between same- and different-environment triplets was calculated by subtracting the response time of the last object presentation of a same-environment triplet from the first object presentation of a different-environment triplet. We define this transition from a same-environment triplet to a different-environment triplet as an incidental sequence boundary. Trials in which there was no response were excluded from analyses. Each of the three runs was 9 min and 24 s, resulting in a 28 min and 12 s duration for the incidental sequence task. On average, no response was given on 4.0227% of trials (± 9.342% SD) across runs. In a postscan questionnaire completed at the end of day 2, none of the participants indicated awareness of the underlying sequence manipulation during the incidental sequence task.

Environment localizer

Following the incidental sequence task, participants completed three runs of a localizer task intended to decode brain activation patterns associated with perception of each of the four learning environments. Within each run, participants viewed still images from each of the four environments (1.5 s presentation with a 0.5 s ITI; Fig. 1C and Fig. S2A). The presentation order of images was blocked by environment. Per run, there were two blocks of images from each environment; block order was randomized such that successive block presentations could not be from the same environment. Within each block, participants saw four images from an environment twice. Successive image presentations were separated by at least two other image presentations within the environment block. Participants performed a one-back task during the localizer, indicating via button press whether each scene was new or a repeat of the immediately preceding picture (one repeat per block). One of the four environment images was randomly selected as a repeat image presentation as the behavioral cover task. Participants were instructed to press a key for each image presentation to ensure they were on task. When the repeat image was seen, participants were asked to press a different key indicating they viewed a repeat image. Across runs, all four environment images had to serve as the repeating image before being selected again. Additionally, there were five randomly distributed blocks of null fixation (black plus sign centrally fixated on a white screen, 16 s duration). Each of the three runs was 8 min, resulting in a 24 min duration for the environment localizer.

Imaging acquisition

Imaging data were acquired on a 3T Siemens Skyra MRI at the University of Texas at Austin Biomedical Imaging Center. High-resolution whole-brain functional images were acquired using a T2^*-weighted multiband-accelerated EPI pulse sequence (TR: 2 s, TE: 30 ms, flip angle: 73° FOV: 220 mm, 75 slices, matrix: 128 × 128, 1.7 mm isotropic voxels, multiband factor: 3, GRAPPA factor: 2, phase partial Fourier: 7/8). A field map was collected on each day of scanning (TR: 628 mm, TE: 5 and 7.46 ms, flip angle: 5°, 1.7 × 1.7 × 2 mm voxels) to correct for distortions in the magnetic field. A T1-weighted 3D MPRAGE (TR: 1.9 s, TE: 2.43 ms, flip angle: 9° FOV: 256 mm, 192 slices, 1 mm isotropic voxels) was acquired on each day for co-registration and normalization of the functional data to an anatomical template.

Image preprocessing

Functional and anatomical images were preprocessed using FMRIB Software Library version 5.0 (FSL: http://fsl.fmrib.ox.ac.uk/fsl/) and Advanced Normalization Tools (ANTs) (Avants et al. 2011). Structural MPRAGE images from day 1 and day 2 were first corrected for potential bias field signal using N4BiasFieldCorrection. To combine structural images across days, both MPRAGEs were co-registered using antsRegistration and antsApplyTransforms, then scaled using a single multiplicative value, and finally averaged. The averaged MPRAGE image was then used in FreeSurfer (Fischl 2012) to automatically segment cortical and subcortical areas. A brain mask was created by dilating the reconstructed cortex estimated from FreeSurfer and intersecting with the automatically generated mask from FreeSurfer. The resulting brain mask was then used to remove nonbrain tissue from the MPRAGE. A group-level T1 template was created from the buildtemplateparallel program in ANTs, using brain-extracted MPRAGE scans from all individual participants (N = 30; rigid initial target, 3 affine iterations, 10 nonlinear iterations). The group template was then registered to the FSL 1 mm MNI template brain using affine registration implemented in ANTs.

Functional scans were corrected for motion using spline interpolation in MCFLIRT. Functional scans were unwarped using a modified version of epi_reg from FSL. This modification allowed for use of boundary-based registration implemented in FSL, followed by ANTs to refine registration between functional scans and the MPRAGE acquired on the same day. Registration refinement was completed over 20 iterations, with an unwarped functional target image updated on each iteration according to the latest registration (using FSL’s FUGUE tool) (Jenkinson 2003). The brain mask derived from FreeSurfer was projected into native functional space and used to remove nonbrain tissue from the unwarped functional scans. Average brain-extracted unwarped functional scans were then registered to a single reference scan (the fourth scan from the pre-learning exposure from the day 1 session) using ANTs. After calculating all transformations, motion correction, unwarping, and registration to the reference functional scan were applied to the raw functional data using B-spline interpolation (in two steps to minimize interpolation). The bias field for the average functional image in each run was estimated for each scan using N4BiasFieldCorrection implemented in ANTs. The bias field was then removed by dividing the timeseries by a single estimated bias field image. Functional timeseries were high-pass filtered (128 s) and smoothed at 4 mm FWHM using FSL’s SUSAN tool.

Regions of interest

Based on prior work examining cognitive map formation in humans (Marchette et al. 2014; Schlichting et al. 2015; Constantinescu et al. 2016; Deuker et al. 2016; Garvert et al. 2017; Morton et al. 2020; Peer and Epstein 2021), we targeted our analyses on three regions of interest (ROI): hippocampus, parahippocampal cortex (PHC), ventromedial prefrontal cortex (vmPFC), and retrosplenial cortex (RSC). We also further subdivided PHC into its anterior and posterior functional subdivisions, given that anterior PHC in particular has been associated with contextual representation of objects and formation of cognitive maps (Bar and Aminoff 2003; Aminoff et al. 2013; Baldassano et al. 2013; Morton et al. 2020). In particular, we hypothesized that anterior PHC would show evidence for predictive reactivation of environmental contexts during sequential presentation of objects on day 2. Hippocampal and vmPFC ROIs were manually demarcated on a custom template image in 1 mm MNI space based on a cytoarchitectonic atlas (Öngür et al. 2003; Price and Drevets 2010). Each ROI was then dilated to allow for variability in neocortical anatomy across sampled participants by smoothing with a sigma of 1 mm and thresholding at 0.01. The RSC and PHC ROIs were created from a probability map created in group template space using the FreeSurfer isthmus of the cingulate gyrus and parahippocampal gyrus regions, respectively. Anterior and posterior delineations of the PHC were created from the group template ROI based on an equal split of slices along the long axis (Bar and Aminoff 2003). Native participant masks were warped to 1-mm resolution group template space using nearest-neighbor interpolation in ANTS. The resulting warped participant masks were then averaged to create a probability map for gray matter in template space, threshold at 0.75 (to include voxels labeled as gray matter for at least 75% of participants). All ROIs were reverse normalized using ANTs into each participant’s native functional space.

Estimation of object-specific neural patterns evoked during pre- and post-learning exposure tasks

Neural patterns associated with individual objects were estimated under the assumptions of the general linear model (GLM) using a least squares-separate (LS-S) approach (Mumford et al. 2012). By estimating neural patterns for individual objects in the pre- and post-learning exposure phases, we were able to investigate learning-related changes in the neural representations of the individual objects. For each participant, separate event-specific univariate GLMs were conducted for each run of the pre- and post-learning exposure tasks. Object presentations were modeled as 0.3 s events (Schlichting et al. 2015) and were convolved with the canonical (double gamma) hemodynamic response function (HRF) in FSL. The four presentations of each object within a run were modeled as a single regressor. Nuisance regressors that accounted for the remaining signal variance within a trial were also modeled, including delay periods, six motion parameters and their temporal derivatives, frame-wise displacement (FD), and DVARS (Power et al. 2012). Additional regressors were created to remove time points with excessive motion (defined as greater than 0.5 mm of frame-wise displacement and greater than 0.5% change in BOLD signal for DVARS), as well as one time point before and two time points after each high-motion frame. The model was conducted on the spatially preprocessed data and high-pass temporal filtering (128 s) was further applied to each regressor in the modeling. The resulting beta images consisted of voxelwise parameter estimates for each of the 64 object-based activation patterns (16 per run) in each of the eight runs of the pre- and post-learning exposure task, totaling 128 images per participant.

Estimation of condition-specific neural patterns evoked during incidental sequence task

A separate model estimated separate event-specific univariate GLMs for the individual object presentations during the incidental sequence task on day 2 using the LS-S method. By estimating individual object presentations during the incidental sequence task, we measured whether environment-related memories were reactivated across object presentations learned in the same context. The object presentation positions were based on environment assignment in the spatial learning task (same-environment, different-environment) and position within a triplet (position 1, 2, or 3). Hence, the six conditions isolated within the model were three same-environment regressors corresponding to triplet positions 1–3 and three different-environment regressors, similarly corresponding to the three triplet positions (Fig. 3A). Object presentations were modeled as 2 s events and were convolved with the HRF in FSL. The sixteen presentations of each environment/position condition were modeled as a single regressor. Nuisance regressors and filtering were carried out in the same manner as outlined above. This process resulted in one voxelwise beta image for each of the six object positions per four environment assignments per run of the incidental sequence task (24 per run; 3 runs total), totaling 72 images per participant.

Estimation of trial-level neural patterns evoked during incidental sequence task

To quantify the effect of reinstated context at event boundaries in the incidental sequence task, we extracted neural patterns for each of the 96 individual object presentations (sixteen presentations of same- and different-environment conditions with three object positions per condition; see section “Incidental sequence task” above). For each scanned incidental sequence task run, separate event-specific univariate GLMs were created using the LS-S method, with individual object presentations modeled as individual 2 s regressors convolved with the HRF in FSL. Nuisance regressors and filtering were carried out in the same manner as outlined above. The resulting beta images consisted of trial-level parameter estimates for each of the 288 object-based activation patterns (96 per run; 3 runs total) for each participant. These trial-level inputs were used as inputs in our classification analysis below (see section “Relating reactivation differences of environment content at implicit sequence boundaries with trial- level univariate response”).

Quantifying learning-related representational change

Representational similarity analysis (RSA) was conducted using the PyMVPA toolbox (Hanke et al. 2009) and custom Python routines in the native space of each participant. The goal of this analysis was to compare how the representation of individual objects shifted from pre- to post-learning. Specifically, we sought to identify voxels within our four a  priori anatomical ROIs—hippocampus, PHC, vmPFC, and RSC—for which the pattern of activation elicited by objects assigned to the same environment became more similar after learning, while simultaneously becoming more distinct for objects assigned to different environments (Fig. 2B).

Using a searchlight approach (radius: three voxels), similarity matrices were generated by calculating the pairwise Pearson’s correlation values for each of the 128 statistics images derived from the pre- and post-learning exposure runs corresponding to individual object presentations; correlation values were then transformed to Fisher’s z scores. Changes in the correlation patterns as a function of spatial learning were then measured by subtracting the pre-learning correlation values from the post-learning similarity values in corresponding cells (Fig. 2A). After the change in representational similarity (henceforth referred to as ΔRS) was calculated, ΔRS values were sorted into two vectors depending on whether the value was for a same-environment comparison or a different-environment comparison. Separating the data into these two vectors allowed us to determine how representational change was influenced by whether two objects shared an environmental context or not. Importantly, only ΔRS values from across-run comparisons were used to reduce potential bias possibly introduced from auto-correlation in the BOLD signal (Mumford et al. 2012). Independence of the activation patterns going in to ΔRS calculations would thus preserve the false-positive rate (Mumford et al. 2014).

For each searchlight sphere, contrasts representing our hypothesized effect were computed: (same environment_Post—different environment_Post)—(same environment_Pre—different environment_Pre). Contrasts were converted to p-values by comparing the observed contrast value to a permutation-based null distribution. We generated null distributions by randomly shuffling same- versus different-environment ΔRS values and then re-calculating the statistic of interest for each of 1,000 iterations. Significance was calculated for the center voxel of each searchlight sphere by reporting the proportion of the null distribution with values greater than or equal to the observed similarity change (i.e. p-value). The resulting p-value was assigned to the center voxel of the current sphere; the sphere was shifted and the entire procedure repeated. Our searchlight RSA analysis resulted in a participant-specific p-value map for each of the three anatomical ROIs.

To identify brain regions that support the hypothesized interaction effect across participants, individual participant searchlight maps were normalized to a group template for significance testing. Each participant’s p-value images were converted to z-statistic transformations (allowing for both positive and negative values) and the resulting images were warped to the template space using ANTs. Z-statistic maps were then combined across participants using voxel-wise, nonparametric one-sample t-tests implemented in Randomize, part of FSL (Winkler et al. 2014). Significant clusters were identified by applying a voxel-wise threshold of p< 0.05 to all group statistics images to identify voxels surpassing this initial p-value threshold. For cluster correction, we estimated the spatial smoothness of the data based on residuals from the object-level GLM. Significant cluster sizes within each anatomical ROI were then calculated separately using the AFNI function 3dClustSim with smoothness estimates derived from 3dFWHMx based on the spatial Auto Correlation Function (ACF) (Cox 1996). Cluster sizes that occurred with a probability of less than 0.05 across 10,000 simulations using two-sided thresholding with second-nearest neighbor clustering were considered statistically significant. The minimum cluster sizes were determined to be 277 voxels for hippocampus, 1,170 voxels for medial PFC, 422 voxels for RSC, 308 voxels for the PHC, and 4,922 voxels for gray matter. All clusters exceeding these criteria within our four a  priori anatomical regions of interest are reported in the Results (see section “Hippocampus, ventromedial prefrontal cortex, and retrosplenial cortex form cognitive maps reflecting object-environment relationships”) and at the whole-brain gray matter level are reported in the Supplemental Material (see section “Representational change across learning at the whole-brain level”).

Decoding reactivation of environment content during object viewing

We further assessed whether information about the individual environments was reinstated when viewing individual objects after the spatial learning task. To do so, we used multivariate pattern classification approach, training a neural classifier on data from the environment localizer task and applying it to decode evidence for reinstatement of the environments during object viewing in other experiment phases: pre- and post-learning exposure along with the incidental sequence task.

Pattern classification analyses were implemented using PyMVPA (Hanke et al. 2009) and custom Python routines. A linear support vector machine (SVM) classifier was trained to discriminate activation patterns elicited by each of the four environments during the localizer task. A regressor matrix labeled the image presentations by environment on the functional timeseries data of the localizer task. To account for the hemodynamic lag, condition labels were shifted back by three scans (6 s) with respect to the functional timeseries (Zeithamova et al. 2012). We focused on decoding viewpoint-independent environment reactivation within the anatomical PHC (see Fig. S2B for classification accuracy in our a  priori anatomical ROIs) based on its hypothesized role in context representation (Bar and Aminoff 2003; Aminoff et al. 2007; Bar et al. 2008), its ability to decode scene content (Epstein and Kanwisher 1998; Polyn et al. 2005) as well as its role in reinstating scene content predictively (Turk-Browne et al. 2012; Gershman et al. 2013).

We evaluated classifier accuracy using leave-one-out cross-validation on the three runs of the localizer task. Classifier performance during the localizer task was assessed by taking the average accuracy across the cross-validation folds within each participant. The SVM classifier was then used to predict environment reactivation during object presentations in the pre- and post-learning exposure tasks and the incidental sequence task. Probability estimates of the environment labels were extracted from the SVM classifier for the three runs of the localizer task and used to index the amount of classifier evidence for the four environments during each object presentation. Each evidence estimate was labeled based on predicted environment classified for that trial. To test the significance of environment reactivation, we compared the predicted environment labels to a null distribution. The null distribution was created over 1,000 iterations by shuffling environment labels and re-calculating the difference in reactivation every iteration. The center voxel of each searchlight sphere within our a  priori anatomical ROIs (Fig. S2B) reported the proportion of the null distribution with reactivation indices greater than or equal to the observed reactivation index (i.e. p-value).

To assess whether representational similarity in our a  priori ROIs resulted from reactivation of contexts during post-learning exposure, we compared classification accuracy within the PHC to the representational similarity change across learning in hippocampus, vmPFC, and RSC. The change in representational similarity was calculated based on the difference in the correlation patterns for objects as a function of spatial learning by subtracting the pre-learning correlation values from the post-learning similarity values. An across-participant correlation, using a robust regression to down weight outliers that may influence an overall effect (Winkler et al. 2014), assessed the relationship between the context reactivation and representational change in hippocampus, vmPFC, and RSC (Fig. 2C).

Additionally, we quantified how environment information was reinstated in the anatomically-defined PHC and its anterior and posterior subregions during presentation of objects during the incidental sequence task. Anterior PHC have been shown to represent both spatial and nonspatial associations, allowing for more generalizable codes between events and the context in which they occurred (Bar and Aminoff 2003; Bar et al. 2008). Here, we predict that the entire PHC may be better at decoding viewpoint-independent scenes, such as in our localizer. However, when reinstating context associated with an object in a nonspatial scenario, such as our incidental sequence task, context reinstatement may be focused within the anterior PHC. To examine predictive reinstatement of the environments during the incidental sequence task, the SVM classifier trained on the localizer data was applied to the incidental sequence task to provide a measure of environment-specific reactivation during individual object presentations in the incidental sequence task. Probability estimates of the environment labels were extracted from the SVM classifier and used to index the amount of classifier evidence for the four environments during each object presentation across all voxels in our PHC ROIs. We defined two conditions when weighing classifier accuracy: target and non-target evidence. The target for each object presentation was based on the environment in which the object was located during the spatial learning task. Non-target evidence was calculated as the average evidence for the three remaining environments. For each trial, we calculated the difference between target and non-target activation in our PHC ROIs and averaged those differences for each condition (same-environment, different-environment) as well as a function of triplet position (first, second, third). Classification accuracy was z-scored and entered into a repeated-measures ANOVA with triplet position (first, second, third) and condition (same-environment, different-environment) as factors to assess linear trends of context reactivation within the incidental sequence task.

Quantifying univariate activation differences at implicit sequence boundaries

To assess sensitivity to implicit sequence boundaries, we performed a univariate GLM analysis of the incidental sequence task using FEAT (fMRI Analysis Toolbox) version 6.00 in FSL. There were six regressors in the GLM, corresponding to the temporal position (first, second, third) in the same- and different-environment sequences. Object presentations were modeled as events with 0.5 s durations with one regressor for each condition of interest. The model was convolved with the canonical (double-gamma) HRF. Motion parameters and their temporal derivatives were added as additional regressors of no interest. After modeling functional data within each run, the resulting statistics images were warped into 1 mm resolution group template space using ANTs. The resulting images were combined across runs for each participant using a fixed effects model, and then across participants using a mixed effects model.

A linear contrast then tested for activation increases after a triplet boundary was crossed (Object After > Object Before). Significant clusters were identified by applying a voxel-wise threshold of p < 0.05 to all group statistics images to identify voxels surpassing this initial p-value threshold. For cluster correction, we estimated the spatial smoothness of the data based on residuals from the group-level GLM. Significant cluster sizes within hippocampus and vmPFC were then calculated separately using the AFNI function 3dClustSim with smoothness estimates derived from 3dFWHMx based on the spatial ACF (Cox 1996). Cluster sizes that occurred with a probability of less than 0.05 across 10,000 simulations using two-sided thresholding with second-nearest neighbor clustering were considered statistically significant. The minimum cluster sizes were determined to be 342 voxels for hippocampus, 1,209 voxels for medial PFC, and 5,084 voxels for gray matter. All clusters exceeding these criteria within our a  priori anatomical regions of interest are reported in the Results (see section “Hippocampus and ventromedial prefrontal cortex activation at implicit sequence boundaries”) and at the whole-brain gray matter level are reported in the Supplemental Material (see section “Whole-brain activation at implicit sequence boundaries”).

Relating trial-level reactivation differences of environment content at implicit sequence boundaries with univariate response

To assess the effect of context reinstatement in the implicit sequence task on boundary sensitivity, we predicted that context reinstatement in the anterior PHC may represent a generalizable context code that would relate to boundary sensitivity in the implicit sequence task. As discussed above (see section “Decoding reactivation of environment content during object viewing”), the anterior PHC represents generalizable codes related to learned associations (Bar and Aminoff 2003; Bar et al. 2008). Here, we predicted that anterior PHC would evince reinstatement of environmental context associated with each object during the incidental sequence task. To examine predictive reinstatement of the environments at the trial level during the incidental sequence task, our SVM classifier trained on the localizer data was applied to the incidental sequence task to provide a measure of environment-specific reactivation during individual object presentations in the incidental sequence task (see section “Decoding reactivation of environment content during object viewing above, in particular reinstatement of environment content during the incidental sequence task”). Probability estimates of the environment labels were extracted from the SVM classifier and used to index the amount of classifier evidence for the four environments during each object presentation across all voxels in the anterior PHC ROI. Similar to our decoding analysis above, we defined two conditions when weighing classifier accuracy: target and non-target evidence. The target for each object presentation was based on the environment in which the object was located during the spatial learning task. Non-target evidence was calculated as the average evidence for the three remaining environments. For each trial, we calculated the difference between target and non-target activation in the anterior PHC ROI at the trial level. To calculate the change in classification accuracy at the implicit boundaries, we subtracted the z-scored classification accuracy before the implicit boundary minus the z-scored classification accuracy after the implicit boundary.

To assess whether boundary sensitivity was related to changes in reactivation of environmental content in the anterior PHC, we performed a univariate GLM analysis of the incidental sequence task using FEAT (fMRI Analysis Toolbox) version 6.00 in FSL. There were two regressors in the GLM, corresponding to an implicit boundary from the same-environment condition to the different-environment condition and regressors of no interest. Here, we predict that neural sensitivity at the implicit boundaries may be related to the change in the reinstated context in anterior PHC. Each boundary was modeled as events with durations from the onset of the object presentation before the implicit boundary (same-environment third position) through the presentation of the first object of the subsequent different-environment triplet (different-environment first position). The change in anterior PHC environment classification accuracy for the implicit boundary was included as a within-trial parametric modulator. The model was convolved with the canonical (double-gamma) HRF. Motion parameters and their temporal derivatives were added as additional regressors of no interest. After modeling functional data within each run, the resulting statistics images were warped into 1 mm resolution group template space using ANTs. The resulting images were combined across runs for each participant using a fixed effects model, and then across participants using a mixed effects model.

Significant clusters were identified by applying a voxel-wise threshold of p < 0.05 to all group statistics images to identify voxels surpassing this initial p-value threshold. For cluster correction, we estimated the spatial smoothness of the data based on residuals from the group-level GLM. Significant cluster sizes within hippocampus and vmPFC were then calculated separately using the AFNI function 3dClustSim with smoothness estimates derived from 3dFWHMx based on the spatial ACF (Cox 1996). Cluster sizes that occurred with a probability of less than 0.05 across 10,000 simulations using two-sided thresholding with second-nearest neighbor clustering were considered statistically significant. The minimum cluster sizes were determined to be 310 voxels for hippocampus and 1,193 voxels for medial PFC. All clusters exceeding these criteria within our two a  priori anatomical regions of interest are reported in the Results (see section “Hippocampal boun dary sensitivity relates to environment reinstatement in anterior parahippocampal cortex”).

Quantifying changes in functional connectivity at implicit sequence boundaries

Functional connectivity during the incidental sequence task was examined using a psychological-physiological interaction approach (PPI) carried out in Feat. PPI analyses assessed how connectivity among our ROIs was altered at implicit sequence boundaries. Specifically, we examined whether connectivity among our ROIs decreased after a boundary, as measured by a linear contrast between activation before a boundary (third object in a same-environment triplet) relative to after the boundary (first object in a different-environment triplet). The first eigenvariate of the filtered timeseries (derived from the univariate analysis as described above, see section “Quantifying univariate activation differences at implicit sequence boundaries”) was extracted from the hippocampus, vmPFC, and RSC regions that showed evidence of cognitive map formation (Fig. 2B). The PPI regressor of interest was generated as the interaction between the psychological and physiological regressors. An additional task regressor was included to account for variance associated with all object presentations across the two conditions (same- and different-environment triplets) included in the incidental sequence task design.

For all PPI models, stimulus presentation was modeled as events with 2 s durations. Task-related regressors and their temporal derivatives were convolved with the canonical (double-gamma) hemodynamic response function (HRF) and filtered. Physiological and PPI regressors were not convolved with the HRF or filtered, as these regressors were derived from neural signal that had previously undergone temporal filtering. Motion parameters and their temporal derivatives calculated during motion correction were added as an additional regressor of no interest.

After modeling functional data within each run, the statistic images associated with the PPI regressor were warped to the 1 mm group template brain using ANTs. The resulting PPI images were combined across runs for each participant using a fixed effects model, and then across participants using a mixed effects model. We hypothesized that when prediction is uncertain, i.e. at implicit sequence boundaries, the connectivity between vmPFC and hippocampus might decrease and further modulate the slowing of response time at those boundaries. As such, the average, mean-centered difference in preference response times at the triplet boundary was included as a within-participant parametric modulator of connectivity in the mixed effect model. Correction for multiple comparisons was performed within hippocampal and vmPFC cognitive map ROIs using small volume correction implemented in 3dClustSim as described above (see section “Quantifying univariate activation differences at implicit sequence boundaries”).

We further investigated whether changes in hippocampus and vmPFC coupling across implicit sequence boundaries corresponded to decreases in context reinstatement across those boundaries as measured via our trained neural classifier. Theories of hippocampal-vmPFC interactions predict that hippocampus sends predictive information about spatial context to vmPFC when re-experiencing previously encountered information (Eichenbaum et al. 2007; Preston and Eichenbaum 2013; Place et al. 2016). We hypothesized that decreased hippocampal and vmPFC coupling across implicit sequence boundaries would be modulated by greater reinstatement prior to transitioning between same- and different-environment triplets (average environment classification accuracy for the third object of a same-environment triplet). The average environment classification accuracy for the third object in anterior PHC was included as a within-participant parametric modulator of hippocampal and vmPFC connectivity in the mixed effect model. Correction for multiple comparisons was performed within a  priori hippocampal and vmPFC ROIs using small volume correction implemented in 3dCLustSim as described above (see section “Quantifying univariate activation differences at implicit sequence boundaries”).

We then assessed the correlation across-participants between hippocampal-vmPFC functional decoupling and the decrease in context reactivation across event boundaries. For the functional decoupling, parameter estimates were extracted from our hippocampal functional cluster (Fig. 4E) identified as significantly relating to our decrease in functional connectivity analyses seeded with vmPFC and our vmPFC functional cluster (Fig. 4F) identified as significantly relating to our decrease in functional connectivity analyses seeded with hippocampus. The change in contextual reactivation across event boundaries was calculated by subtracting the classification accuracy for the first object presentation of a different-environment triplet from the classification accuracy of the third object presentation from a same-environment triplet.

Fig. 4.

Neural sensitivity to implicit sequence boundaries. A) The change in object representational similarity within the anterior hippocampus (HPC) pre- to post-learning is linked with a slowing in response time across implicit sequence boundaries in the incidental sequence task. Individuals who formed more coherent HPC representations of the spatial similarities and differences between objects showed greater response time slowing after a boundary. B) Environment reactivation in anterior PHC increased across sequential presentations of objects learned in the same environment on day 1 (SE = same-environment). Reactivation of environments were not significant during presentation of objects in different-environment triplets (DE = different-environment). Error bars denote SEM. Asterisk indicates significance of p < 0.05. Related to Fig. S2D. C) vmPFC activation was increased after an implicit sequence boundary, as quantified by a linear contrast between the first presentation within a different-environment triplet relative to the third presentation of the same-environment triplet. Related to Fig. S3A and Table S1. D) A similar increase was observed in anterior HPC when the linear contrast was weighted by response time slowing after the boundary, indicating that participants who showed the largest increase in HPC activity after the boundary were those who showed the greatest response time slowing. Related to Fig. S3B and Table S1. E) At the trial-level, the decrease in anterior PHC predictive context reinstatement as object presentations crossed a boundary relates to signaling in hippocampus, indicating that a decrease in spatial context reinstatement at an implicit nonspatial boundary may contribute to hippocampal boundary sensitivity. Connectivity between HPC and vmPFC decreased after implicit sequence boundaries, both when F) vmPFC and G) HPC served as seeds in the analysis. H) Weighted by high contextual expectations prior to the event boundary, functional decoupling between cognitive map regions HPC and vmPFC was modulated by a decrease in context reactivation at triplet condition boundaries. In A) and H), the light gray within the plot indicates the 95% confidence interval of the regression fit.

Relating neural responses to behavior at implicit sequence boundaries

A multiple linear regression analyses was performed to further assess the degree to which hippocampal cognitive map formation, hippocampal boundary sensitivity, and hippocampal and vmPFC functional decoupling measures were independently related to behavior at implicit sequence boundaries. Hippocampal cognitive map formation refers to the difference in representational similarity calculated based on changes in the correlation patterns for objects as a function of spatial learning by subtracting the pre-learning correlation values from the post-learning similarity values (Fig. 2A). For our univariate boundary sensitivity hippocampal regions, parameter estimates were extracted from within the hippocampal functional cluster (Fig. 4D) identified as significantly relating to an increase in neural response across event boundaries. For functional connectivity, parameter estimates were extracted from our vmPFC and hippocampal functional clusters (Fig. 4E and F) identified as significantly relating to a decrease in functional connectivity seeded with hippocampus or vmPFC, respectively. A linear regression model was run using the mean-centered difference in response times across implicit sequence boundaries as the dependent variable. The difference in response times across triplet condition boundaries in the incidental sequence task was calculated by subtracting the response time of the last object presentation of a same-environment triplet from the first object presentation of a different-environment triplet. Representational change in hippocampus, hippocampal univariate boundary sensitivity, hippocampal seed and vmPFC seed functional connectivity measures for each participant were entered into the regression as predictors. Participants were treated as a random effect.

Results

Object locations were learned through virtual navigation

In the spatial learning task, participants learned the locations of sixteen novel objects by actively navigating in four distinct virtual reality environments across six repetitions (Fig. 1A). Participants learned the locations of four objects within their assigned environment (Fig. 1B) as indicated by an increase in path efficiency and a decrease in response time across object repetitions (Fig. 1D; see Supplemental Material for analysis of behavioral data). The observed increase in path efficiency and decrease in response time across object repetitions indicates participants were moving to the object locations in a more precise manner as learning occurred. After learning, participants were tested on the location of each object in its assigned environment (Fig. 1D). Memory for object locations within environment at test was on average 92.33% (3.38% SEM), indicating participants had established a high degree of knowledge for the object locations across environments.

Hippocampus, ventromedial prefrontal cortex, and retrosplenial cortex form cognitive maps reflecting object-environment relationships

The organization of knowledge into cognitive maps is thought to rely on memory integration and differentiation processes that organize memory elements according to their shared properties as well as their differences (Schlichting and Preston 2015; Morton et al. 2017). Here, we tested how neural organization of the object representations shifted after learning to reflect the spatial commonalities and differences acquired during the virtual navigation task. Given their predicted roles in cognitive map formation (Tse et al. 2007, 2011; Marchette et al. 2014), we interrogated representational shifts in hippocampus, PHC, vmPFC, and RSC from pre- to post-learning. We hypothesized that these regions may form hierarchical representations reflecting the spatial commonalities and differences between individual objects. Specifically, we quantified whether representation of objects experienced within the same virtual environment became more similar to one another after learning, while simultaneously being differentiated from objects experienced in distinct virtual environments.

To test these hypotheses, participants viewed the individual objects in isolation during fMRI scanning both immediately before and after the spatial learning task. We quantified learning-related changes in neural representations of each object, calculating within-environment object similarity increases and across-environment object differentiation using representation similarity analysis (RSA) (Kriegeskorte et al. 2008) searchlights constrained to our ROIs. We isolated searchlight regions exhibiting the hypothesized hierarchical representation pattern, wherein voxel patterns elicited by objects experienced within the same environment during spatial learning became more similar after learning, while simultaneously becoming less similar for objects experienced within different environments (Fig. 2A). The hypothesized hierarchical representational pattern was not found in either anterior or posterior PHC (see Supplemental Material for RSA in anatomical subdivisions of PHC). We did observe regions in left anterior hippocampus (cluster center of gravity in Montreal Neurological Institute (MNI) template coordinates (mm): x, y, z = −19, −9, −23), left posterior ventromedial prefrontal cortex (vmPFC; MNI coordinates (mm): x, y, z = −1, 24, −25) extending bilaterally, and bilateral RSC (MNI coordinates (mm): x, y, z = 3, −46, 11) that demonstrated hierarchical representation of objects based on their spatial commonalities and differences (Fig. 2B).

Post-learning representations in hippocampus, ventromedial prefrontal cortex and retrosplenial cortex are unrelated to reactivation of environmental contexts

Next, we ruled out a potential alternative interpretation of our representational similarity results. It is possible that changes in representational similarity among the objects observed in hippocampus, vmPFC, and RSC may not reflect alterations in the object representations themselves, but rather are a byproduct of reactivation of perceptual representations of the virtual environments during object viewing. In other words, when viewing objects in the post-exposure that were experienced within the same environment, activation patterns may be more similar because participants were thinking about the common environment during those trials. Likewise, reactivation of the environment associated with each object may thus also account for greater dissimilarity between different-environment objects. To rule out this interpretation of our cognitive map findings, our analysis plan was 2-fold. First, we isolated the regions in which we could decode reliable environment reactivation while participants viewed the objects in isolation post-learning. Second, we then examined whether the formation of hierarchical cognitive maps in hippocampus, vmPFC, and RSC as measured during the post-learning exposure correlated with environment reactivation.

To measure whether perceptual representations of the virtual environments were reactivated as participants viewed individual objects during the post-learning exposure, we first trained a multivoxel pattern analysis (MVPA) classifier to differentiate activation patterns elicited by each spatial environment using data from a separate localizer task (Fig. 1C). We focused this analyses on PHC, based on its hypothesized role in context representation (Bar and Aminoff 2003; Bar et al. 2008) and its role in predictively reinstating scene contexts during memory-based decision making (Turk-Browne et al. 2012; Julian et al. 2018) (see Fig. S2B for classification accuracy in a  priori anatomical ROIs beyond PHC). The trained classifier was then applied to data from the pre- and post-learning exposure phases to detect reactivation of the unseen environment when viewing an associated object in isolation (see Fig. S2C for reactivation index in a  priori anatomical ROIs beyond PHC).

Environment decoding in the pre-learning exposure was not significantly above chance in PHC (t₍₂₉₎ = −0.112, p = 0.908; Cohen’s d = 0.032, Fig. S2C, left), which was predicted given that objects have yet to be associated with spatial environments prior to spatial learning. After spatial learning, however, we observed significant environment decoding in PHC as participants viewed objects during the post-learning exposure (t₍₂₉₎ = 2.593, p = 0.015; Cohen’s d = 0.706, Fig. S2C, right). This finding indicates that objects were bound to their spatial environments during learning, eliciting reactivation of perceptual representations of the environments in PHC when objects were viewed in isolation. We then performed across-participant regression analyses relating representational change in hippocampus, vmPFC, and RSC to the degree of reactivation observed in PHC during the post-learning exposure. Environment reactivation was not linked with object representational change in any region (Fig. 2C): hippocampus (r₍₂₉₎ = 0.104, p = 0.605, Bayes Factor (B₁₀) = 3.836453), vmPFC (r₍₂₉₎ = −0.136, p = 0.497, B₁₀ = 3.2205), or RSC (r₍₂₉₎ = 0.135, p = 0.501, B₁₀ = 3.359905). Taken together, while environment reactivation occurs post-learning, the simultaneous representation of commonalities and differences among objects observed in hippocampus, vmPFC, and RSC does not strongly rely on reactivation of perceptual features of the virtual environments.

Cognitive maps support memory-based prediction in new contexts

An important advantage of cognitive maps is their flexibility; information learned in one context can be generalized to new situations (McKenzie et al. 2014). We next sought to provide a representational account of this flexibility by characterizing how spatial cognitive map formation impacts processing in nonspatial contexts. We quantified the influence of cognitive maps in new contexts using a multifaceted approach, which we overview in this section.

Twenty-four hours after completing the spatial learning task as well as the pre- and post-learning exposure phases (Fig. 1C), participants returned for a second scan session in which they completed an incidental sequence task. In this task, objects from the spatial learning task were presented in sequential triplets based on whether they were located in the same or different environment during the spatial learning task (Fig. 3A). The participants were not instructed about the sequential triplet organization, nor did they endorse awareness of the organization when queried after completion of the experiment (see Methods section “Incidental sequence task”). The incidental sequence task design thus allowed us to assess behavioral and neural sensitivity at implicit sequence boundaries between same- and different-environment triplets that result from formation of cognitive maps during the spatial learning task.

First, we hypothesized that participants would reactivate spatial knowledge acquired on day 1 during the incidental sequence task. Specifically, we hypothesized that as participants viewed sequential presentations of objects from the same environment, a progressive buildup of context reactivation would be observed (Fig. 3B). We theorized that the buildup of reactivated contextual information may help participants access information about the other objects associated with the context. In other words, by reactivating the context, one may retrieve the constellation of objects within a cognitive map that is associated with the same environmental context; therefore, predictions can be made about what object might be presented next in the sequence.

However, after a transition from a same-environment triplet to a different environment triplet, we hypothesized that there would be a decrease in predictive signaling about contextual information (Fig. 3C). A decrease in contextual reactivation may lead to increased uncertainty about what objects might be expected to appear next in the sequence. Increased uncertainty may thus result in enhanced neural responses reflecting prediction error or expectancy violations (Kim et al. 2014, 2017; Jang et al. 2019; Hansen et al. 2021). Based on past work showing enhanced hippocampal and vmPFC activation signaling memory-based prediction errors (Schapiro et al. 2016; Baldassano et al. 2017; Clewett et al. 2019), we predicted that these regions would similarly signal such increased uncertainty at implicit sequence boundaries here (Fig. 3D).

We further predicted that hippocampus and vmPFC connectivity would be altered at the implicit boundary between same- and different-environment triplets (Fig. 3E). Electrophysiological work in rodents has shown that contextual information, represented in the hippocampus, is sent to the vmPFC to help guide reactivation of the appropriate actions to take in a given context (Place et al. 2016; Wikenheiser et al. 2017). We hypothesized that a decrease in contextual reactivation at the implicit sequence boundaries would thus correspond to a decrease in reciprocal connectivity between these two structures. Moreover, we predicted that increased uncertainty at implicit sequence boundaries would be reflected in behavior as participants made preference judgments. Specifically, we predicted that response time would be slower after an implicit boundary (Fig. 3F), which would be tracked by neural signatures that indexed context reinstatement and uncertainty signaling (i.e. increased hippocampus and vmPFC activation and hippocampal-vmPFC decoupling).

Implicit contextual boundaries influence behavior

To address the behavioral predictions from our analysis framework, which predicted a response time slowing when making preference judgments at implicit sequence boundaries (Fig. 3F), we compared the average response time for the last object of a same-environment triplet (0.7788 s ± 0.0157 SEM) to response time for the first object of a different-environment triplet (0.7896 s ± 0.0162 SEM). We observed that preference response times slowed on average when moving across implicit sequence boundaries (t₍₂₆₎ = −2.054, p = 0.049, Cohen’s d = 0.086), a behavioral marker showing that spatial knowledge acquired during day 1 influenced how people processed information during the nonspatial temporal sequence task. Leveraging the variance in behavior at the transition between same- and different-environment triplets not only across participants but also on a trial-by-trial basis provides further understanding of the underlying neural mechanisms that are sensitive to the implicit boundaries in the sequence.

Hippocampal cognitive maps track behavioral sensitivity to implicit sequence boundaries

Next, we examined how the coherence of cognitive maps formed on day 1 related to behavior during the incidental sequence task. If participants generalize spatial knowledge to the sequence tasks, our framework proposes that cognitive map coherence should relate to behavioral slowing at implicit sequence boundaries between same- and different-environment triplets. Thus, we predicted that individuals with more coherent hierarchical representations of the spatial similarities and differences between objects would exhibit slower response times at implicit sequence boundaries. To test this hypothesis, we performed regression analyses relating the difference in preference response time at implicit sequence boundaries with the change in object representational similarity derived from our three a  priori regions of interest quantified on day 1 (Fig. 2B). We found that hippocampal representation of the spatial similarities and differences between objects, quantified as the interaction between exposure phase (pre- or post-learning) and environment (same or different) with higher values reflecting more coherent cognitive map formation, was related to slower response times at implicit sequence boundaries (r₍₂₆₎ = 0.411, p = 0.034; Fig. 4A). Representational structure in vmPFC was not related to behavioral sensitivity to implicit sequence boundaries (r₍₂₆₎ = 0.158, p = 0.432). RSC representational structure was trending but not significantly related to behavioral sensitivity to implicit sequence boundaries (r₍₂₆₎ = 0.378, p = 0.052).

Environment reactivation increases across sequential presentations of objects that share a spatial context

While the prior findings indicate that hippocampal spatial cognitive maps influence behavior in nonspatial contexts, our framework (Fig. 3) makes specific predictions about the neural mechanisms associated with such generalization. During successive presentations of objects from the same environment, we hypothesized that reactivation of the associated environment would increase to support temporal predictions about what items from that same environment might appear next in the sequence (Fig. 3B), subsequently falling off after an implicit boundary (Fig. 3C).

As a first step toward testing this hypothesis, we trained neural classifiers to differentiate the distributed patterns of activation associated with each spatial environment using data from the localizer task. Classifiers trained on all of our a  priori regions of interest were able to reliably differentiated perceptual representations of the spatial environments (Fig. S2B). Classifiers were then applied to data from the incidental sequence task to detect reactivation of the associated environment during each object presentation. When examining predictive reinstatement of environments during the incidental sequence task in our a  priori regions of interest, we found environment decoding only in anterior PHC, consistent with its role in associating objects to particular spatial contexts (Bar and Aminoff 2003; Bar et al. 2008). Evidence for reactivation of associated environmental contexts increased across sequential presentations of objects from the same-environment (t₍₂₆₎ = −1.631, p = 0.0575, Cohen’s d = 0.535), and showed significant reactivation during the third object position (SE1: t₍₂₆₎ = −0.623, p = 0.539, Cohen’s d = 0.170; SE2: t₍₂₆₎ = 0.814, p = 0.423, Cohen’s d = 0.222; SE3: t₍₂₆₎ = 2.111, p = 0.045, Cohen’s d = 0.575; Fig. 4B). In contrast, environment decoding within the anterior PHC was not significant for any of the object positions within different-environment triplets (all p > 0.05).

Hippocampus and ventromedial prefrontal cortex activation at implicit sequence boundaries

The fact that we did not observe reactivation of perceptual representations of the environments during the first object presentation of different-environment triplets suggests that predictive signals were reduced after implicit sequence boundaries, leading to heightened uncertainty about upcoming sequence items (Fig. 3D). Therefore, we predicted that activation in hippocampus and vmPFC would increase after an implicit sequence boundary (Kim et al. 2014, 2017) indicative of prediction error, as measured by a linear contrast between activation after a boundary (first object in a different-environment triplet) relative to before the boundary (third object in a same-environment triplet). Consistent with our prediction, we observed an increase in vmPFC activation (MNI coordinates (mm): x, y, z = −1, 34, −16; Fig. 4C) after an implicit sequence boundary. However, this effect was not significant in hippocampus (for whole-brain results, see Fig. S3A and Table S1).

We then further interrogated whether uncertainty signaling was reflected in individuals’ behavioral sensitivity at the implicit sequence boundaries. To examine this hypothesis, we examined whether the difference in activation across boundaries corresponded to the slowing of response times after a boundary. Using the difference in preference response time before and after a sequence boundary as a parametric regressor in the general linear model, we found that participants who showed greater behavioral slowing at implicit sequence boundaries also showed increased anterior hippocampal activation after a boundary (MNI coordinates (mm): x, y, z = 26, −10, −25; Fig. 4D; for whole-brain results, see Fig. S3B and Table S1). Thus, while anterior hippocampus did not show an overall increase in response after a boundary, it did track individuals’ behavioral sensitivity to those boundaries.

Hippocampal boundary sensitivity relates to environment reinstatement in anterior parahippocampal cortex

We further interrogated how hippocampus and vmPFC activation and connectivity related to predictive reinstatement of environment information in anterior PHC. Specifically, we hypothesized that a decrease in predictive reinstatement of environment information across the transition from a same to different environment triplet would be associated with increased uncertainty signaling in hippocampus and vmPFC (Fig. 3D). To test this hypothesis, we calculated the change in reactivation evidence within anterior PHC before and after a sequence boundary. This difference score was then included as parametric regressor in the univariate model assessing the change in activation across the same boundary. We found that participants with a greater decrease in context reinstatement at implicit sequence boundaries showed a corresponding increase in hippocampal activation after the boundary (MNI coordinates (mm): x, y, z = 22, −36, 0; Fig. 4E). We did not find a significant relationship between the change in context reinstatement at implicit boundaries and neural responses in vmPFC.

Decreased hippocampal-ventromedial prefrontal cortex functional coupling at implicit sequence boundaries

Electrophysiological work in rodents previously demonstrated bi-directional connections between hippocampus and vmPFC drive decisions that differ based on spatial context (Navawongse and Eichenbaum 2013; Place et al. 2016; Yu et al. 2018). These studies have shown that contextual information represented by the hippocampus and surrounding medial temporal lobe is sent to the vmPFC to help guide reactivation of the appropriate actions to take in a given context (Place et al. 2016; Wikenheiser et al. 2017). When factored together with these rodent findings, the present evidence that environment reactivation decreases after an implicit boundary suggests that hippocampal-vmPFC functional connectivity should decrease after an implicit sequence boundary. In other words, after a boundary, predictions from the medial temporal lobe were absent and thus not communicated to vmPFC (Fig. 3E).

We examined this hypothesis using a psychophysiological interaction (PPI) approach, in which the hippocampus and vmPFC regions showing cognitive map representations on day 1 were each used as seeds in PPI analyses. This analysis was used to determine how hippocampal-vmPFC connectivity was altered after an implicit sequence boundary, as measured by a linear contrast between activation before a boundary (third object in a same-environment triplet) relative to after the boundary (first object in a different-environment triplet). We further included the within-participant behavioral sensitivity to implicit sequence boundaries (the average difference in response time across a boundary) in the model as a parametric regressor. When hippocampus served as the seed region, we found a significant cluster in bilateral vmPFC (MNI coordinates (mm): x, y, z = 1, 8, −14; Fig. 4G) where coactivation with hippocampus decreased after the transition between a same- and different-environment triplet for the participants who showed the greatest behavioral sensitivity to implicit sequence boundaries. Similarly, when vmPFC was the seed, we identified a region of left hippocampus (MNI coordinates (mm): x, y, z = −25, −24, −13; Fig. 4F) where connectivity to vmPFC was reduced after a boundary. Our results indicate that functional decoupling between hippocampus and vmPFC further track participants’ behavior at implicit sequence boundaries.

Change in predictive reactivation across implicit sequence boundaries tracks hippocampal-ventromedial prefrontal cortex functional decoupling

Given prior electrophysiological work demonstrating that hippocampus sends contextual information to vmPFC (Place et al. 2016; Wikenheiser et al. 2017), we might expect that the buildup of contextual expectations in the incidental sequence task may modulate functional decoupling at event boundaries when hippocampus is the seed region. To test this hypothesis, the average anterior PHC environment reactivation for the third object presented within a same-environment triplet was included in the model as a within-participant parametric regressor of interest. When hippocampus served as the seed, we found a significant cluster in bilateral vmPFC following small volume correction (MNI coordinates (mm): x, y, z = 6, 44, −7), indicating that hippocampal functional decoupling with the vmPFC across an event boundary was modulated by predictive reinstatement of environment information immediately preceding a sequence boundary. Moreover, greater hippocampal-vmPFC functional decoupling modulated was significantly correlated with the decrease in contextual reactivation across the event boundary (r₍₂₆₎ = 0.588, p = 0.001; Fig. 4H), indicating that greater functional decoupling between hippocampus and vmPFC was linked with a greater decrease in contextual reactivation across the sequence boundary. While prior work indicates that hippocampus sends contextual information to vmPFC (Place et al. 2016), our results demonstrate that when there is no context to feed the hippocampus decouples functionally with vmPFC.

Changes in hippocampal activation and hippocampal-ventromedial prefrontal cortex connectivity show the strongest relationship to behavioral boundary sensitivity

Our results demonstrate that participants were slower to respond to an object across implicit sequence boundaries, suggesting that spatial knowledge influences how individuals segment information in time. Furthermore, several of our neural measures were related to the slowing of behavioral responses at implicit sequence boundaries. Hence, we wanted to examine which of these neural factors (e.g. hippocampal cognitive map formation, hippocampal boundary sensitivity, and hippocampal-vmPFC functional decoupling) explained the most variance on behavioral slowing at event boundaries. Using a multiple regression approach, we found that both increased hippocampal response and hippocampal-vmPFC decoupling after implicit sequence boundaries were significant predictors of response time slowing (hippocampal boundary sensitivity: ß = 0.456, t₍₂₆₎ = 2.783, p = 0.011; hippocampal functional decoupling: ß = 0.358, t₍₂₆₎ = 2.322, p = 0.030; vmPFC functional decoupling: ß = 0.129, t₍₂₆₎ = 0.763, p = 0.454; hippocampal cognitive map: ß = 0.274, t₍₂₆₎ = 1.841, p = 0.079; overall model fit: R² = 0.524, F_(4,22) = 6.063, p = 0.002, η²_p = 0.542), with the overall coherence of the initially formed hippocampal cognitive map trending as an additional predictor of decision speed.

Discussion

Consistent with emerging rodent (McKenzie et al. 2014; Knudsen and Wallis 2021; Wikenheiser et al. 2021) and human work (Tavares et al. 2015; Constantinescu et al. 2016; Deuker et al. 2016; Mack et al. 2020; Park et al. 2020), we quantified formation of structured cognitive maps within hippocampus, vmPFC, and RSC, further demonstrating how those maps alter object representations to augment their spatial similarities and differences (Deshmukh and Knierim 2013). Yet, the fundamental importance of cognitive maps is their flexibility. While theoretically proposed (Wikenheiser and Schoenbaum 2016; Epstein et al. 2017; Stachenfeld et al. 2017), cognitive map flexibility has only, to the best of our knowledge, been demonstrated neurally within a single cognitive domain. Rodent electrophysiology has shown that context-specific experiences become hierarchically organized in hippocampus and guide behavior in similar spatial contexts (McKenzie et al. 2014; Baraduc et al. 2019). Here, we provide representational evidence for generalization of knowledge across two domains of experience—spatial and temporal processing. We show that spatial knowledge formed one day was used predictively in a subsequent temporal task 24 hours later, biasing behavioral and neural response when participants “crossed” a boundary during the temporal sequence that was derived from the spatial learning task. When viewing sequences of objects previously located within the same spatial environment, participants predictively reactivated the common spatial context, even though task demands did not require it. Predictive reinstatement decreased when the sequence transitioned to an object from a different environment. This decrease may reflect greater predictive uncertainty at the implicit boundary, which was further accompanied by slowed decision making. In the absence of predictive reinstatement, hippocampal-vmPFC coupling decreased, and activation in both regions increased, further tracking the increased uncertainty at boundaries. Collectively, these findings show how medial temporal lobe regions work in concert with vmPFC to establish event boundaries based on previously acquired knowledge of spatial regularities.

Classic theories of hippocampal and medial temporal lobe function (Eichenbaum et al. 1994; Eichenbaum 1997) posit that representations are flexible—what is learned in one circumstance can be readily generalized to new situations, including in different domains of experience. Such flexibility is in contrast to more rigid forms of procedural learning that do not rely on hippocampus or medial temporal lobe structures (Eichenbaum and Cohen 2008). Despite the long-standing impact of these theories, there is limited evidence at the representational level for how hippocampus and surrounding medial temporal regions guide expression of knowledge across experiential domains. An earlier imaging study (Kumaran et al. 2009), showed that hippocampus was uniquely engaged when participants had to generalize a concept to a new problem set. Animal electrophysiology has further shown that hippocampal representations coding context-specific object reward values map to new objects experienced in the same contexts (McKenzie et al. 2014), a form of within-domain generalization. Yet, despite the significance of these findings for our understanding of hippocampal function, they do not address the central tenet of cross-domain flexibility ascribed to the hippocampus.

As we have expanded of our understanding of how associative memory structures are formed, the term cognitive map has encompassed more forms of associative memory. Cognitive maps can represent knowledge from at a wide range of scales (Behrens et al. 2018), ranging from precise mapping of spatial metrics (O’Keefe and Nadel 1979; Hafting et al. 2005) to abstract categorical spaces (Mack et al. 2016; Morton et al. 2020). Here, we use the term cognitive map given that our spatial learning task is highly similar to the rodent spatial memory tasks from which this term originally emerged (Tolman 1948; O’Keefe and Nadel 1979; Johnson and Redish 2007; Preston and Eichenbaum 2013) as well as other more recent findings using object (or landmark) location learning to study cognitive maps (Brown et al. 2016; Deuker et al. 2016). Regardless as to what one’s preferred definition of the term cognitive map may be, our results show how associative knowledge formed in one task domain influences processing and decision making in a distinct task context.

It is well understood that there are hippocampal neurons that code both spatial (O’Keefe and Nadel 1979; Moser et al. 2008) and temporal (Dragoi and Buzsáki 2006; Pastalkova et al. 2008; MacDonald et al. 2011) aspects of events. Furthermore, separate studies have shown that hippocampal responses are sensitive to both spatial and temporal boundaries (Radvansky and Zacks 2017; Brunec et al. 2018; Alexander et al. 2020). An open question is how hippocampal spatial and temporal representations may influence one another (Behrens et al. 2018). Here, we hypothesized that hippocampal representations differentiating spatial context (at the level of environment) would bias temporal processing. Consistent with this hypothesis, individuals who formed the most coherent spatial maps were also those who showed greater behavioral sensitivity to implicit sequence boundaries in the sequence task, demonstrating how spatial knowledge biases behavior in nonspatial contexts. Furthermore, anterior PHC representations of spatial context were predictively reactivated during the sequence task, and when such predictive signals were reduced at implicit sequence boundaries, neural response in hippocampus, as well as vmPFC, was altered. Boundaries in space that influence temporal decision making have been theoretically predicted (Radvansky and Zacks 2017; Brunec et al. 2018; Zacks 2020) and have some behavioral support (Brunec et al. 2020). Here, we show that medial temporal lobe representations of spatial experience were brought to bear in a predictive manner during the later sequential decision making task, confirming the long theorized role of medial temporal lobe representational flexibility (Eichenbaum et al. 1994; Eichenbaum 1997).

Our results provide further empirical evidence that hippocampus and vmPFC act in concert to use prior knowledge during memory-guided decision making (Place et al. 2016; Das and Menon 2022). Theories and empirical data from rodents indicate that hippocampus sends contextual information through its direct connections with vmPFC (Barbas and Blatt 1995; Eichenbaum et al. 2007); vmPFC then biases hippocampal-mediated reactivation toward behaviorally relevant memories as individuals make choices (Preston and Eichenbaum 2013; Place et al. 2016). To date, such evidence has focused on decision making within a single domain of experience, typically knowledge of rewards in a spatial context (Place et al. 2016). What is not well understood is how hippocampus and vmPFC interact during cross-domain generalization. Our results show that reinstatement of context information in the medial temporal lobe builds up when a sequence of items share a spatial context and decreases when they do not. When predictive reinstatement is reduced at boundaries between objects that do not share a spatial environment, hippocampal-vmPFC coupling decreases in proportion to the drop off in predictive reinstatement. This finding is consistent with the theoretical prediction that when memory-based predictions are absent, or less reliable, hippocampal interactions with vmPFC should be reduced (Preston and Eichenbaum 2013; Rajasethupathy et al. 2015). Importantly, while several of neural measures were correlated with response slowing at implicit sequence boundaries, changes in hippocampal-vmPFC coupling at boundaries were most predictive of behavior, suggesting a unique role for hippocampal-vmPFC interactions in generalization.

Increased hippocampal and vmPFC responses after implicit sequence boundaries may reflect their purported roles in signaling memory-based prediction errors (Garrido et al. 2015; Schapiro et al. 2016; Baldassano et al. 2017; Clewett et al. 2019). Theoretical models propose that event boundaries are created when predictions based on prior experience are not met (Gershman et al. 2013; Radvansky and Zacks 2017; Clewett et al. 2019; Zacks 2020). Neuroimaging studies have previously observed enhanced hippocampal and vmPFC responses at event boundaries (Ezzyat and Davachi 2011; DuBrow and Davachi 2014; Sols et al. 2017), and vmPFC representations are sensitive to the temporal order of common events (Baldassano et al. 2017), such as the sequence of actions when visiting a restaurant. Here, we show that there is a buildup in contextual expectations across successive presentations of same-environment objects, which are diminished at triplet boundaries. In the context of our task, expectations derived from spatial experience may prime expectations during the sequence task. The buildup of contextual reactivation during a same environment triplet may create the expectation that the next object would be another from the same environment. When an object learned in a different environment appears, that prediction is violated, resulting in increased hippocampal and vmPFC response at the transition between same- and different-environment triplets.

Alternatively, enhanced hippocampal and vmPFC responses after an implicit boundary might be interpreted as signaling uncertainty. As we observed in the current study, predictive reinstatement of contextual information is reduced after an implicit boundary, increasing uncertainty about what may occur next in the sequence. Prior studies have shown hippocampus (Harrison et al. 2006; Davis et al. 2012; Rigoli et al. 2019) and vmPFC (Kim et al. 2014, 2017; Garrido et al. 2015) responses increase in proportion to the degree of uncertainty associated with memory decisions. Here, hippocampal response after a boundary was conversely related to predictive reinstatement; larger decreases in prediction across implicit sequence boundaries were associated with greater hippocampal response after the boundary. Regardless of the interpretation of hippocampal and vmPFC responses as prediction errors or uncertainty signals, they do suggest a role for these regions in forming event boundaries, which indicate where associative relationships among sequentially presented content changes.

A further interpretation of environmental reactivation in the incidental sequence task may be that participants were retrieving a more abstract representation of context, such as a categorical context label associated with each object. One point that might argue against this possibility is that the spatial learning task not only required participants to remember a categorical tag about the object-environment relationship, but also further required that participants remember the precise location within the environment. In other words, a categorical representation alone would not support the successful spatial memory acquisition that we observed across learning in our participants. Additionally, data from our debriefing report suggests that participants do not predominately rely on a contextual labeling strategy. When queried after the incidental sequence task, participants did not explicitly recognize that objects were arranged in triplets, according to environment. That is, people did not explicitly recognize that three objects from the same environment occurred in succession. This anecdotal evidence suggests that an explicit categorical label was not necessarily accessible during the implicit sequence task. However, even if participants were reactivating categorical (rather than more detailed environmental) information, our findings still demonstrate how abstract spatial knowledge influences temporal processing on the next day.

Specifically, our findings reveal how predictions derived from spatial experience bias processing and behavior during a temporal sequence task, resulting in increased uncertainty and decision speed at implicit sequence boundaries. While this finding suggests parallels to the growing literature on event segmentation (Radvansky and Zacks 2017; Franklin et al. 2020), there are some notable differences between our approach and typical event segmentation tasks. In the present study, we did not probe directly when participants consciously perceived boundaries during the sequential object presentations as is typical in event segmentation tasks (Kurby and Zacks 2008; Schapiro et al. 2013; Clewett and Davachi 2017). In fact, our postexperiment survey indicated that participants were not aware of event boundaries as they made preference judgments. Our findings indicate that pre-existing knowledge may automatically prime expectations in new task settings, impacting both neural processing and behavior, even if participants are not consciously aware of such biases. Future studies could test how such automatic predictions might be further linked to the conscious experience of event boundaries during continuous experience.

In the future, our findings might also be extended to study how consolidation impacts knowledge generalization. Here, representations of spatial experience were brought to bear in a predictive manner during the sequential decision-making task 24 hours after spatial learning occurred. This 24-hour delay was included for practical reasons because of the length of the paradigm. While we do not have the proper controls in the present paradigm to isolate the impact of consolidation windows on generalization, the fact that we find vmPFC circuits involved in knowledge generalization after a temporal delay is perhaps not surprising given other data implicating this region in restructuring knowledge during consolidation (Maguire 2014). For instance, when participants learn about overlapping visual associations, medial PFC does not link related memories together immediately but only shows evidence of memory integration after a 24-hour delay (Tompary and Davachi 2017). Our paradigm could be adapted in the future to formally assess how both hippocampal and vmPFC cognitive maps formed on day 1 change across consolidation, for instance by quantifying whether differentiation between environments increases over time, as well as how consolidation-related changes in knowledge organization impact generalization.

Future studies may also focus on the RSC role in generalizing associations to support prediction and decision making. Initially, we hypothesized that distortions in RSC based on spatial experience may guide how individuals process information in a subsequent temporal task. The RSC has been implicated in representing both spatial and nonspatial associations (Aminoff et al. 2007; Bar et al. 2008; Pudhiyidath et al. 2021) and thus may have been a key region in reinstating context in the incidental sequence task. However, while we found that spatial distortions were evident in RSC, such contextual associations were less invovled in the generalization process. More data may be required to understand how RSC representations may be generalized to influence processing across cognitive domains.

Taken together, our findings show how complex knowledge of spatial relationships formed during navigation exert a persistent influence on behavior and neural response during a nonspatial task, guiding how individuals process information the temporal sequence task. Over the years, there has been substantial debate about whether hippocampus represents spatial experience or memory more generally (Cohen and Eichenbaum 1991; Eichenbaum et al. 1999; Burgess et al. 2002; Eichenbaum and Cohen 2014; Schiller et al. 2015). Our findings speak to this debate by indicating that hippocampal spatial representations generalize to nonspatial tasks and are used as a framework for processing and interpreting nonspatial events. Thus, hippocampal representations are flexible and expressed across domains of experience—here from space to time—to support decision making, consistent with a general role for hippocampus in memory (Zeidman and Maguire 2016; Stachenfeld et al. 2017; Behrens et al. 2018). Furthermore, our findings provide empirical support for theoretical models of hippocampal-vmPFC interactions during memory-based decision making, building upon prior work in rodent models (Rajasethupathy et al. 2015; Place et al. 2016; Wikenheiser et al. 2017; Zhou et al. 2019). Our data provide direct evidence for how hippocampus and vmPFC interact to extend knowledge about environmental regularities formed on one day to influence decision making on the next.

CRediT authors statement

Katherine Sherrill (Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Resources, Software, Visualization, Writing – original draft, Writing – review & editing), Robert J. Molitor (Methodology, Software, Writing – review & editing), Ata Karagoz (Data curation), Manasa Atyam (Data curation), Michael Mack (Methodology, Writing – review & editing), Alison Preston (Conceptualization, Funding acquisition, Methodology, Resources, Supervision, Writing – review & editing)

Funding

This research was supported by the National Institute of Mental Health (R01 MH100121–01 to A.R.P.) and the National Institute of Neurological Disorders and Stroke (National Research Service Award F32 NS098808 to K.R.S.) of the National Institutes of Health.

Data and code availability

De-identified data is available upon request to the corresponding author. The Unity Build and relevant code are publicly available on GitHub.

Conflict of interest statementa

None declared.