Reinstatement of Event Details during Episodic Simulation in the Hippocampus

Abstract

According to the constructive episodic simulation hypothesis, episodic simulation (i.e., imagining specific novel future episodes) draws on some of the same neurocognitive processes that support episodic memory (i.e., recalling specific past episodes). Episodic retrieval supports the ability to simulate future experiences by providing access to episodic details (e.g., the people and locations that comprise memories) that can be recombined in new ways. In the current functional neuroimaging study, we test this hypothesis by examining whether the hippocampus, a region implicated in the reinstatement of episodic information during memory, supports reinstatement of episodic information during simulation. Employing a multivoxel pattern similarity analysis, we interrogated the similarity between hippocampal neural patterns during memory and simulation at the level of individual event details. Our findings indicate that the hippocampus supports the reinstatement of detail-specific information from episodic memory during simulation, with the level of reinstatement contributing to the subjective experience of simulated details.

Introduction

Episodic memory refers to the ability to recall specific past episodes (Tulving 2002). Because episodic memory is prone to error and distortion (e.g., Loftus 2003; Schacter and Slotnick 2004), it has been characterized as a constructive process whereby the individual features of a prior event (e.g., people, locations, and objects) are linked together in a coherent episode at the time of retrieval (e.g., Bartlett 1932; Schacter et al. 1998). From a functional–adaptive perspective (e.g., Howe 2011; Schacter et al. 2012), the constructive nature of episodic memory provides adaptive value to other cognitive abilities that draw on similar processes. For example, according to the “constructive episodic simulation hypothesis” (Schacter and Addis 2007, 2019), an important function of episodic memory is to support the episodic simulation of future events—the ability to draw on elements of past experiences in order to construct novel episodes and imagine what might happen in the future—by providing access to the episodic details that comprise memories (e.g., people, locations, and objects) that can be flexibly recombined in novel ways.

Over the past decade, a growing number of findings have provided support for the constructive episodic simulation hypothesis by demonstrating overlap in the cognitive processes and neural substrates involved in episodic memory retrieval and simulation (for reviews, see Schacter et al. 2012, 2017). Benoit and Schacter (2015) conducted a meta-analysis of functional magnetic resonance imaging (fMRI) studies reporting activity associated with episodic memory retrieval and simulation. They identified a common “core network” of neural regions, comprising the lateral parietal and temporal cortex and hippocampus, among other regions, that are jointly recruited during episodic retrieval and simulation. Evidence of a core network has been taken as support for the constructive episodic simulation hypothesis because it is consistent with common constructive processes during episodic retrieval and simulation (see also, Buckner and Carroll 2007; Hassabis and Maguire 2007).

Although the prior findings suggest a strong relationship between episodic retrieval and simulation, a critical tenet of the constructive episodic simulation hypothesis remains untested. If episodic retrieval allows access to memorial details from which novel future episodes are constructed, one would expect episodic retrieval and simulation to be linked at the level of individual event details (Schacter and Addis 2007, 2019). Some evidence has suggested that memories and simulations share similar content (e.g., D’Argembeau and Van der Linden 2004, 2012; Szpunar and McDermott 2008; Thakral et al. 2019a). For example, in a prior study (Thakral et al. 2019a), we had participants recall past episodes each comprising two event details, a personally familiar location and person (see Fig. 1A, top). Participants also simulated novel future episodes using recombined pairs of person and location details taken from different recalled episodes (see Fig. 1A, bottom). For each detail during recall and simulation, participants rated the vividness with which they experienced each location and person. We reasoned that if content associated with episodic details is shared across memory and simulation, the subjective experience of details during simulation (i.e., vividness) should covary with the experience associated with the details during the original episodic memory. In line with our prediction, the vividness for person and location details during memory covaried with the vividness ratings for the same details when part of a novel future episode (Thakral et al. 2019a). These behavioral findings suggest that past and future episodes share similar episodic content. Outside of this behavioral evidence, no study has formally tested whether episodic simulation entails the sampling of individual episodic details from memory and whether this relationship is supported by a common neural mechanism, as maintained by theoretical perspectives that emphasize mechanistic overlap of episodic retrieval and simulation (e.g., DeBrigard 2014; Michaelian 2016; Addis 2018).

Figure 1.

A. Representative trials from the episodic memory and simulation tasks. On each trial of the episodic memory task (top), participants were presented a memory cue that they had generated from an initial session (e.g., “Pictures with seagulls”). Each memory comprised two details, a person and a location (e.g., “Janine W.” and “Ocean beach pier”). Participants were instructed to recall the original episode focusing on the person and location comprising the memory, until the onset of the rating screens. On each trial of the episodic simulation task (bottom), participants were presented with recombined person and location details across separate memories (e.g., “Ocean beach pier” taken from the memory “Pictures with seagulls”). Participants were instructed to continually simulate a novel future episode that focused on and integrated the cued details until the onset of the rating screens. Durations are shown on the bottom of each frame. B. Overview of the pattern similarity analysis. We calculated for each participant the similarity between memory and simulation trials when matched as a function of a shared event detail (e.g., the shared location of “Ocean beach pier”; see solid arrow). To assess the detail specificity of reinstatement, these matching correlations were compared to their mismatching counterparts (i.e., where each memorial detail was correlated to all other simulations not containing that detail; see dashed arrows). Both matching and mismatching correlations were computed for details as a function of high and low vividness of that detail during episodic simulation (see full text for details).

Of direct relevance to this question are neural models of episodic memory (e.g., Marr 1971; Norman and O’Reilly 2003; Rugg et al. 2015). According to such models, successful episodic retrieval depends on “cortical reinstatement” (i.e., the reactivation of neural processes/representations that occurred during the initial encoding of an event during the time of retrieval). These models stipulate that cortical reinstatement provides access to the qualitative information that comprises episodic content. In these models, the hippocampus plays a privileged role in both binding event details during encoding and subsequently facilitating their reinstatement during successful episodic retrieval.

A number of fMRI studies have supported these models of episodic memory and have identified cortical reinstatement effects (for reviews, see Rissman and Wagner 2012; Rugg et al. 2015; Xue 2018). For example, studies employing multivoxel pattern analyses (MVPA) have identified reinstatement of individual event features from encoding during retrieval (e.g., Kuhl and Chun 2014; Wing et al. 2015; Chen et al. 2017). Of importance, some studies have directly linked the level of reinstatement with hippocampal activity (e.g., Ritchey et al. 2013; Bosch et al. 2014; Gordon et al. 2014a; Wing et al. 2015). These findings lend critical support to models of episodic memory highlighting the role of the hippocampus in mediating the reinstatement of information from encoding (for evidence that retrieved content can be decoded directly from hippocampal patterns of neural activity, see Chadwick et al. 2010, 2011; Tompary et al. 2016; for a discussion of these studies, see Thakral et al. 2017b).

The aim of the current fMRI study was to examine whether the hippocampus, a region implicated in the reinstatement of episodic information during successful retrieval, also supports the reinstatement of individual event details during episodic simulation. We capitalized on the above well-characterized cortical reinstatement framework of episodic memory in order to provide a novel test of the constructive episodic simulation hypothesis (Schacter and Addis 2007, 2019) with the adoption of a well-validated paradigm from a prior study (Thakral et al. 2019a). First, we conducted a behavioral analysis to replicate our previous findings (Thakral et al. 2019a) and examined whether the vividness of specific retrieved memorial details covaries with the vividness of those same details during episodic simulation. Critically, we also employed MVPA and interrogated the similarity between patterns of hippocampal neural activity across memory and simulation at the level of individual event details (see Fig. 1B). We calculated for each participant the similarity between memory and simulation trials when matched as a function of a shared event detail (i.e., matching correlations; see Fig. 1B, solid line). Critically, to assess the specificity of reinstatement, these matching correlations were compared to their mismatching counterparts (i.e., where each memorial detail was correlated to all other simulations not containing that detail; see Fig. 1B, dashed lines). Both matching and mismatching correlations were computed for details as a function of high and low vividness during episodic simulation. By examining pattern similarity as a function of vividness during simulation, we were able to test whether the reinstatement of information from memory contributes to successful simulation (i.e., in the form of greater reinstatement for simulated details that are higher in vividness). This procedure not only provides a direct test of the constructive episodic simulation hypothesis, which states that episodic details from “memory” are used (i.e., reinstated) when constructing episodic simulations, but also parallels prior pattern similarity studies of episodic memory that have examined reinstatement from encoding during retrieval, with pattern similarity examined as a function of memorial performance (e.g., Ritchey et al. 2013; Kuhl and Chun 2014; Wing et al. 2015).

If the hippocampus supports the reinstatement of individual details from episodic memory during episodic simulation, then similarity in the hippocampus should be greater for those details that match relative to those that mismatch. If this effect covaries with the vividness with which the details during episodic simulation are subjectively experienced (i.e., greater similarity for high relative to low vivid details), such a pattern would suggest that the strength of hippocampal reinstatement from memory contributes to the phenomenological experience of episodic information during simulation. This finding would be the first to indicate that a critical neural mechanism of episodic memory retrieval (i.e., hippocampal-mediated reinstatement) supports episodic simulation, thereby providing a stronger basis for mechanistic interpretations of how episodic retrieval supports simulations of novel personal events.

Relevant to the role of the hippocampus in mediating reinstatement of event details during episodic simulation, recent studies have identified a temporal dissociation within the hippocampus during episodic simulation (Thakral et al. 2017c, 2017d). In these studies, we demonstrated that relative to other processes, such as encoding and binding, retrieval-related processing during episodic simulation occurs transiently (see also Vilberg and Rugg 2012, 2014). In light of these findings, in the current study, we examined memory–simulation similarity at different timepoints during simulation, expecting that pattern similarity effects within the hippocampus would largely be present early relative to later in time (e.g., Thakral et al. 2017c, 2017d). Such a finding would support our previous findings demonstrating that retrieval-related processing within the hippocampus is a transient process during simulation.

As noted above, the present study builds on an extensive prior literature that has linked the hippocampus with various manipulations of event detail and phenomenology during episodic memory and simulation (e.g., Addis et al. 2011; Martin et al. 2011; Gaesser et al. 2013; Thakral et al. 2017c, 2017d; Palombo et al. 2018; for reviews and discussion, see Addis and Schacter 2012; Sheldon and Levine 2016). In contrast to this study, these prior studies are limited because they examined hippocampal involvement during episodic memory and/or simulation at the “event-level” (e.g., by comparing remembered and imagined events in terms of their level of specificity or temporal orientation; Addis and Schacter 2008) and have not probed the memory–simulation link at the level of the specific details comprising remembered and imagined episodes. Therefore, these findings do not speak to the aim of the current experiment to assess whether the hippocampus supports processing during episodic simulation by reinstating specific details from previously established episodic memories, which is now possible with this new methodological approach. Such a finding would provide novel support for a critical tenet of the constructive episodic simulation hypothesis (see above) and deepen our mechanistic understanding regarding the construction of imagined events.

In addition to our primary region of interest (ROI), the hippocampus, we sought to investigate how different brain regions are involved in the specificity and subjective experience of event details during episodic simulation. Specifically, we aimed to identify regions that would demonstrate a possible dissociation with the hippocampus. To this end, we interrogated pattern similarity within two additional core network ROIs (Benoit and Schacter 2015): the left middle temporal gyrus/anterior temporal lobe (MTG/ATL) and left angular gyrus (AG). These regions, in contrast to the hippocampus, have been strongly associated with episodic and semantic memory processing (for reviews, see Binder et al. 2009; Binder and Desai 2011; Kim 2016). For example, these regions have been suggested to support the retrieval and representation of “personal semantics” (i.e., the generalized facts that define personally-relevant stimuli; Renoult et al. 2012). In support of this idea are recent MVPA studies demonstrating that the MTG/ATL is associated with the retrieval and representation of social knowledge (see, Wang et al. 2017; see also, Graham et al. 2003; for a review, see, Olson et al. 2013). With respect to the AG, although prior MVPA studies have indicated that episodic content can be decoded from this region (e.g., Kuhl et al. 2013; Kuhl and Chun 2014; Bonnici et al. 2016; Chen et al. 2017; Thakral et al. 2017b, 2019b), neural patterning within the AG is largely insensitive to the subjective experience of episodic content (Kuhl and Chun 2014; Thakral et al. 2017b, 2019b; but see, Bonnici et al. 2016). These findings have been taken to suggest that the role of the AG may not be selective to representing content of an episodic experience specifically (Rugg and King 2018). In light of this latter evidence, if the MTG/ATL and AG support the retrieval of autobiographical/semantic information associated with individual event details, pattern similarity within these regions should be greater for matching relative to mismatching details but should not be sensitive to the subjective experience of episodic simulation (cf., Wing et al. 2015). Alternatively, and akin to the hippocampus, these regions may be associated with the retrieval of detail information tied to specific past episodes and therefore covary with the subjective memorial experience from which the details originated.

In addition to examining pattern similarity within the three core network ROIs described above, we also examined a set of content-selective ROIs: the parahippocampal cortex (PHC) and dorsomedial prefrontal cortex (DMPFC). A prior study from our group (Benoit et al. 2014) revealed activation of the PHC and DMPFC during the imagination of familiar locations and people, respectively (see also, Hassabis et al. 2014; Szpunar et al. 2014). Here, we aimed to extend our prior investigations to test whether these regions would demonstrate sensitivity at the level of the individual detail (i.e., locations in the PHC and people in the DMPFC), again with the broader aim of providing a basis for a deeper mechanistic understanding of how episodic retrieval underpins simulations of novel personal events.

Materials and Methods

Participants

The experimental protocol was approved by the Institutional Review Board of Harvard University and informed consent was obtained prior to participation. Twenty-four undergraduate and graduate students from the local community consented to participate in the study. All participants were right-handed, native English speakers, had normal or corrected-to-normal vision, no history of neurological impairment, and were not currently taking any psychoactive medications. Three participants were excluded for having an incomplete data set or for task noncompliance. This final sample size of 21 (mean age of 20.6 years [range 18–29], 13 females) is identical to, and in some instances larger than, prior studies of episodic memory employing similar analytic approaches (i.e., MVPA see also, Ritchey et al. 2013; Kuhl and Chun 2014; Wing et al. 2015; Tompary et al. 2016). In brief, the study comprised two sessions. Session 1 was a pre-scan phase where episodic memories were collected from the participant. Session 2 was the scanning phase where the data from Session 1 (i.e., the episodic memories and recombined person and location details) were employed for the episodic memory and simulation tasks, respectively.

Experimental Procedure

Session 1: Stimuli Collection

In Session 1, participants came to the laboratory and were asked to recall 120 personal memories from the past 5 years. These memories had to be unique, in that they did not share locations or people. Each memory had to be of a personal experience (i.e., not an event they merely heard about) that was specific in time and place and that lasted only a few minutes to a few hours. Participants were also instructed to avoid events that blended into other similar events (e.g., a general scenario of going supermarket shopping vs. a particular occasion of doing so).

For each memory, participants entered into a spreadsheet a brief description of the event, which was later used by the experimenter to ensure that the memories provided were specific in time and place (i.e., episodic in nature). Participants specified the location where the event had occurred and a person of interest (other than themselves) who participated in the event. If there were multiple people at the event, participants were asked to choose the person of primary importance. Participants were instructed to be as specific as possible with respect to the location name (i.e., the location name should allow them to quickly and easily imagine the exact location, e.g., relative to “Boston apartment,” “kitchen of Boston apartment”) but to not include the person’s name in the location name (e.g., do not write “Bob’s apartment kitchen”). In addition to the brief event descriptions and person/location details, participants created a memory cue that would allow them to quickly and easily recall the memory (see Fig. 1A, top). The memory cue, which had to be as short as possible, could not include the location or the person’s name associated with the memory. The experimenter provided example memory descriptions, location/person details, and memory cues and also checked the first 10 memories to ensure task instructions had been understood.

Participants rated each memory, person, and location on a number of features, including familiarity. The familiarity rating was employed to replicate our earlier findings (Thakral et al. 2019a) and assess whether any behavioral effects attributable to vividness are above and beyond those attributed to familiarity. For familiarity, participants rated their knowledge of/familiarity with the person or location in everyday life, ranging from not very familiar to very familiar on a scale of 1–5 (see also, Benoit et al. 2014). As noted in the Introduction, in our prior behavioral study, we found that the vividness of individual simulated details is correlated with the vividness of those same details in the episodic memories from which they were drawn (with these ratings collected in Session 2, see below). In that study, we also assessed whether this correlation as a function of vividness was attributable, in part, to the familiarity of the detail. Therefore, as a replication of our prior study, we investigated whether after accounting for the level of familiarity, there would be evidence for a significant positive correlation between the vividness of remembered and imagined person and place details.

Prior to Session 2, the 120 memory cue–location–person triplets were randomly sorted. After randomization, 84 triplets were chosen for the experiment. There were a total of 8 fMRI scanning runs (4 memory runs and 4 simulation runs), each with 21 trials. Critically, the design required that each memory run had a corresponding simulation run, and the stimuli for the two runs were derived from the same set of 21 memory cue–location–person triplets. Specifically, for the memory run, only the memory cues were used as stimuli (to elicit retrieval of the original memories). For the corresponding simulation run, however, 21 novel location–person pairings were used as stimuli (to elicit novel simulations), created by randomly recombining the location and person details from the same 21 triplets as used in the memory run. Although the order of the runs was counterbalanced across participants (i.e., odd runs were selected to be memory and even runs were selected to be simulation, or vice versa), the two corresponding memory and simulation runs (i.e., where stimuli were derived from the same set of memory cue-person-location triplets) were always presented in succession. We adopted this method to equate the delay between recalling a given memory and the simulation of the novel recombination of associated location and person details.

As described above and illustrated in Figure 1, the stimuli for the memory runs comprised only the memory cue and did not include the person or location names used in the corresponding simulation runs. We explicitly chose to avoid perceptual overlap in order to ensure that any similarity observed across patterns of brain activity during memory and simulation trials reflected information retrieved from memory in response to the cue and not elicited by the cue itself. Note also that this design reduces shared perceptual processing across memory and simulation trials, which could inflate any across-trial correlations (e.g., common cue processing, see also, Kuhl and Chun 2014; Wing et al. 2015).

Session 2: Experimental Phase

Participants returned for the fMRI portion of the study (Session 2) between 2 and 7 days after Session 1. Before beginning Session 2, participants were familiarized with the cue–location–detail triplets that they had generated in Session 1. During Session 2, participants completed two episodic tasks: an episodic memory task and an episodic future simulation task. On each trial of the episodic memory task, participants were presented with a memory cue generated during Session 1 (see Fig. 1A, top). The task was to silently remember the same specific experience generated in Session 1 as quickly and vividly as possible once the cue appeared on the screen, including remembering how the person and location were involved. They were further instructed to actively remember for the entire duration of the trial the other person’s actions (e.g., what they were saying or doing) and what that location looked like from a first-person perspective (i.e., through their own eyes and not from an external vantage point) as if they were re-experiencing the prior episode. On each trial of the episodic future simulation task, participants were presented with a recombined person–location pair generated from Session 1 as described above (see Fig. 1A, bottom). The task was to silently imagine as vividly as possible a specific and novel future episode where they were interacting with the given person in a manner specific to the given location. Participants were warned that, for some combinations of details, it may be more difficult to imagine interacting with the person at the given location, but to nevertheless always try to imagine the novel episode as quickly as possible. Analogous to the episodic memory task, participants were instructed to continually imagine for the whole duration of the trial, to do so from a first-person perspective, and to restrict the simulation to the specified person and location details.

In addition to performing 21 memory or 21 simulation tasks per run, participants also completed 3 trials of a non-episodic sentence control task per run (for full details, see Thakral et al. 2017c). In brief, participants were shown two object nouns (e.g., pencil and hammer) and were instructed to silently put them in a sentence focusing on the physical sizes of the objects from smallest to largest. Once the sentence was created, participants elaborated on the representation of the nouns, generating as much detail about the meaning of the nouns (including visually imagining the objects). The sentence task was included to serve as a non-episodic baseline task relative to the memory and simulation tasks, as it involves assembling, maintaining, and integrating non-episodic information in response to a cue (see also, Benoit et al. 2014; Thakral et al. 2017c). For each run, three sentence trials were pseudo-randomly placed within each third of the run.

Every trial had a similar structure, irrespective of task. First, one of three task instruction cues was shown for 2 s (“Recall Memory,” “Novel Simulation,” or “Create Sentence”). Second, the relevant task stimuli were shown for 10 s (i.e., memory cues, person/location names, or object words, respectively). Third, participants completed a series of ratings on the task they had just completed. For the episodic tasks, participants completed four ratings: (1) how difficult it was to remember/simulate the episode, (2) how vivid the remembered/simulated episode was, (3) how vivid the person in the memory/simulation was, and (4) how vivid the location of the memory/simulation was. For the non-episodic task, participants completed two ratings: (1) how difficult it was to create the sentence and (2) how vivid the sentence was (i.e., how much detail about the object words was generated). Each rating scale was 5 points, ranging from low to high, and shown for 2 s; participants were instructed to respond as quickly as possible without sacrificing accuracy. Finally, the trial ended with a jittered fixation period (i.e., inter-trial interval of 2, 4, or 6 s). All stimuli were presented on a black background in white 25-point Arial font. Participants responded to ratings using a button box in their right hand (no response was required during the 10 s task period). Stimuli were presented using the Cogent software package (http://www.vislab.ucl.ac.uk/cogent.php) as implemented in MATLAB (The MathWorks, Natick, MA, USA).

Before beginning Session 2, participants completed two practice runs, comprising 5 trials each (1 sentence and 4 memory or simulation trials in each run). Following completion of the experiment, participants were debriefed about the experiment. As in our prior behavioral study (Thakral et al. 2019a), participants did not report using the repetition of the details as a way to complete the tasks (e.g., explicitly remembering the memory from a prior run to build the novel future episode for the simulation task). This observation suggests that the memory and simulation tasks were approached as independent.

In addition to completing the main memory/simulation runs, participants also completed one run of a functional localizer task during which they imagined familiar people and places in isolation (Benoit et al. 2014; see also, Hassabis et al. 2014). We collected a localizer to identify and extend our MVPA to a set of content-selective ROIs: PHC and DMPFC. The localizer comprised 40 trials: 20 people and 20 locations. The stimuli were selected from a subset of memories generated in Session 1 that were not used as stimuli in the main memory/simulation tasks. On each trial, participants were shown either a person or location name and were asked to vividly picture that detail in isolation (no more than three details of a given type were presented in succession). For person names, participants were instructed to vividly imagine the person and not to imagine themselves interacting with the person and not to imagine that person in any specific place. For location names, participants were instructed to concentrate on what the place looked like and not to picture themselves or familiar people at those places. Each detail was shown for 8 s. At the end of each trial, participants were asked to rate the vividness of each person or location on a 5-point scale ranging from low to high. Rating scales were shown for 2 s and the trial ended with a variable fixation period (2, 4, or 6 s).

Image Acquisition and Analysis

Functional and anatomic images were acquired on a 3 Tesla Siemens Prisma scanner equipped with a 32-channel head coil. Functional images were acquired with a multiband echo-planar imaging sequence (University of Minnesota C2P sequence: repetition time (TR) = 2 s, echo time = 30 ms, matrix size of 136 × 136, field-of-view = 204 mm, 84 slices [3 slices acquired simultaneously], 1.5 mm³ resolution, multiband factor of 3). The slices were autoaligned to an angle 20° toward coronal from anterior–posterior commissure alignment. For each memory/simulation run, 277 images were acquired, and for the localizer run, 288 images were acquired. Anatomic images were acquired with a magnetization-prepared rapid gradient echo sequence (1 mm³ resolution).

fMRI data were analyzed using both a univariate general linear model (GLM) and MVPA. Univariate analyses were conducted using Statistical Parametric Mapping (SPM12, Wellcome Department of Cognitive Neurology, London, UK). MVPA was conducted using the Princeton MVPA Toolbox (https://code.google.com/p/ princeton-mvpa-toolbox/) and custom MATLAB scripts.

Univariate Analyses

Functional image preprocessing included slice-time correction, two-pass spatial realignment, and normalization into Montreal Neurological Institute (MNI) space (images were not resampled). For univariate analyses, functional images were smoothed with a 3 mm full-width half-maximum Gaussian kernel. Anatomic images were normalized into MNI space using an analogous procedure to that employed for the functional images.

Univariate analyses for the memory/simulation data (concatenated across eight runs) and localizer data were conducted in a two-step mixed effects GLM. In the first step, neural activity associated with each event was modeled at event onset with a delta/stick function with a 0 s duration. The associated blood-oxygen-level-dependent (BOLD) response was modeled via convolution with a canonical hemodynamic response function yielding regressors in a GLM that modeled the BOLD response for each event. For the analyses of the memory/simulation data, there were three events of interest: episodic memory, episodic simulation, and non-episodic/sentence trials, corresponding to the onset of the task stimuli (i.e., memory cues, person/location names, or the object words, respectively). There were three additional events of no interest: the onset of task stimuli which participants failed to respond to, as well as the instruction cue period and for each rating within a trial (as with the events of interest, events of no interest were modeled with stick functions). For the localizer analysis, there were two events of interest: person and location trials (corresponding to the onset of person or location names, respectively) and two additional events of no interest: the onset of task stimuli for which participants failed to respond and the rating period. The design matrix for the memory/simulation and for the localizer analyses also included six regressors representing movement-related variance (three for rotation and three for rigid-body translation) and, for the memory/simulation design matrix, regressors modeling each run. An AR(1) model was used to estimate and correct for nonsphericity of the error covariance (Friston et al. 2002). Temporal smoothing was conducted before estimation of the parameter estimates using the default high-pass filter of 128 s in SPM12.

In the second step, parameter estimates for the events of interest and for each participant were entered into repeated measures ANOVA with participants modeled as a random effect. Unless otherwise noted, an individual threshold of P < 0.001 was combined with a cluster extent threshold of 19 voxels to yield a threshold corrected for multiple comparisons of P < 0.05 (Slotnick et al. 2003; Slotnick 2017, 2018; for description of the cluster extent computation, see also Thakral et al. 2017c). To identify the core network ROIs (see Introduction), we contrasted the episodic memory and simulation trials against the non-episodic sentence trials (contrast weights: +1 +1 –2). This analysis identifies voxels where mean signal is reliably greater for episodic relative to non-episodic trials. To identify content-selective activity during the localizer, we contrasted the two classes of localizer trials (people > locations and vice versa, each contrast thresholded at P < 0.001 with a 19 voxel cluster extent threshold).

Multivoxel Analyses

Feature selection

Pattern similarity analyses were conducted within three regions of the core network: hippocampus, left MTG/ATL, and left AG (Fig. 2A), in addition to the set of content-selective ROIs: PHC and DMPFC (Fig. 2B). ROIs were identified by the above-described univariate analysis. The bilateral hippocampal cluster identified by the univariate analysis spanned across other medial temporal lobe regions. To ensure selectivity of the hippocampal ROIs, we inclusively masked the contrast of episodic memory + simulation > non-episodic/sentence with an anatomically defined hippocampal mask created by manually tracing the hippocampus (using standard anatomical landmarks) in both hemispheres on the across-participant mean anatomic image (Frisoni et al. 2015; for similar approach, see Thakral et al. 2017d). The mean peak MNI coordinates for the different regions were as follows: left hippocampus: −23, −22, −16, with a size of 129 voxels; right hippocampus: 24, −13, −25, 179 voxels; left MTG/ATL: −57, −13, −12, 2326 voxels; left AG: −42, −75, 37, 1138 voxels; left PHC: −27, −43, −10, 668 voxels; right PHC: 30, −37, −12, 658 voxels; and DMPFC: 2, 55, 23, 283 voxels (the latter ROI although extending across hemispheres, fell primarily in the right hemisphere).

Figure 2.

A. Core network ROIs examined for the MVPA overlaid on the across-participant mean T1-weighted anatomical image. Hippocampus shown in green, left MTG/ATL shown in blue, and left AG shown in red. Each core ROI was identified with the contrast of (memory + simulation) > sentence. The coronal slices are spaced every 5 mm with the most anterior (top left) and posterior (bottom right) corresponding to y = 22 and y = −90, respectively. B. Content-selective ROIs examined for the MVPA overlaid on the across-participant mean T1-weighted anatomical image. PHC is shown in magenta and the DMPFC shown in cyan. Content-selective ROIs were identified with the contrast of locations > people and vice versa (for the PHC and DMPFC, respectively). The sagittal slices are spaced every 5 mm with the most right (top left) and left (bottom right) corresponding to x = 39 and x = −36, respectively.

Multivoxel pattern similarity analyses

Functional data from each ROI were preprocessed before MVPA (for similar preprocessing steps, see Kuhl and Chun 2014; Koen and Rugg 2016; Thakral et al. 2017b). First, preprocessing was conducted as described above with the exception of spatial smoothing. Second, data were detrended to remove linear and quadratic trends and z-scored across volumes within each scanning run. Additional z-scoring was conducted on data within each ROI both across trials and across voxels (the mean response at each voxel and the mean response across memory and simulation trials were zero; cf., Kuhl et al. 2013). The resulting z-transformed values were used in the pattern similarity analyses described below.

Pattern similarity analyses were conducted on data from an “early” time window (i.e., TRs 2–3 or 2–6 s) and a “late” time window (i.e., TRs 4–5 or 6–10 s) of the task stimuli presentation (for a similar procedures see, Kuhl et al. 2013). Specifically, the duration of the stimulus presentation (i.e., 10 s) was split into two data bins each consisting of 2 TRs, and the average z-transformed BOLD signal within each time window was extracted. We separately examined these two time windows (i.e., early vs. late), given our recent studies indicating that the hippocampus supports retrieval early, relative to late, in time (see Introduction; Thakral et al. 2017c, 2017d; see also, Vilberg and Rugg 2012, 2014). TR 1 was avoided due to the abrupt onset of the stimuli and associated cue reading (see also, Addis et al. 2007).

Figure 1B provides an illustration of the pattern similarity analysis. We calculated the Fisher z-transformed Pearson’s correlations between patterns of neural activity across memory and simulation trials. Specifically, we first computed correlations between memory and simulation trials from corresponding runs that comprised the same person or location detail (i.e., “matching” correlations; see solid arrow in Fig. 1B, both the memory and corresponding simulation comprise the same location, “Ocean beach pier”). As matching correlations were conducted across memories and simulations comprising identical details (either person or location), they measure the level of reinstatement at the level of the individual detail. Critically, we also computed a baseline correlation to measure the base level of reinstatement of a specific memorial detail in all other simulations not containing that detail (i.e., “mismatching” correlations; see dashed arrows in Fig. 1B, both the memory and corresponding simulations do not share the same location or person). As mismatching correlations were conducted across memories and simulations that did not share the given memorial detail of interest (e.g., “Ocean beach pier”), we reasoned that a greater matching correlation relative to the corresponding mismatching correlation would reflect detail-specific reinstatement from memory (cf., Richey et al. 2013, Wing et al. 2015).

Matching and mismatching correlations were further broken down as a function of the vividness associated with each detail during the simulation task. As prefaced in the Introduction, this approach provides a direct test of the constructive episodic simulation hypothesis (Schacter and Addis 2007, 2019). That is, by examining pattern similarity during simulation, we were able to test whether reinstatement of information from memory contributes to simulation (e.g., greater reinstatement from memory for details higher in vividness during simulation). To ensure sufficient trials in each vividness bin, the 5-point rating scale was split into two bins of roughly equivalent numbers of trials (“high vividness” consisting of ratings 4 and 5 and “low vividness” consisting of ratings 1, 2, and 3; for trial numbers see Table 1; for a similar approach, see, Ritchey et al. 2013). This method yielded 35–45 trials for the high vividness bins across tasks (mean [± 1 standard error, SE] numbers of high vividness trials: person, memory: 39.51 ± 2.40; person, simulation: 35.10 ± 2.54; location, memory: 45.05 ± 3.17; location, simulation: 40.24 ± 3.29) and 31–40 trials for the low vividness bins (mean [±1 SE] numbers of low vividness trials: person, memory: 36.71 ± 2.52; person, simulation: 39.71 ± 2.61; location, memory: 31.05 ± 2.75; location, simulation: 33.90 ± 2.87).

Table 1.

Mean (±1 SE) number of trials for each level of vividness as a function of the individual details comprising memories and simulations

	Low (1)	2	3	4	High (5)
Memory
Person	3.24 (0.62)	12.67 (1.40)	20.81 (1.70)	25.67 (2.10)	13.90 (1.65)
Location	4.05 (0.79)	10.10 (1.63)	16.90 (1.24)	22.71 (2.05)	22.33 (2.50)
Simulation
Person	3.57 (0.68)	14.23 (1.57)	21.90 (2.05)	24.48 (1.92)	10.61 (1.49)
Location	4.57 (1.00)	12.71 (1.72)	16.62 (1.59)	22.76 (2.19)	17.48 (283)

To assess pattern similarity within each region, Fisher z-transformed correlations were entered into an ANOVA with factors Time Window (early and late), Correlation Type (match and mismatch), Vividness (high and low), Detail Type (person and location), and when necessary, Hemisphere (left and right). Although we report all results from each ANOVA, we focus on the interactions because they trumped the main effects and explicitly addressed our motivated a priori hypotheses. ANOVAs were followed up with paired t-tests (two-tailed; uncorrected for multiple comparisons). For all significant results (at the P < 0.05 level), we report the relevant effect sizes (partial η² in the case of F-tests, d for t-tests). P-values are reported to three decimal places and in cases where P < 0.001, we report the P-value as P < 0.001. As stated above, a reliable difference between matching and mismatching correlations reflects processes that support retrieval or reinstatement of detail-specific information from memory during simulation. By further examining whether this latter difference also varies as a function of subjective vividness, we were able to test which regions carry episodic information (i.e., information tied to a specific prior experience) relative to detail-specific conceptual information not specific to a prior episode (see Introduction).¹

Results

Behavioral Results

We first compared the three tasks as a function of overall difficulty and vividness (see Ratings 1 and 2 in Fig. 1A). Wilcoxon signed rank tests revealed greater difficulty for simulation relative to both the memory and sentence tasks (mean [±1 SE] difficulty rating of 2.59 ± 0.08, 1.99 ± 0.07, and 2.11 ± 0.10, respectively; Zs > 3.04, Ps < 2.36 × 10⁻³). The sentence and memory tasks did not differ in relative difficulty (Z = 1.10, P = 0.274). With respect to vividness, Wilcoxon signed rank tests revealed lower vividness during episodic simulation relative to both recalled episodes and the sentence task (mean [±1 SE] vividness ratings of 3.11 ± 0.07, 3.42 ± 0.07, and 3.54 ± 0.12, respectively; Zs > 3.56, Ps < 0.001). The sentence and memory tasks did not differ in relative vividness (Z = 1.34, P = 0.181). These differences in task difficulty and vividness replicate prior findings indicating that episodic simulations are experienced as more difficult and lower in vividness relative to recalled episodes (e.g., D’Argembeau and Van der Linden 2004, 2006; Arnold et al. 2011; Thakral et al. 2019a). Replicating our prior results (Thakral et al. 2019a), and consistent with the difference in vividness for recalled and simulated episodes, when comparing the individual details (see Ratings 3 and 4 in Fig. 1A), both people and location details were associated with lower levels of vividness when simulated relative to when recalled (mean [±1 SE] vividness ratings for details during simulation: person, 3.33 ± 0.07; location, 3.47 ± 0.10; and during memory: person, 3.49 ± 0.06; location, 3.67 ± 0.09; Zs > 2.49, Ps < 0.013).

In our next set of behavioral analyses, we aimed to replicate our prior results and examine whether the vividness for person and location details during memory could be used to predict the vividness ratings for the same details when used during episodic simulation (Thakral et al. 2019a). To test this possibility, we conducted a set of behavioral correlations analogous to the MVPA employed on the fMRI data. On an individual participant basis, the vividness rating was correlated across memory and simulation trials matched as a function of the shared detail (i.e., location and person; see Table 1 for the number of trials that entered the analyses). Replicating our prior results, these correlations were significantly greater than 0 across participants (mean [±1 SE] correlations: r_people = 0.36 ± 0.04 and r_location = 0.41 ± 0.06; ts(20) > 6.90, Ps < 0.001, ds > 1.51). As in our prior report (Thakral et al. 2019a), we conducted a follow-up analysis to determine whether the correlations simply reflect the underlying familiarity of the element shared across memories and simulations (i.e., by partialing out the person and location familiarity rating provided in Session 1, see Materials and Methods). Although the correlations were reduced after partialing out familiarity, we again observed a significantly positive correlation across participants for both people and location details (mean [±1 SE] correlations: r_people = 0.26 ± 0.03; r_location = 0.28 ± 0.04; ts(20) > 6.45, Ps < 0.001, ds > 1.41).

MVPA Results

MVPA was conducted within three regions of the core network: hippocampus, left MTG/ATL, and left AG (Fig. 2A), in addition to the set of content-selective ROIs: PHC and DMPFC (Fig. 2B; for a description of how these ROIs were identified, see Methods).

Core Network ROIs: ROI-Specific Analyses

Hippocampus

Figure 3 illustrates memory–simulation similarity (i.e., Fisher z-transformed correlation between patterns of neural activity) within the hippocampal ROIs as a function of Time Window (early and late), Detail Type (person and location), Correlation Type (match and mismatch), Vividness (high and low), and Hemisphere (left and right). An ANOVA with these factors revealed a significant Correlation Type by Vividness interaction (F(1, 20) = 5.07, P = 0.036, partial η² = 0.29). Follow-up t-tests (collapsed across Hemisphere, Detail Type, and Time Window) revealed that pattern similarity was significantly greater for matching correlations associated with high relative to low vividness (t(20) = 2.62, P = 0.016, d = 0.57), with no corresponding difference of vividness for mismatching correlations (t(20) = 1.51, P = 0.15). Matching correlations associated with high vividness were also significantly greater than either of the two mismatching correlation types (i.e., high or low vividness; ts(20) > 2.44, Ps < 0.024, ds > 0.53). As matching correlations associated with low vividness were not significantly greater than either of the two mismatching correlation types (i.e., high or low vividness; ts(20) < 0.76, ps > 0.458), the reinstatement effect (match > mismatch) was present only when vividness was high. The ANOVA also revealed a significant main effect of Vividness (i.e., high > low; F(1, 20) = 8.15, P < 0.001, partial η² = 0.29). Although the Hemisphere by Time Window interaction approached significance (F(1, 20) = 4.25, P = 0.052, partial η² = 0.18), follow-up t-tests (collapsed across detail type, vividness, and correlation type) failed to reveal any significant hemispheric differences (ts(20) < 1.91, Ps > 0.070). No other ANOVA results were significant (Fs(1, 20) < 3.17, Ps > 0.090).

Figure 3.

Memory–simulation similarity (i.e., Fisher z-transformed Pearson’s correlation, r) within the hippocampus as function of Time Window (early and late), Correlation Type (match and mismatch), Vividness (high and low), Detail Type (person and location), and Hemisphere (left and right). Error bars denote mean (±1 SE) similarity.

Left MTG/ATL

Figure 4A illustrates memory–simulation similarity within the MTG/ATL ROI as a function of Time Window (early and late), Detail Type (person and location), Correlation Type (match and mismatch), and Vividness (high and low). An ANOVA with these factors revealed a Correlation Type by Time Window interaction (F(1, 20) = 5.32, P = 0.032, partial η² = 0.21). Follow-up t-tests (collapsed across Detail Type and Vividness) revealed that matching correlations were significantly greater than mismatching correlations only in the late time window (t(20) = 2.56, P = 0.019, d = 0.56), with no corresponding difference in the early time window (t(20) = 0.188, P = 0.86). The main effect of Time Window was also significant (i.e., late > early; F(1, 20) = 4.59, P = 0.045, partial η² = 0.19). No other ANOVA results were significant (Fs(1, 20) < 2.38, Ps > 0.14).

Figure 4.

Memory–simulation similarity (i.e., Fisher z-transformed Pearson’s correlation, r) within the MTG/ATL (A) and AG (B) as a function of Time Window (early and late), Correlation Type (match and mismatch), Vividness (high and low), and Detail Type (person and location). Error bars denote mean (±1 SE) similarity.

Left AG

Figure 4B illustrates memory–simulation similarity within the AG ROI as a function of Time Window (early and late), Detail Type (person and location), Correlation Type (match and mismatch), and Vividness (high and low). ANOVA with these factors revealed solely a main effect of Correlation Type (i.e., match > mismatch; F(1, 20) = 6.00, P = 0.024, partial η² = 0.23). No other ANOVA results were significant (Fs(1, 20) < 1.63, Ps > 0.217).

Core Network ROIs: Across-ROI Analyses

Consistent with our prediction, the hippocampus alone was found to be sensitive to the vividness of simulation, as indicated by the significant Correlation Type by Vividness interaction as well as the main effect of Vividness. To statistically test whether the hippocampus was uniquely sensitive to the vividness of details during episodic simulation relative to other ROIs (i.e., MTG/ATL and AG), we conducted a follow-up ANOVA and tested specifically for an interaction with factor of Region. Z-transformed correlations were first collapsed across the MTG/ATL and AG as an ANOVA conducted on the data from these two regions failed to reveal any interactions with factor of Region (Fs(1, 20) < 2.18, Ps > 0.155). Consistent with the ROI-specific ANOVAs reported above, this ANOVA did reveal a main effect of correlation type (i.e., match > mismatch; F(1, 20) = 6.34, P = 0.020, partial η² = 0.24) and the Correlation Type by Time Window interaction approached significance (F(1, 20) = 4.33, P = 0.051, partial η² = 0.18). The latter marginal interaction reflects the larger match > mismatching correlations in the later relative to early time window (driven primarily by the MTG/ATL, see above). The ANOVA also revealed a main effect of Region (F(1,20) = 22.07, P = 1.38 × 10⁻⁴, partial η² = 0.53), with all other ANOVA results not significant (Fs(1,20) < 2.11, Ps > 0.155).

We then conducted the critical ANOVA to directly compare the data from the hippocampus relative to the nonhippocampal core network ROIs to test for the presence of interactions with factor of Region. These interactions would provide evidence that the hippocampus was indeed uniquely sensitive to the vividness of simulated details relative to the AG and MTG/ATL (i.e., in these latter two regions, analyses failed to reveal interactions or main effects with factor of Vividness). The ANOVA contained factors Region (hippocampus [collapsed across hemisphere] and non-hippocampus [collapsed across MTG/ATL and AG]), Time Window (early and late), Detail Type (person and location), Correlation Type (match and mismatch), and Vividness (high and low). Importantly, this ANOVA revealed both a Region by Vividness by Correlation Type interaction (F(1, 20) = 4.54, P = 0.046, partial η² = 0.19) and a Region by Vividness interaction (F(1, 20) = 5.63, P = 0.028, partial η² = 0.22). We report the following ANOVA results for completeness but they are secondary to the significant interactions reported above. The ANOVA also revealed main effects of Region (F(1, 20) = 36.81, P < 0.001, partial η² = 0.65), Vividness (F(1, 20) = 6.16, P = 0.022, partial η² = 0.24), and Correlation Type (F(1, 20) = 6.22, P = 0.022, partial η² = 0.24). All other ANOVA results were not significant (Fs(1,20) < 3.54, Ps > 0.075).

Content-Selective ROIs: ROI-Specific Analyses

Parahippocampal Cortex

Figure 5A illustrates memory–simulation similarity within the PHC as a function of Time Window (early and late), Detail Type (person and location), Correlation Type (match and mismatch), Vividness (high and low), and Hemisphere (left, right). This ANOVA revealed solely a main effect of Time Window (i.e., late > early time window; F(1, 20) = 4.53, P = 0.046, partial η² = 0.19). All other ANOVA results were not significant (Fs(1, 20) < 4.29, Ps > 0.051).

Figure 5.

Memory–simulation similarity (i.e., Fisher z-transformed Pearson’s correlation, r) within the PHC (A) and DMPFC (B) as a function of Time Window (early and late), Correlation Type (match and mismatch), Vividness (high and low), Detail Type (person and location), and Hemisphere (left and right). Error bars denote mean (±1 SE) similarity.

Dorsomedial Prefrontal Cortex

Figure 5B illustrates memory–simulation similarity within the DMPFC ROI as a function of Time Window (early and late), Detail Type (person and location), Correlation Type (match and mismatch), and Vividness (high and low). This ANOVA revealed two significant interactions. There was a significant Detail Type by Vividness by Correlation Type interaction (F(1, 20) = 5.62, P = 0.028, partial η² = 0.22) as well as a Detail Type by Vividness interaction (F(1, 20) = 5.84, P = 0.025, partial η² = 0.23). To interrogate the three-way interaction, two follow-up ANOVAs were conducted separately for the data pertaining to each detail type. The ANOVA conducted on the location data revealed no significant results (Fs(1, 20) < 0.71, Ps > 0.409), including a null Correlation Type by Vividness interaction (F(1,20) = 0.38, P = 0.545) and main effect of Vividness (F(1, 20) = 0.14, P = 0.709). In contrast, the ANOVA conducted on the person data revealed solely a main effect of Vividness (i.e., high > low; F(1, 20) = 6.42, P = 0.020, partial η² = 0.24); all other results from this ANOVA, including the Correlation Type by Vividness interaction (F(1,20) = 3.15, P = 0.091), were not significant (Fs(1, 20) < 2.58, Ps > 0.124).

Content-Selective ROIs: Across-ROI Analyses

We conducted a follow-up ANOVA directly contrasting the data across the two content-selective regions in order to test for the selectivity of the content-specific effect observed in the DMPFC relative to the PHC (i.e., the vividness effect in pattern similarity for person details that was not evident for location details in the DMPFC). This ANOVA contained factors Region (DMPFC and PHC [collapsed across hemisphere]), Time Window (early and late), Detail Type (person and location), Correlation Type (match and mismatch), and Vividness (high and low). Again, we tested specifically for the presence of interactions with factor of Region, which would provide evidence that, relative to PHC, the DMPFC was uniquely sensitive to the vividness of person details. In support of this, the ANOVA revealed two significant interactions with factor of Region: Region by Detail Type by Vividness by Correlation Type interaction (F(1, 20) = 6.22, P = 0.022, partial η² = 0.24) and the Region by Detail Type by Vividness interaction (F(1, 20) = 6.66, P = 0.018, partial η² = 0.25). For completeness, we also report the subsidiary ANOVA results, which included a main effect of Region (F(1, 20) = 12.65, P = 0.002, partial η² = 0.39). No other ANOVA results were significant (Fs(1, 20) < 3.46, Ps > 0.078).

Discussion

In this study, we aimed to test a critical tenet of the constructive episodic simulation hypothesis (Schacter and Addis 2007, 2019) by examining whether the hippocampus, a region implicated in the reinstatement of episodic information during successful memory retrieval, also supports the reinstatement of individual episodic details during the simulation of future events. Replicating our prior behavioral findings (Thakral et al. 2019a), the subjective vividness of the individual details comprising episodic memories covaried with the vividness ratings of the same details during episodic simulation. Employing MVPA, we interrogated the similarity across patterns of neural activity within the hippocampus as a function of the individual event details (i.e., people and locations) across episodic memories and episodic simulations. Hippocampal pattern similarity across these episodes was not only specific to the matching of individual event details (i.e., similarity was greatest for past and future episodes when those episodes shared an event detail) but modulated as a function of the vividness with which participants experienced those details during later simulation (i.e., pattern similarity was greatest for details associated with high relative to low vividness during episodic simulation). This influence of vividness on reinstatement was specific to the hippocampus, as pattern similarity within the MTG/ATL and AG, two regions also previously associated with episodic memory and simulation (e.g., Benoit and Schacter 2015), was not sensitive to the vividness of simulated information; pattern similarity in these regions was only sensitive to the matching of individual detail information.

We also extended our MVPA analysis to a set of content-selective regions, the PHC and DMPFC, regions previously shown to be selective to the processing of locations and people during episodic simulation, respectively (Benoit et al. 2014; Szpunar et al. 2014). Analysis of these regions revealed that pattern similarity within the DMPFC, while not sensitive to the matching of individual person details, scaled with the vividness of person details comprising simulations (i.e., greater pattern similarity for person details experienced with high relative to low vividness).

Taken together, these findings provide novel mechanistic insight into how episodic retrieval supports simulation of novel personal events, consistent with the constructive episodic simulation hypothesis (Schacter and Addis 2007, 2019) and related perspectives that emphasize the mechanistic overlap of episodic memory and simulation (e.g., DeBrigard 2014; Michaelian 2016; Addis 2018). We discuss additional implications of these findings below.

Core Network ROIs

Prior theories of episodic memory (e.g., Marr 1971; Norman and O’Reilly 2003; Rugg et al. 2015) state that episodic memory is supported by hippocampal processes that mediate the reinstatement of neural patterns associated with an event during initial encoding. Complementing prior fMRI studies employing MVPA linking hippocampal activity with reinstatement and successful memory retrieval (e.g., Chadwick et al. 2010, 2011; Gordon et al. 2014; Ritchey et al. 2013; Tompary et al. 2016), we found evidence that individual event details are reinstated similarly within the hippocampus when remembering or imagining events comprising those details, and greater similarity is associated with greater vividness of detail-specific information during episodic simulation. Given that the vividness of event details during episodic memory retrieval was correlated with the vividness of those same details during episodic simulation, these findings indicate that the hippocampus supports the reinstatement of detail information from specific prior episodes. Taken together, these findings lend critical and novel support to the constructive episodic simulation hypothesis (Schacter and Addis 2007, 2019) by identifying a hippocampally mediated neural mechanism that supports the reinstatement of individual event details from episodic memory during simulation.

Based on our prior findings indicating that the hippocampus supports a transient retrieval process during simulation (Thakral et al. 2017c, 2017d; see also, Vilberg and Rugg 2012, 2014), we predicted that pattern similarity effects within the hippocampus would be observed early in time. This prediction was not supported by the current data because similarity effects were not selective to the early time window. We highlight that the current null finding with respect to selective similarity effects early in time could be attributable to any number of reasons. For example, our prior study (Thakral et al. 2017c, 2017d) was specifically designed to disambiguate transient and sustained neural activity with the inclusion of a variable delay period. The current study did not include such a manipulation and also employed a shorter task period (i.e., 10 s relative to the 13–17 s employed in Thakral et al. 2017c, 2017d). Therefore, with the employment of a longer task period, it is possible that reinstatement effects like those observed here could be dissociated from other hallmark hippocampal processes (e.g., encoding; Addis and Schacter 2012). Additionally, our initial studies identifying a temporal dissociation within the hippocampus employed univariate analyses, not MVPA. Therefore, the present null findings could reflect the differential sensitivity of these analyses to participant-level and voxel-level variability, respectively (Davis et al. 2014). Future work is necessary not only to replicate the current hippocampal results but also to identify the conditions under which temporal dissociations within the hippocampus can be identified.

In contrast to the hippocampus, pattern similarity in both the MTG/ATL and AG, although sensitive to individual event detail information, did not vary as a function of the subjective experience of the event details (i.e., vividness). We interpret pattern similarity effects that did not vary with vividness as likely reflecting (at least in part) reinstatement of semantic information associated with the matching person or location detail (for similar logic, see Wing et al. 2015). Importantly, such reinstatement cannot be attributed simply to the person/location cue, as the detail was only explicitly cued during the simulation trial. The existence of these effects is consistent with prior proposals suggesting that the inferior parietal lobe and anterior temporal lobe support the retrieval of personal semantics (i.e., acontextual knowledge that defines self-relevant stimuli such as familiar people and locations; Renoult et al. 2012). Further support for this proposal comes from recent MVPA findings demonstrating that multivoxel patterns within the left ATL and within the left inferior parietal lobule in the vicinity of the AG reflect different aspects of specific person knowledge (e.g., identity information and social status information, respectively; Wang et al. 2017). Whereas Wang et al. (2017) examined social concepts in the context of specific people, our results demonstrated that the MTG/ATL and AG are also sensitive to specific locations. The location stimuli, akin to the people, were familiar and thus personally significant, and therefore, our findings are broadly in line with proposals of MTG/ATL function as mediating the representation of conceptual information for both “social” (e.g., people) and “non-social” stimuli (e.g., objects and locations) that carry personal significance (i.e., the role of the MTG/ATL as a “personal semantic store”; Olsen et al. 2013). Lastly, we note that the pattern similarity effects within the MTG/ATL were selective to the “late” time window. These findings are consistent with prior findings indicating that elaboration of both past and future episodes engages both the left temporal pole and MTG (Addis et al. 2007), particularly when the content of memories and simulations is largely conceptual (Addis et al. 2011). In those studies, elaboration effects for past and future events were identified by collapsing across individual episodes. The current result extends these findings by showing that these regions not only support elaboration-related processing during memory and simulation but also support the retrieval and representation of detail-specific information.

The present AG findings are relevant to theories of AG function during memory (for reviews, see Davis et al. 2018; Ramanan et al. 2018; Rugg and King 2018). For example, some have argued that the AG supports the online representation of episodic information, akin to an “episodic buffer” (Vilberg and Rugg 2008), and others have argued that the AG acts as a “convergence zone” binding episodic information (Shimamura 2011; see also, Ramanan et al. 2018). In contrast to these ideas stressing an “episodic” function, others have argued for a broader function. Binder et al. (2009) conducted a meta-analysis of semantic memory fMRI studies and demonstrated that every region of the “core network” with the notable exception of the hippocampus overlaps regions associated with semantic memory processing (see also, Kim 2016). Binder et al. (2009) argued that the overlap reflects the fact that retrieval of episodic information necessarily entails the retrieval of conceptual knowledge (e.g., in order to recall an episode such as “I played tennis last weekend,” one needs to retrieve both general and self-relevant concepts such as “tennis” and “I like to play tennis,” respectively). The current AG results are more consistent with this latter proposal given that pattern similarity within the AG was not sensitive to one critical index of information signaling episodic event information (i.e., the vividness associated with event details from specific prior episodes; for similar MVPA findings also demonstrating that neural patterning in the AG is insensitive to the subjective experience of episodic content see Kuhl and Chun 2014; Thakral et al. 2017b; but see, Bonnici et al. 2016). An important goal for future studies will be to specify the nature of AG representations. The present findings suggest that these representations may include “personal semantics,” akin to the MTG/ATL (see above), or instead may reflect more complex event information (e.g., “event concepts”; Binder and Desai 2011; see also, Rugg and King 2018). Distinguishing between these and other possibilities and integrating the current findings with prior work, suggesting that the AG in particular subserves episodic retrieval and simulation (Thakral et al. 2017a, 2017c), will require additional research linking the key measure of episodic processing used here (i.e., vividness) with measures of episodic processing used in previous studies.

One limitation of the present experiment comes from the fact that, due to trial numbers, we were only able to examine pattern similarity as a function of two levels of vividness (i.e., high relative to low; for similar procedures, see Ritchey et al. 2013). It has been previously shown that dichotomizing variables, as was done with the vividness ratings, can distort results (MacCallum et al. 2002). To examine this possibility, we interrogated pattern similarity in a subset of participants (N = 16) as a function of each level of vividness (minimum of 1 trial per rating). The results of this analysis were null due to the large reduction in statistical power both at the participant- and trial-level. Nonetheless, the overall pattern of an interaction between Correlation Type and Vividness was present in the hippocampus with no such effect present in either the left MTG/ATL or AG. Given that this study is the first to employ MVPA to examine reinstatement for detail-specific information during episodic simulation, the reliability of the current results need to be examined in future studies using larger trial numbers that allow the meaningful assessment of vividness effects at more than two levels of vividness. Until then, some caution should be exercised when interpreting the present findings.

Content-Selective ROIs

Our analysis of the content-selective ROIs revealed that multivoxel patterning during memory and simulation within the DMPFC was uniquely associated with people relative to locations, with the level of pattern similarity greater for people associated with high relative to low vividness. However, this effect was not specific to the reinstatement of individual person detail as the effect of vividness (high > low) was statistically equivalent across correlation types (i.e., matching vs. mismatching correlations), suggesting that this region supports the processing of people more generally and not at the level of the individual person (cf., Wing et al. 2015). The finding that pattern similarity within the DMPFC was selective to people versus locations is consistent with our prior univariate findings, indicating that this region supports processing associated with familiar people during memory and simulation (Benoit et al. 2014; Szpunar et al. 2014). That pattern similarity within the DMPFC was greater for high relative to low vivid people can be interpreted in light of prior fMRI studies demonstrating that the DMPFC supports other relative to self-related judgments during mentalizing (for review, see Denny et al. 2012). Thus, it is possible that simulations comprising familiar people for whom participants could engage other referential processing more successfully were experienced as higher in vividness. Relevant to this possibility is an MVPA study demonstrating that patterning within the DMPFC reflects different personality characteristics, such as agreeableness and extraversion, associated with people during simulation (Hassabis et al. 2014; for a review, see Wagner et al. 2018). These findings would suggest that the current pattern similarity effects might reflect the engagement of this kind of other-related processing (i.e., the processing of personality characteristics).

One important avenue for future work will be to assess the extent to which content-selective pattern similarity effects during memory and simulation relate to those observed in the hippocampus. Neural models of episodic memory (e.g., Marr 1971; Norman and O’Reilly 2003; Rugg et al. 2015) suggest that memory retrieval begins when a cue activates the trace of a memory in the hippocampus and is completed when sensory details are reinstated in the cortex (e.g., Ritchey et al. 2013; Bosch et al. 2014). If the hippocampus supports the reinstatement of individual person or location information during episodic simulation, the hippocampal representation may be related to the representations in the corresponding content-selective ROIs (DMPFC or PHC, respectively). This is a topic for future research.

Conclusion

In the present study, we employed MVPA and demonstrated that the hippocampus supports the reinstatement of detail-specific episodic information during memory and simulation, and critically, this reinstatement varies with the vividness of event details during simulation. In contrast to the hippocampus, pattern similarity within the AG and MTG/ATL, although sensitive to the individual details shared across memory and simulation, did not vary with respect to the vividness of the details during simulation. These findings suggest that the AG and MTG/ATL support the reinstatement of semantic information associated with individual event details during simulation. This distinction should motivate further inquiry into identifying the role of individual core network regions in specific aspects of simulation. The current findings are also relevant to recent work implicating the hippocampus, and core network more broadly, in generative tasks, which akin to simulation, also involve the retrieval and recombination of episodic details for completion, such as divergent creative thinking and means-end problem solving (for reviews, see Schacter et al. 2012, 2017; Moscovitch et al. 2016). For example, a recent fMRI study employing a univariate analysis demonstrated that the hippocampus is jointly recruited during episodic memory, simulation, and divergent creative thinking (Beaty et al. 2018). In light of the present findings, it is possible that the hippocampus supports the reinstatement of vivid detail-specific episodic information not only during episodic simulation but also during related cognitive functions, such as divergent creative thinking.

Notes

We thank Jyotika Bindra for assistance in data collection. Conflict of Interest: None declared.

Funding

National Institute of Mental Health (grant R01MH60941 to D.L.S.); National Institute on Aging (grant F32AG059341 to K.P.M.); Canada 150 Research Chairs Program (to D.R.A.); National Institute of Health Shared Instrumentation (grant S10OD020039).