Content-specific phenomenological similarity between episodic memory and simulation

Abstract

Numerous studies have indicated that remembering specific past experiences (i.e., episodic memory) and imagining specific novel future experiences (i.e., episodic simulation) are supported by common mental processes. An open question, however, is whether and to what extent the content of specific past episodes is sampled when simulating a specific future episode. The current study aimed to answer this question. Participants recalled past episodes each comprising two episodic details, a personally familiar location and person. Participants also simulated novel future episodes using recombined pairs of person and location details taken from different recalled episodes. Participants rated the vividness of each location and person in their memory and simulation. We conducted a multi-level analysis where the vividness rating during memory was used to predict the vividness rating during simulation at the level of individual shared details (i.e., location or person). The vividness of the memorial detail co-varied with the vividness of the simulated detail; this relationship persisted even after accounting for the underlying familiarity of the details. These findings strongly suggest that simulations of specific future experiences are based upon the contents of specific prior episodes.

Content-specific phenomenological similarity between memory and simulation

Numerous studies have revealed striking cognitive and neural similarities between remembering past experiences and imagining future experiences (Conway, Loveday, & Cole, 2016; Schacter, Benoit, & Szpunar, 2017). According to the constructive episodic simulation hypothesis (Schacter & Addis, 2007), many of these similarities arise because imagining or simulating future experiences (episodic simulation) draws on the ability to recall specific past episodes (episodic memory). Specifically, episodic retrieval supports the ability to simulate future experiences by providing access to episodic details (e.g., people, locations, and objects) that can be recombined in novel ways. Behavioral and neural evidence support this hypothesis. For instance, episodic remembering and future simulation similarly engage direct and generative retrieval mechanisms (Anderson, Dewhurst, & Nash, 2012), and functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation studies indicate that episodic memory and simulation are both supported by a ‘core network’ of neural regions (e.g., Benoit & Schacter, 2015; Thakral, Madore, & Schacter, 2017).

The prior findings suggest a strong link between the processes that support memory and simulation. It remains unclear, however, whether and to what extent the same contents associated with episodic memories are used to build episodic simulations. If past and future episodes draw on similar stored episodic information, as stipulated by the constructive episodic simulation hypothesis, one would expect strong overlap in the elemental episodic components (e.g., people and locations) that comprise future simulations and the episodic memories from which they are drawn. There is some evidence to support this prediction. For example, past and future episodes are generally described as occurring in similar contexts (e.g., episodes at parties, school, and work comprise the large majority of past and future episodes; D’Argembeau & Van der Linden, 2004), and future episodes occurring in familiar relative to unfamiliar contexts (e.g., a current apartment versus the jungle) are rated as higher in sensory detail and vividness (Szpunar & McDermott, 2009; see also, Szpunar, Chan, & McDermott, 2009; D’Argembeau & Van der Linden, 2012). Importantly, however, there are a number of caveats attached to the findings of the above studies. First, familiar relative to unfamiliar stimuli are not only associated with prior episodic memories but are also associated with a greater degree of personal semantic knowledge (i.e., the autobiographical facts that define personally relevant stimuli; Renoult, Davidson, Palombo, Moscovitch, & Levine, 2012). Second, prior studies have not examined the extent to which the content across individual past and future episodes is shared.

The current study provides a novel test of the constructive episodic simulation hypothesis by assessing whether content-specific information from episodic memory, such as people or locations, is recruited when constructing a novel episodic simulation. Participants recalled past episodes each comprising two episodic details, a personally familiar location and person. Participants also simulated novel future episodes using recombined pairs of person and location details taken from different recalled memories. For both original memories and recombined simulated pairs, participants rated the vividness with which they experienced each location and person in their memory and simulation. To examine whether the content associated with single episodic details is shared across memory and simulation, we employed a hierarchical lineal model (HLM), a multi-level analytic approach that preserves individual trial information. Here, we used HLM to examine whether the vividness rating for individual episodic details during memory could be used to predict the vividness rating for the same episodic detail during simulation. If future simulations are based upon the contents of specific episodic memories, then the phenomenological quality (i.e., vividness) of the individual details comprising simulations should vary based on the episodic memories from which those details have been sampled, and therefore result in a significant predictive relationship (for similar logic, see Szpunar & McDermott, 2008; Szpunar et al., 2009). We also collected a measure of the underlying familiarity of the details comprising the memories and simulations. This allowed us to assess the extent to which the shared vividness between individual details across memories and simulations is attributable to the overlap in detail familiarity. A significant relationship between memories and simulations at the level of individual episodic details even after accounting for detail-specific familiarity would provide strong evidence to indicate that individual episodic details from memories are sampled when simulating a future episode.

Method

Participants

Data from 24 participants were collected (mean age of 20.63 years (range 18–25), 17 females). The experimental protocol was approved by the Institutional Review Board of Harvard University and informed consent was obtained prior to participation.

Procedure

Session 1, Stimuli collection

Participants were asked to recall 100 personal memories from the past 5 years, as is typical in memory and imagination paradigms that involve recombination of episodic details (e.g., Addis, Pan, Vu, Laiser, & Schacter, 2009; McLelland, Devitt, Schacter, & Addis, 2014). Each memory had to be unique with respect to the location it occurred in and the person involved that was of primary importance. To facilitate retrieval, participants were told that they could use their cell phone or social media and were also provided with an extensive list of event cues.

For each memory, participants were instructed to provide a brief description of the event. These descriptions were used by the experimenter to ensure the memories provided were specific in time and place (i.e., were episodic in nature). Participants were also instructed to create a memory cue. These cues had to be as short as possible and were meant to serve as a tag that would allow the participant to instantaneously recall the memory the cue referred to. The memory cue could not include the location or the person’s name associated with the memory. Participants then specified the person of interest who participated in the event and the location of interest where it occurred. When writing down a location, participants were asked to specify a short location name that would allow them to instantaneously imagine the exact location of the memory. Participants were asked to not include the people’s names in the location name. Participants were instructed to generate events where they interacted with the person listed and were physically present at the specified location (i.e., not include events that they had only heard about).

For each memory, person, and location, participants provided three ratings: personal significance, familiarity, and vividness. These ratings were used to assess whether the shared vividness between individual details across memories and simulations was due, in part, to the overlap in these extraneous factors, primarily familiarity (see Introduction). For familiarity, participants were asked to rate their knowledge of the memory, person, or location in their daily life (ranging from not very familiar to very familiar on a scale of 1 to 5). Following previous studies (Benoit, Szpunar, & Schacter, 2014; McLelland et al., 2014), this rating was assumed to measure the amount of semantic knowledge one has regarding the event, person, and location (i.e., ‘personal semantics’; see, Renoult et al., 2012).

Prior to Session 2, the 100 memory cue-location-person triplets were randomly sorted. Eighty-four triplets were then chosen for the experiment. There were a total of 6 runs (3 memory runs and 3 simulation runs). Each memory run comprised 28 memory cues. For each memory run, there was a corresponding simulation run comprising randomly recombined location-person pairs created from the memory cues (i.e., each cue had an original location-person pair). Order of the runs was counterbalanced across participants (i.e., odd runs were selected to be memory and even runs were selected to be simulation, and vice versa). Regardless of whether odd runs were memory or simulation runs, the memory and corresponding simulation run were always presented in succession (e.g., if the first run was a memory run, the second simulation run consisted of the recombined location-person details taken from the preceding memory run). We adopted this method to equate the delay between recalling a given memory, and the simulation of the novel recombined location-person pair. Within each run, we also incorporated 3 trials of a non-episodic, sentence task (see, Benoit et al., 2014).

Session 2, Experimental phase

Session 2 occurred between 2 and 7 days following Session 1. Before beginning Session 2, participants were familiarized with the memory cues and associated details that they had generated in Session 1. During Session 2, participants completed two tasks, an episodic memory task and an episodic future simulation task. On each trial of the memory task (see Figure 1A), participants were presented with a memory cue generated during Session 1 (i.e., from the prior stimulus collection phase). The task was to silently remember the same specific experience generated in Session 1 as vividly as possible from a first-person perspective focusing on how the person and location were featured in the corresponding memory. On each trial of the simulation task (see Figure 1B), participants were presented with two details (i.e., a person and location name generated from Session 1). The task was to silently imagine a specific and novel future episode where they were interacting with the details cued in a location-specific manner as vividly as possible. They were required to imagine the episode from a first-person perspective and to restrict the imagined future episode to only the details cued. Following each memory and simulation trial, participants were asked to rate how difficult it was to remember/simulate the episode and the vividness of the 1) remembered/simulated episode, 2) person in the memory/simulation, and 3) location of the memory/simulation. All ratings were done on a 5-point scale ranging from low to high. All stimuli were presented on a black background in 25-point Arial font.

Figure 1.

A. Two representative trials from the episodic memory task. On each trial, participants were presented a memory cue that they had generated from an initial session (e.g., ‘Pound cake, Valentine’s Day’). Each memory comprised two details, a person and a location (e.g., ‘Janine W.’ and ‘Kitchen, Brighton apartment’). B. Representative trial from the episodic simulation task. On each trial, participants were presented with recombined person and location details across separate memories (e.g., ‘Patrick W.’ from the memory ‘Pound cake, Valentine’s Day’ (panel A, top) and ‘Kitchen, Brighton apartment’ from the memory ‘West Wing marathon’ (panel A, bottom)).

As illustrated in Figure 1, across the memory and the simulation trials, task-relevant information did not contain overlapping perceptual information (i.e., the memory cues did not include the person or location names used for the simulation task; see, Session 1, Stimuli collection). We chose to avoid perceptual overlap in an attempt to reduce shared perceptual processing across memory and simulation trials, which may have inflated any across-trial relationship (e.g., common cue processing). Following the experiment, participants were debriefed regarding the purpose of the study. No participants reported that they noticed that any detail was repeated across runs. When told about the repetition, participants did not report using the repetition of the details as a way to complete the tasks. This finding suggests that the memory and simulation tasks were approached as independent tasks.

Results

Unless otherwise noted, all analyses were conducted on data collected from Session 2 (i.e., the experimental phase). In our first set of analyses, we compared the tasks as a function of difficulty and vividness to replicate prior known differences across episodic memory and episodic future simulation (e.g., simulated future episodes are generally experienced as lower in vividness and more difficult to generate relative to recalled episodes; e.g., D’Argembeau & Van der Linden, 2004; Addis et al., 2009; Arnold, McDermott, & Szpunar, 2011). In our second set of analyses, we tested our prediction that individual details during memory are sampled during episodic simulation. To test this, we conducted a set of multi-level analyses where the vividness rating for individual details (i.e., location and person) during memory was used to predict the vividness for those same details during simulation. Critically, an additional control analysis was performed to assess whether the shared vividness of a specific detail could be accounted for by the familiarity of the detail across memories and simulations.

Task analyses

Mean difficulty and vividness for each task is listed in Table 1. A Wilcoxon signed rank test revealed greater difficulty for simulation relative to the memory task (Z = 4.23, p < 0.001). In addition, recalled episodes were associated with greater vividness relative to simulated episodes (Z = 4.14, p < 0.001). These task differences replicate prior studies showing that simulations are generally less detailed (e.g., D’Argembeau & Van der Linden, 2004; Addis et al., 2009) and are more difficult to generate (e.g., Arnold et al., 2011) than memories. Consistent with the difference in vividness for recalled and simulated episodes, when comparing the individual details comprising those episodes (i.e. locations and people), both details were associated with lower levels of vividness when simulated relative to when recalled (Zs > 2.20, ps < 0.05).

Table 1.

Mean (± 1 standard error) difficulty and vividness rating for each task and detail comprising the memories and simulations.

	Difficulty	Vividness
Memory
Episode	1.91 (0.10)	3.41 (0.08)
Person	-	3.57 (0.07)
Location	-	3.57 (0.10)
Simulation
Episode	2.62 (0.11)	3.01 (0.09)
Person	-	3.41 (0.08)
Location	-	3.46 (0.09)

Multi-level analyses

Using HLM software (Raudenbush, Bryk, & Congdon, 2011), we created two-level random coefficient models in which a given detail (i.e., person or location) was modeled at the within-participants level and each participant was modeled at the between-participants level. All slopes and intercepts were allowed to vary across participants. Models were estimated using a restricted maximum likelihood method, producing unbiased estimates of covariance parameters (maximum number of 100 iterations). Two models were created to separately examine effects associated with person and location details. For the model examining person details, there were 1842 records at the first level and 24 records at the second level. For the model examining location details, there were 1832 records at the first level and 24 records at the second level. Table 2 lists the mean number of records/trials as a function of vividness rating and detail comprising both memories and simulations that entered the analyses.

Table 2.

Mean (± 1 standard error) number of trials for each level of vividness as a function of the individual details comprising memories and simulations.

	Low (1)	2	3	4	High (5)
Memory
Person	3.46 (0.80)	11.12 (1.39)	18.00 (1.18)	26.83 (1.38)	17.50 (2.01)
Location	5.50 (1.21)	10.54 (1.33)	15.50 (1.34)	25.00 (2.33)	20.21 (2.15)
Simulation
Person	4.96 (0.10)	13.04 (1.68)	19.08 (1.45)	24.75 (1.31)	15.08 (2.06)
Location	6.38 (1.29)	11.75 (1.26)	16.25 (1.29)	25.21 (1.88)	17.04 (1.97)

Initial intercept-only models revealed that a significant portion of variance in the vividness of person and location details during simulation was due to between-participant variation (τ_person(23) = 0.13, p < 0.001 and τ_location(23) = 0.24, p < 0.001) explaining 9.63% and 16.06% of the variance, respectively, indicating that multi-level modelling was appropriate. We next examined whether the model fit was improved relative to the intercept-only models when the detail-specific vividness ratings during memory were entered as level 1 predictors of the vividness of those details during simulation. The new models significantly reduced the deviance statistics (reflecting improved model fit) relative to the intercept-only models (likelihood-ratio tests, χ²_person(2) = 335.97, p < 0.001 and χ²_location(2) = 290.17, p < 0.001). The vividness rating of people during memory significantly predicted the vividness rating of the same person during simulation (B = 0.39, t-ratio(23) = 7.92, p < 0.001). The same relationship held for location details (i.e., the vividness rating of locations during memory predicted the vividness rating of the same locations during simulation; B = 0.35, t-ratio(23) = 7.78, p < 0.001).

The previous results indicate that the subjective experience, in the form of vividness, across memories and simulations is linked at the level of the individual details comprising the episodes. An open question, however, is whether the latter associations simply reflect the underlying familiarity of the element shared across memories and simulations (e.g., a person or location that one has little experience or knowledge of may be experienced as low in vividness during both memory and simulation). To answer this question, we took advantage of the person and location familiarity rating provided in Session 1 (i.e., during initial stimulus collection; see Method) to assess whether the relationship between vividness during memory and simulation at the level of individual details could be identified after variance attributable to familiarity was partialled out. If a significant association is observed after the familiarity of the detail is accounted for, this finding would suggest that the same episodic detail information is sampled across memory and simulation (see Introduction). Adding the person and location familiarity rating as a third predictor improved the model fits (χ²_person(3) = 186.24, p < 0.001 and χ²_location(3) = 114.52, p < 0.001). Critically, the vividness ratings during memory for both people and locations remained significant predictors of the vividness ratings for the same details during simulation (B_person = 0.26, t-ratio(23) = 5.44, p < 0.001 and B_location = 0.24, t-ratio(23) = 6.71, p < 0.001).

Discussion

According to the constructive episodic simulation hypothesis (Schacter & Addis, 2007), episodic retrieval supports the ability to imagine hypothetical future experiences by allowing access to episodic information that can be recombined to build novel events. We tested this hypothesis by examining whether the phenomenological quality, or vividness, of individual elements that comprise episodic simulations can be predicted from the episodic memories from which they are sampled. When simulations and memories were matched as a function of a shared episode detail, either location or person, we found evidence that the vividness of the memorial detail co-varied with the vividness of the simulated detail. This relationship persisted even after accounting for the underlying familiarity of the location or people. Given that the phenomenological qualities of the individual elements co-varied across memories and simulation, these findings strongly suggest that simulations are based upon episodic contents of memory, in line with the constructive episodic simulation hypothesis.

At first glance, the significant across-trial associations may not be seen as surprising. However, there are multiple factors within the present paradigm that work against finding such results. For instance, both the memory and simulation tasks were open-ended. During Session 1, participants were free to generate any episodic memory as long as they fulfilled the experimental criterion. Similarly, participants were free in how they approached the simulation task so long as they integrated the cued details into a novel future episode. Given the open-ended nature of the tasks, the recalled and simulated episodes were relatively independent (i.e., there was no reason why they had to be associated). In addition, given that the location and person details were randomly recombined for the simulation task, the simulations could have varied across any number of characteristics (e.g., plausibility, emotionality, etc.), with these factors possibly affecting the vividness of the individual details. Thus, the present findings are strengthened by the number of experimental factors that likely worked against the possibility of finding an association across the individual details shared across memories and simulations.

It is important to note some possible caveats to the present study. First, the across-trial relationship between memory and simulation could arise because participants performed two tasks across repeated details. We underscore that unless episodic simulations comprise completely novel event details, any relationship between a given detail during simulation and memory (either episodic or semantic) can be due to repetition. Here, we provide evidence indicating that details comprising simulations can be predicted by the episodic qualities of memorial details (i.e., their subjective vividness/sensory detail) over and above the familiarity/personal semantic knowledge of a detail. Second, similar to prior studies (Benoit et al., 2014; McLelland et al., 2014), we measured the underlying familiarity of elements using a rating scale. We adopted this approach such that the vividness and familiarity ratings could be combined in the same analysis. It will be important for future research to adopt other methods to quantify personal semantics and replicate the current findings (for a list of other approaches, see Renoult et al., 2012). Third, our results are limited to the current experimental paradigm (i.e., where participants are explicitly instructed to simulate cued details from memory). It remains to be determined to what extent episodic details would be used for constructing future events when not explicitly cued (e.g., spontaneously or voluntarily).

While we have focused on shared episodic contents between memories and simulations of future experiences, people can also simulate hypothetical episodes in the present or the past (cf., Addis et al., 2009), including, for example, counterfactual simulations of how specific past experiences might have turned out differently (e.g., De Brigard, Addis, Ford, Schacter, & Giovanello, 2013). It will be important to determine whether the same kind of content-specific relation documented here for episodic memories and future simulations also holds for episodic memories and counterfactual simulations of past happenings. Critically, the present experiment provides a novel approach that could be employed in future studies to examine additional factors that link memory with various kinds of simulations at the level of individual episodic details.