Shifting Visual Perspective During Retrieval Shapes Autobiographical Memories

Abstract

The dynamic and flexible nature of memories is evident in our ability to adopt multiple visual perspectives. Although autobiographical memories are typically encoded from the visual perspective of our own eyes they can be retrieved from the perspective of an observer looking at our self. Here, we examined the neural mechanisms of shifting visual perspective during long-term memory retrieval and its influence on online and subsequent memories using functional magnetic resonance imaging (fMRI). Participants generated specific autobiographical memories from the last five years and rated their visual perspective. In a separate fMRI session, they were asked to retrieve the memories across three repetitions while maintaining the same visual perspective as their initial rating or by shifting to an alternative perspective. Visual perspective shifting during autobiographical memory retrieval was supported by a linear decrease in neural recruitment across repetitions in the posterior parietal cortices. Additional analyses revealed that the precuneus, in particular, contributed to both online and subsequent changes in the phenomenology of memories. Our findings show that flexibly shifting egocentric perspective during autobiographical memory retrieval is supported by the precuneus, and suggest that this manipulation of mental imagery during retrieval has consequences for how memories are retrieved and later remembered.

1.1 Introduction

Memories from our personal past are not static, but can be retrieved and thought about in multiple ways. This adaptive and flexible characteristic of memories enables us to construct alternative visual perspectives from which to view the past, in which we shift our first person viewpoint from inside the body to outside the body—seeing ourselves within the memory rather than re-experiencing it directly (Nigro and Neisser, 1983). Moreover, retrieving the past from alternative visual perspectives is not merely epiphenomenal, but impacts our sense of self (Sutin and Robins, 2008), affects our current mood and future behavior (Holmes et al., 2008; Libby et al., 2007), influences our causal attributions (Frank and Gilovich, 1989), and is affected in several mental disorders (Kenny et al., 2009). Thus, understanding the mechanisms by which we can adopt alternative visual perspectives during memory retrieval has important implications for many domains. Here we examine how neural mechanisms that enable the construction of alternative versions of the personal past when adopting different visual perspectives during memory retrieval shape remembering and subsequent memories.

Visual perspective is a necessary feature in order to retrieve memories as remembered events rather than self-knowledge (Rubin and Umanath, 2015). Moreover, the particular egocentric perspective adopted during memory retrieval may also provide insight regarding the constructive nature of memories (also see McDermott et al., 2016). Memories can be retrieved from the visual perspective of our own eyes, as most events are initially experienced, as well as from the visual perspective of an observer, as if we were seeing ourselves in the memory. If remembering was like pressing play on a movie, the observer perspective would be like watching the star in the movie as a member of the audience, whereas the own eyes perspective would be like experiencing the movie as the main star. Recent autobiographical memories are more frequently associated with spontaneously adopting an own eyes perspective, whereas remote memories are associated with an observer perspective (e.g., Nigro and Neisser, 1983; Rice and Rubin, 2009). This consistent pattern of results is thought to reflect the natural transformation of memories overtime (Butler et al., 2016), because memories are not usually formed from an observer perspective (but see Bergouignan et al., 2014; Cardena and Spiegel, 1993; Ozer and Weiss, 2004). Actively shifting visual perspective from an own eyes to an observer perspective during memory retrieval has also been shown to affect the content and phenomenological characteristics of retrieval. For example, adopting an observer visual perspective during retrieval reduces the emotional intensity of memories (Berntsen and Rubin, 2006; Robinson and Swanson, 1993), possibly due to increased detachment or distance from the remembered event (but see Libby and Eibach, 2011). Moreover, these changes in memories as the result of shifting visual perspective during retrieval at one point in time can also contribute to persistent changes in subsequent memories (Sekiguchi and Nonaka, 2014). A number of functional neuroimaging studies have demonstrated that memory retrieval is an active process that can modify memories (Bridge and Paller, 2012; Gershman et al., 2013; St Jacques et al., 2013b), which supports memory theories that emphasize the critical role of reactivation in shaping the brain networks that contribute to long-term memory representations (Mcclelland et al., 1995; Winocur and Moscovitch, 2011). Manipulating visual perspective during memory retrieval could provide an experimentally tractable way to investigate constructive neural mechanisms that potentially shape autobiographical memories in both the short and long-term.

Adopting a particular visual perspective critically depends upon egocentric representations in the posterior parietal cortex (Aguirre and D’Esposito, 1999; Ciaramelli et al., 2010; Wilson et al., 2005), and has been linked in particular to the precuneus (for reviews see Byrne et al., 2007; Cavanna and Trimble, 2006). As the so-called “mind’s eye” (Fletcher et al., 1995), the precuneus has long been associated with mental imagery processes during memory retrieval, as well as visuospatial imagery and self-referential processes (for review see Cavanna and Trimble, 2006). A rich literature has demonstrated that the precuneus also supports the ability to imagine alternative visual perspectives (Jackson et al., 2006; Vogeley et al., 2004; for review see Van Overwalle and Baetens, 2009) and to navigate in space (Ghaem et al., 1997; Spiers and Maguire, 2006; for review see Boccia et al., 2014), perhaps reflecting a more general ability to orient the internal representation of the self with the external world (Peer et al., 2015). According to a prominent neural model of spatial memory and imagery, egocentric frameworks generated during retrieval from long-term memory within the precuneus can be manipulated and updated when people imagine the possible movements they can make within the remembered scene (Byrne et al., 2007). Supporting this model, a number of recent studies have shown that the precuneus contributes to the ability to update internal representations of the world when imagining changes in self-location in space (Dhindsa et al., 2014; Lambrey et al., 2012; Sulpizio et al., 2016; Wolbers et al., 2008).

Much less is known about how visual perspective influences the neural mechanisms of long-term episodic memory retrieval, including autobiographical memories. Autobiographical memory retrieval is supported by a network of brain regions that encompasses lateral and medial parietal cortices, including the precuneus (Cabeza and St Jacques, 2007; Fuentemilla et al., 2014; Spreng et al., 2008; Svoboda et al., 2006). Generating and elaborating upon vivid mental images during autobiographical memory retrieval has been linked to the precuneus (Daselaar et al., 2008; Fuentemilla et al., 2014; Gardini et al., 2006; Söderlund et al., 2012), and recruitment of this region also contributes to the ability to construct a complex and realistic scene of the personal past (Hassabis et al., 2007; Summerfield et al., 2009). In a structural MRI study, Freton and colleagues (2014) found that the volume of grey matter in the precuneus was positively related to the spontaneous retrieval of autobiographical memories from an own eyes perspective, but the rating scale they used precluded examining whether a similar relationship held for memories spontaneously retrieved from an observer perspective (e.g., Rice and Rubin, 2009). The couple of neuroimaging studies that directly examined how adopting an own eyes or observer visual perspective influences memory retrieval have also found inconsistent results concerning the involvement of the precuneus. In a recent fMRI study, Grol, Vingerhoets and De Raedt (2017) found that neural recruitment of the precuneus was greater when adopting an observer compared to an own eyes perspective during autobiographical memory retrieval. In contrast, Eich and colleagues (2009) found that the precuneus was similarly involved when retrieving memories for complex lab-based events from own eyes and observer perspectives. These inconsistent findings with respect to the precuneus involvement, when adopting a particular egocentric perspective during memory retrieval, could reflect different demands on perspective shifting processes (Wolbers et al., 2008). Here we hypothesize that during autobiographical memory retrieval shifting visual perspective involves the manipulation of egocentric mental images in the precuneus.

Using functional magnetic resonance imaging (fMRI) we examined the neural mechanisms that support the ability to retrieve alternative versions of the personal past and to shape online and subsequent memories by manipulating whether participants adopted a dominant or alternative visual perspective during memory retrieval. In an initial session, participants generated a large number of autobiographical memories and provided subjective ratings on the degree to which memories were spontaneously retrieved from own eyes and observer perspectives (see Figure 1). We then selected a subset of memories that were strongly associated with a spontaneous own eyes perspective, which allowed us to control for the initial perspective of memories in order to more effectively manipulate visual perspective shifting during fMRI scanning. We manipulated the degree of visual perspective shifting during fMRI scanning by asking participants to retrieve memories either from the alternative (i.e., observer perspective) or the same (i.e., own eyes) visual perspective as originally reported, thus requiring a shift in perspective or no shift, respectively across study sessions. A repetition suppression approach was employed here to compare the linear decrease in the blood oxygen level dependent (BOLD) response across the two perspective conditions (Grill-Spector et al., 2006). Participants were asked to retrieve memories across three identical repetitions and we examined repetition suppression, or the decrease in the BOLD response, from the first to the third repetition. We predicted that in both egocentric conditions repeated retrieval of memories would result in reductions in neural recruitment across the memory retrieval network in line with repetition suppression effects observed in similar tasks (Szpunar et al., 2014; van Mulukom et al., 2013). However, we hypothesized that shifting visual perspective during memory retrieval likely requires additional processes that enable egocentric perspectives to be updated during long-term memory retrieval (e.g., Byrne et al., 2007). In particular, we predicted that precuneus would show a linear decrease with repetitions when shifting visual perspective during retrieval, reflecting decreased engagement of egocentric updating mechanisms with successive repetitions of memories from an alternative visual perspective. We then examined how this neural signature of egocentric updating during memory retrieval contributed to online and subsequent changes in the phenomenology of memories, where we predicted greater subsequent changes due to shifting perspective compared to maintaining the same perspective.

Figure 1.

Experimental design. During session 1 participants generated specific autobiographical memories from the past 5 years and rated the degree of own eyes and observer perspective on 7-point scales from 1 = low to 7 = high. Only memories associated with a strong own eyes perspective (i.e., >= 5 on the own eyes rating, and < 4 on the observer rating) were included in session 2 and 3. One week later during session 2 participants were asked to retrieve some of these memories from either an own eyes or an observer perspective while undergoing fMRI scanning. In the non-shifted perspective condition the strong own eyes memories were retrieved again from the same own eyes perspective, whereas in the shifted perspective condition memories were retrieved from the alternative observer perspective. Participants then rated the ability to maintain the indicated perspective and the emotional intensity associated with retrieval. Two days after scanning in session 3 participants were asked to retrieve all the memories again, including memories in a baseline condition without additional retrieval experience during scanning, and to rate the visual perspective associated with retrieval.

1.2. Material and Methods

1.2.1. Participants

Participants were aged 18 – 30 years. All participants were right-handed, reported no history of neurological or psychiatric episodes or current use of medication known to affect cognitive function. Participants gave written consent for a protocol approved by the Harvard University Intuitional Review Board. In total, 37 participants (24 women; Mean Age in Years = 22.3, SD = 3.3) gave written informed consent. Three participants were excluded due to an inability to retrieve a sufficient number of strong own eyes memories (for further details see Procedure). Additionally, four participants were excluded from the fMRI analysis because of excessive movement during scanning (i.e., maximum absolute movements greater than 2 mm, more than 5 movements greater than 0.5 mm, and/or a slice signal-to-noise ratio less than 99). Thus, the results were based on 29 participants (20 women, Mean Age in Years = 22.6, SD = 3.2).

1.2.3. Procedure

The study took place across three separate sessions. In session 1, participants were provided with a list of possible events to help generate 240 autobiographical memories from the last five years that were specific to time and place. They were asked to provide a unique title, specific date, and subjective ratings of emotional intensity, reliving, own eyes perspective, observer perspective, and rehearsal (all on 7-point scales from 1 = low to 7 = high). Thus, participants generated memories and then rated the degree to which memories were retrieved from their own eyes and from an observer visual perspective, where their first person viewpoint is centred outside of the body. In line with previous findings (Rice and Rubin, 2009), approximately 72% (SD = 18%) of memories were spontaneously retrieved from an own eyes perspective, as indicated by higher ratings of own eyes perspective (>=5) coupled with lower ratings of observer perspective (< 4). We controlled for the initial perspective of memories by selecting this subset of memories strongly associated with an own eyes perspective and 96 of these memories, hence forth referred to as own eyes memories, were randomly selected for all further sessions. These own eyes memories were then assigned to three separate conditions that were matched in terms of the phenomenological ratings. Thus, there were no initial differences in the phenomenological ratings of memories within each condition.

In session 2, one week later participants were asked to retrieve these own eyes memories while taking either the perspective of their own eyes or the perspective of an observer. Specifically, participants were instructed: “If the perspective is own eyes, mentally reinstate your memory for the event as if seeing it again through your own eyes. If the perspective is observer, mentally reinstate your memory for the event as if viewing it from the perspective of a spectator or observer, watching yourself in the remembered event.” Thus, in the non-shifted perspective condition, memories were retrieved again from the same own eyes perspective, whereas in the shifted perspective condition memories were retrieved from the alternative visual perspective of an observer. Prior to fMRI scanning, participants were first trained on the task and any questions about the visual perspective conditions were addressed. During fMRI scanning, participants were shown titles of 64 of the own eyes memories and they were asked to retrieve them while taking either the perspective of their own eyes or the perspective of an observer (see Figure 1). Participants had 7.5 s to retrieve the memory from the indicated perspective and then were given 2.5 s each to rate the amount of emotional intensity and how well they were able to maintain the perspective indicated (on 5-point scales from 1 = low to 5=high), the order of which was counterbalanced across participants. The timing of the task was based on a previous study that examined retrieval and manipulation of autobiographical information (Szpunar et al., 2014), and we conducted further pilot testing to ensure that participants had sufficient time for memory retrieval. In order to examine repetition suppression effects, participants were asked to retrieve each memory three times within the shifted and non-shifted conditions. Memory repetitions took place within each of 6 functional runs for a total of 42 (run 1) or 30 (run 2 – 6) trials in each run. Run 1 was longer than the other runs due to practical issues related to separating the 64 memories equally across the runs. There were two counterbalanced randomized sequences of the repetitions of trials for each functional run, such that memories were repeated every 2 to 6 trials. For each repetition participants were instructed to retrieve the memory in the same way from the indicated visual perspective. Trials were separated by an active baseline consisting of left/right decisions equally spaced across a variable length (2.5 – 10 s; e.g., Stark and Squire, 2001), distributed exponentially such that shorter inter-trial intervals occurred more frequently than longer.

In session 3, two days later, we investigated the influence of visual perspective shifting on subsequent memory. Participants were asked to retrieve the 64 own eyes memories that had been retrieved during scanning, along with 32 baseline memories that had not been retrieved during session 2. No instructions were provided regarding visual perspective. Thus, similar to session 1, in this final session we examined again the spontaneous perspective that participants adopted during memory retrieval. They were asked to rate the extent to which memory retrieval was associated with an own eyes and an observer perspective, emotional intensity, and reliving, all on 7-point scales (1 = low to 7 = high).

1.2.4. fMRI Data Acquisition and Pre-Processing

Imaging was conducted on a 3T Siemens Magnetom TimTrio Scanner, equipped with a 12-channel head coil at the Center for Brain Science at Harvard University. A laptop computer running Eprime 1.0 software (Psychology Software Tools, Pittsburg, PA) controlled stimulus display via an LCD projector, which projected onto a screen placed at the head of the MRI bore. Participants viewed the screen through a mirror fastened to the head coil. Cushions were used to minimize head movement and earplugs dampened scanner noise. Participants made responses using a five-button box placed in their right hand.

Anatomical images were acquired using a high-resolution three-dimensional magnetization-prepared rapid gradient echo sequence (MPRAGE; 176 sagittal slices, echo time (TE) = 1.64 ms, repetition time (TR) = 2,530 ms, flip angle = 7 degrees, voxel size = 1 × 1 × 1 mm). Functional images were collected using a T2* gradient echo, echo-planar imaging (EPI) sequence sensitive to blood oxygen level-dependent (BOLD) contrast (TR = 2,500 ms, TE = 25 ms, flip angle = 85 degrees, 3 × 3 mm in-plane resolution). Whole-brain coverage was obtained with 41 contiguous slices, acquired in the oblique coronal orientation. An online correction for distortion in the EPI images was conducted by acquiring two EPI images pre-scan with phase-encoding gradients in opposite directions and then computing a displacement map correcting the distortion in each voxel. Following the functional runs, we included a 6 min 12 sec resting state scan in which participants were asked to keep their eyes open while fixating on a crosshair as part of our standard protocol for an analysis that was not the focus of the current study.

Imaging data were preprocessed and statistically analyzed using SPM8 (Wellcome Department of Imaging Neuroscience, London, UK). First, data were preprocessed to remove sources of noise and artifact. Preprocessing included slice-time correction to correct for differences in acquisition time between slices for each whole brain volume; realignment within and across runs to correct for head movement; spatial normalization to the Montreal Neurological Institute (MNI) template (resampled at 2 × 2 × 2 mm voxels); and spatial smoothin at 8 mm full width at half maximum (FWHM) using a Gaussian kernel.

1.2.5. Behavioral Analysis

Behavioral analyses were conducted using repeated measures ANOVAs. To determine online changes in memories due to perspective shifting, we calculated a 2 (Condition: Shifted Perspective, Non-Shifted Perspective) × 3 (Repetition: First, Second, Third) repeated measures ANOVA separately on emotional intensity and perspective maintenance ratings, and response times to make each rating. To determine subsequent changes in the subjective ratings of memories due to perspective shifting, we calculated a 3 (Condition: Shifted Perspective, Non-Shifted Perspective, Baseline) × 2 (Study Session: One, Three) repeated measures ANOVA separately on emotional intensity, reliving, own eyes, and observer ratings. We used the Greenhouse-Geisser correction to adjust the degrees of freedom when Mauchly’s test of sphericity was violated.

1.2.6. fMRI Analysis

Fixed effects analyses included regressors at the onset of each retrieval cue in each perspective condition with a duration of 7.5 s that were modelled with a canonical hemodynamic response function (hrf). An additional regressor of no interest was included at the onset of the first rating with a duration of 5 s (i.e., the total length of the two ratings) and collapsed across both the perspective and repetition conditions, thus allowing us to effectively separate the hrf of the ratings from the retrieval periods of interest. A flexible factorial model, including repetition (first, second, third) and perspective condition (shifted, non-shifted), was used to examine random effects. We computed a linear contrast, First>Second>Third retrieval trials (c = [1 0 −1]), in order to examine the decrease in neural recruitment across the three repetitions of each memory (e.g.,van Mulukom et al., 2013). The repeated retrieval design allowed us to isolate neural regions that contribute more generally to retrieval and those that reflect differences due to updating egocentric perspective, by comparing the linear reduction in the blood oxygen level dependent (BOLD) response across repetitions in the shifted and non-shifted conditions. We used a conjunction approach to isolate common retrieval-related regions and a subtraction analysis to reveal differences in the non-shifted and shifted perspective conditions. A whole-brain analysis with a primary voxel-level threshold of P = .001 and a minimum cluster-extent threshold of k ≥ 61 voxels was used to correct for multiple comparisons at p < .05 as determined by 10000 Monte Carlo simulations (Slotnick et al., 2003). To minimize potential false positives with using cluster thresholding we incorporated the correct smoothing value (i.e. derived from the average FWHM value calculated from the group-analysis in SPM) and used a conservative primary voxel-level threshold (Eklund et al., 2016; Woo et al., 2014).

1.2.7. Linear-Mixed Effects Approach

We conducted an additional general linear model to examine the influence of neural recruitment on behavioral ratings by taking a linear mixed-effects approach (Chen et al., 2013). One of the benefits of using this particular approach is that it provides a way to examine variability across individual memories while also accounting for the fact that memories are clustered within participants. In contrast, standard analysis approaches would typically remove variability by aggregating data across individual memories and by doing so they target the level of the individual rather than the level of the memory (for further discussion see Wright, 1998).

The same onsets and durations were used as above, except separate regressors were calculated for each retrieval trial (64 memories x 3 repetitions = 192 total regressors). Average beta values within 6mm spheres centered on the peak voxels identified from the main analysis above were then extracted for each retrieval trial in each participant. We then examined how neural recruitment (either BOLD response on each trial or repetition suppression effects) predicted behavior (perspective maintenance, emotional intensity, or pre-versus-post-retrieval changes in visual perspective; for further details see Results) while accounting for the clustering of retrieval trials within subjects, differences in the average neural recruitment of ROIs across participants (i.e., inclusion of a random intercept), and potential differences among participants in the relationship between neural recruitment and behavior (i.e., inclusion of a random slope). The linear mixed-effects model contained two levels, as follows:

$$\begin{array}{ll}\text{Level}1:\hfill & {\mathrm{Y}}_{ij}={\mathrm{\beta }}_{0j}+{\mathrm{\beta }}_{1\mathrm{j}}\ast ({\text{PerspectiveCondition}}_{ij})+{\mathrm{\beta }}_{2\mathrm{j}}\ast ({\text{BOLD}}_{ij}-{\text{meanBOLD}}_j)+{\mathrm{r}}_{ij}\hfill \\ \text{Level}2:\hfill & {\mathrm{\beta }}_{0j}={\mathrm{\gamma }}_{00}+u_{0j}\hfill \\ \hfill & {\mathrm{\beta }}_{1\mathrm{j}}={\mathrm{\gamma }}_{10}\hfill \\ \hfill & {\mathrm{\beta }}_{2\mathrm{j}}={\mathrm{\gamma }}_{20}\hfill \end{array}$$

At level 1, the model is expressed such that behavior (Y_ij) on each trial is a linear combination of the intercept for each participant (β_0j), the influence of perspective condition β_1j*(PerspectiveCondition_ij), the influence of BOLD response in the ROI (β_1j*(BOLD_ij - meanBOLD_j)), and the random error (r_ij) associated with the each retrieval trial (i) in each participant (j). The mean BOLD response was grand mean centered across participants for better interpretation of the parameter estimates in the model (Kreft and others 1995). At level 2, the participant level intercepts (β_0j) are reflected by the sum of the overall mean (γ₀₀) and random deviations from the mean (u_0j) for each participant. In a maximal random effect model, we initially allowed the slopes of the predictors to vary across participants, but including these parameters decreased the fit of some of the models (as indicated by a >2 increase in AIC values) and we did not find a significant effect of variance in the slopes. Thus, here we report models in which the within-trial slope of the perspective condition and BOLD response predictors were fixed (i.e., they do not vary across participants) and thus both have a variance equal to zero. Incorporating level 2 into level 1 provides the full multilevel model:

$${\mathrm{Y}}_{ij}={\mathrm{\gamma }}_{00}+{\mathrm{\gamma }}_{10}\ast ({\text{PerspectiveCondition}}_{ij})+{\mathrm{\gamma }}_{20}\ast ({\text{BOLD}}_{ij}-{\text{meanBOLD}}_j)+u_{0j}+{\mathrm{r}}_{ij}$$

The fixed part of the model includes the overall intercept (γ₀₀) and level 1 predictors for the perspective condition (γ₁₀*(PerspectiveCondition_ij)) and the BOLD response (γ₂₀*(BOLD_ij – meanBOLD_j)), whereas the random part contains the variation in participant intercepts (u_0j) and within participant residual effect (r_ij). We also tested for the interaction of perspective condition and neural recruitment. All models were estimated using REML, separately for each ROI. A similar linear-mixed effects regression analysis was used to examine the influence of behavioral responses during retrieval on long-term changes in memories. Instead of using the BOLD responses as predictors, however, we examined how mean centered values of perspective maintenance and emotional intensity ratings predicted changes in visual perspective as the result of retrieval experience.

In order to determine the validity of the linear mixed-effects approach, we estimated the proportion of the total variance that occurs for retrieval trials clustered within participants. Across each of our models, approximately 20% of the variance in behavior was accounted for by this clustering. Thus, use of the linear mixed-effects model was deemed to be necessary for analyzing these effects. Effect sizes for the linear mixed-effects models were calculated by dividing the beta coefficient values by the standard deviations.

1.3. Results

1.3.1. Behavioral Results

The manipulation of visual perspective during memory retrieval affected subjective ratings of perspective maintenance in session 2. There was a main effect of condition on perspective maintenance ratings during fMRI scanning in session 2, F (1,28) = 21.69, p = .00007, η_p² = .44, reflecting a greater decrease in the shifted compared to the non-shifted perspective conditions (for means and SD see Table 1).¹ However, there was no main effect of repetition, F (2, 56) = .53, p = .59, η_p² = .02, or an interaction between condition and repetition, F (2, 56) = .87, p = .42, η_p² = .03. Response times to make perspective maintenance ratings were also slower overall in the shifted compared to the non-shifted perspective conditions, F (1, 28) = 4.32, p = .047, η_p ² = .13, but this effect was primarily driven by slower response times on the first retrieval trial (p = .007) as reflected by a condition x repetition interaction, F (2, 56) = 4.06, p = .023, η_p² = .13. Thus, is was more difficult to maintain a shifted than a non-shifted perspective during memory retrieval, but additional retrieval attempts did not generally increase the subjective ease with which memories were retrieved from the indicated perspective.

Table 1.

Online Ratings during fMRI Scanning in Session 2

	Non-Shifted Perspective			Shifted Perspective
	1st Trial	2nd Trial	3rd Trial	1st Trial	2nd Trial	3rd Trial
Perspective Maintenance
Mean Rating	4.15	4.20	4.19	3.59	3.62	3.58
SD	0.44	0.43	0.45	0.63	0.70	0.68
Mean RT (ms)	933.88	960.67	958.03	997.42	964.94	988.96
SD	198.10	203.61	210.23	209.75	230.84	228.03
Emotional Intensity
Mean Rating	3.05	2.96	2.90	2.87	2.75	2.66
SD	0.64	0.68	0.69	0.66	0.70	0.70
Mean RT (ms)	1003.85	951.23	934.27	1075.71	985.99	979.74
SD	201.01	184.26	190.04	237.32	191.06	192.21

Not surprisingly, there were no initial differences in subjective ratings among the conditions in session 1 because we equated memories on these qualities in each participant before assigning them to the conditions (for means and SD see Table 2). We examined changes in the phenomenology of memories due to perspective shifting during memory retrieval in two ways. First, we examined how shifting perspective contributed to online changes in the subjective ratings of emotional intensity during fMRI scanning in session 2. We found that shifting visual perspective during retrieval reduced subjective ratings of emotional intensity when compared to maintaining the same visual perspective, as reflected by a main effect of condition, F (1,28) = 18.22, p = .0002, η_p² = .39 (see Figure 2A), and slower rating responses in the shifted perspective condition, F (1, 28) = 7.91, p = .009, η_p² = .22. There was also an overall linear decrease in emotional intensity ratings with repetition, F (2,56) = 18.31, p = .00002, η_p² = .40, coupled with faster response times, F (1.376, 56) = 9.36, p = .002, η_p² = .25. However, there was no interaction between repetition and condition on emotional intensity ratings, F (2,56) = 1.67, p = .20, η_p² = .06. Thus, memories retrieved from the shifted perspective continued to be experienced with less emotional intensity on every repetition when compared to the non-shifted perspective condition.

Table 2.

Behavioral Results from Session 1 & 3

	Non-Shifted Perspective		Shifted Perspective		Baseline
	Mean	SD	Mean	SD	Mean	SD
Session 1
Remoteness (days)	417.04	134.02	421.38	145.39	432.09	138.02
Reliving	4.77	0.86	4.78	0.86	4.79	0.88
Emotional Intensity	3.77	0.87	3.77	0.89	3.81	0.90
Own Eyes Rating	6.16	0.46	6.14	0.49	6.16	0.45
Observer Rating	1.70	0.40	1.70	0.39	1.68	0.43
Rehearsal	2.67	0.73	2.67	0.76	2.69	0.72
Session 3
Reliving	4.51	0.98	4.42	1.02	4.56	1.00
Emotional Intensity	3.70	0.86	3.68	0.95	3.93	0.96
Own Eyes Rating	5.52	0.75	5.20	0.83	5.57	0.67
Observer Rating	2.65	1.08	2.98	1.14	2.62	1.05
Session 3 - Session 1
Reliving	−0.26	0.74	−0.35	0.84	−0.23	0.78
Emotional Intensity	−0.07	0.60	−0.09	0.82	0.12	0.73
Own Eyes Rating	−0.63	0.74	−0.94	0.75	−0.58	0.59
Observer Rating	0.95	0.93	1.30	0.91	0.93	0.85

Figure 2.

Behavioral results. (A) Emotional intensity ratings indicated a significant reduction in the shifted perspective condition compared to the non-shifted perspective condition, revealing that perspective shifting during retrieval leads to online memory changes. (B) The overall difference in visual perspective ratings from session 1 to session 3 indicated that there were more long-term memory changes in the shifted perspective condition compared to the non-shifted perspective or baseline conditions. (C) A linear mixed-effects regression model (see Online Methods) revealed that perspective maintenance ratings made during memory retrieval in Session 2 significantly predicted the amount of long-term memory changes (i.e., the difference in visual perspective from session 1 to session 3), but differently in each perspective condition. (D) A separate linear mixed-effects regression model revealed that emotional intensity ratings during memory retrieval in session 2 also significantly predicted the amount of long-term memory changes (i.e., the difference in visual perspective from session 1 to session 3), but differently in each perspective condition. Predicted values of the difference in visual perspective from the regression model are plotted on the y-axis and grand mean centered values of perspective maintenance (C) or emotional intensity (D) ratings are plotted on the x-axis. Each regression line indicates a separate participant with a variable intercept, and the markers on each line represent values for individual memories. Error bars indicate ± standard error.

Second, we examined how shifting perspective during retrieval biases subsequent memories by comparing subjective ratings made before retrieval (i.e., during session 1) and after retrieval (i.e., during session 3). We found that there were significant differences in the overall visual perspective that participants spontaneously adopted during memory retrieval (calculated as the difference between observer and own eyes ratings), F (2, 56) = 11.12, p = .00009, η_p² = .28 (see Figure 2B). Pairwise follow-up comparisons revealed that this significant main effect of condition was reflected by greater changes in the shifted perspective compared to non-shifted perspective, p = .003, and baseline condition, p = .0002, but no differences between the non-shifted perspective and baseline conditions, p = .57. Further inspection revealed that the change in visual perspective from session 1 to session 3 was due to significant differences between the conditions in both the own eyes perspective ratings, F (2, 56) = 8.58, p = .001, η_p² = .24, and observer perspective ratings, F (2, 56) = 12.29, p = .00004, η_p² = .31. Follow-up pairwise comparisons showed the expected decreases in the own eyes ratings and increases in observer ratings between the shifted and non-shifted conditions, as well as the shifted and baseline conditions (all p’s < .0001). Perspective shifting did not influence pre-versus post-retrieval changes in subjective ratings of reliving or emotional intensity. However, there was an overall reduction in emotional intensity across the study conditions, F (2,56) = 4.15, p = .02, η_p² = .13, which was reflected by greater reductions in the both shifted and non-shifted conditions compared to baseline (p’s < .05). Thus, shifting visual perspective during retrieval biased the egocentric perspective people adopted when subsequently remembering.

Critically, long-term changes in memories due to perspective shifting were also predicted by retrieval experience in session 2 (see Figure 2C & 2D). First, we found that subjective ratings of perspective maintenance during retrieval predicted the degree of long-term changes in perspective differently in each retrieval condition, β = .87, SE = .16, p < .0001, d = .13. In the non-shifted condition, higher ratings of perspective maintenance during retrieval protected memories from subsequent changes in perspective, β = −.62, SE = .13, p < .0001, d = .11, whereas, in the shifted condition, higher perspective maintenance ratings contributed to greater subsequent changes in the perspective of memories, β = .25, SE = .11, p = .019, d = .06. Second, we found that subjective ratings of emotional intensity during retrieval also contributed to long-term changes in perspective differently in each condition, β = .25, SE = .11, p = .037, d = .05. There was no influence of emotional intensity on perspective changes in the non-shifted perspective condition, β = .01, SE = .09, p = .893. In contrast, in the shifted perspective condition reductions in emotional intensity during retrieval predicted greater changes in the perspective of subsequent memories, β = −.23, SE = .09, p = .014, d = .06. In sum, the behavioral results provide strong evidence that shifting perspective during retrieval contributes to long-term changes in how memories are subsequently remembered, above and beyond the effects of verbatim retrieval alone or due to changes in memories over time.

1.3.2. fMRI Repetition Suppression Effects Contributing to Perspective Shifting

The main goal of the fMRI analysis was to determine neural mechanisms during retrieval that contribute to perspective shifting and their impact on memories. To achieve this goal, we compared the linear trend in the reduction in the BOLD response in the shifted perspective and non-shifted perspective conditions. As expected, we found robust common repetition suppression effects across the perspective conditions in a number of regions including lateral and medial prefrontal cortices, lateral temporal lobes, medial temporal lobes (including the hippocampus and parahippocampal cortices), parietal cortices, and posterior midline regions across both studies (see Figure 3), which overlaps with default and other networks that are frequently engaged during autobiographical memory retrieval (Andrews-Hanna et al., 2014; St Jacques et al., 2013a). Thus, these regions support retrieval-related processes irrespective of the degree of perspective shifting required, perhaps reflecting the fluency or ease of reactivating memories with repetition.

Figure 3.

Common regions supporting memory retrieval as revealed by a conjunction of the repetition suppression findings in the non-shifted and shifted perspective conditions.

In contrast, examining the difference between the perspective conditions revealed greater repetition suppression effects in other neural regions when shifting perspective compared to the non-shifted condition. There was a greater linear reduction in the BOLD response in the shifted perspective condition in both the central portion of the precuneus and right angular gyrus (see Figure 4), suggesting that these regions contributed to processes that enabled the ability to adopt an alternative visual perspective during memory retrieval. Moreover, linking these findings directly to behavior, in both central precuneus, β = .05, SE = .01, p = .0001, d = .05, and angular gyrus, β = .08, SE = .02, p = .00005, d = .06, neural recruitment on each trial predicted the ability to maintain the indicated perspective during retrieval. Repetition suppression effects were also found in bilateral prefrontal cortex, however, the influence of this region on the ability to maintain a particular perspective was more complex (see Figure S1). There were no reliable effects of right PFC on perspective maintenance ratings. However, within the left PFC there was a significant interaction between condition and BOLD response, β = .22, SE = .07, p = .001, d = .05, such that greater recruitment of this region was associated with a better ability to maintain perspective in the non-shifted condition, β = .16, SE = .05, p = .001, d = .05, but not in the shifted condition β = −.06, SE = .05, p = .23. The differential relationship between left PFC recruitment and behavior across the conditions suggests that control processes that support perspective ability may become less effective with increasing demands to shift perspective. There were no regions that showed greater repetition suppression effects in the non-shifted compared to the shifted perspective condition. The differences in repetition suppression effects between the conditions support our predictions regarding the important role of posterior parietal cortices, in particular central precuneus and right angular gyrus, in adopting an alternative visual perspective during memory retrieval.

Figure 4.

fMRI repetition suppression effects contributing to perspective shifting. There were greater repetition suppression effects (i.e., reduction in percent signal changes from the 1^st to the 3^rd retrieval trial) in both the central precuneus and right angular gyrus, indicating that neuronal populations in these regions are associated with perspective shifting. Error bars indicate ± standard error. MNI = Montreal Neurological Institute Coordinates.

1.3.3. fMRI Effects of Perspective Shifting on Memories

A central aim was to investigate how neural mechanisms that support shifts in visual perspective during memory retrieval also contribute to the potential modification of memories. To assess the involvement of central precuneus and right angular gyrus in online and subsequent changes in memories we conducted two additional analyses. First, we examined whether the degree of perspective shifting undertaken during retrieval, as reflected by the amount of repetition suppression, was associated with online changes in the emotional intensity of memories. We found that the amount of repetition suppression in the precuneus for individual memories (i.e., difference in BOLD response for first versus third repetition in each memory) predicted changes in emotional intensity across the retrieval trials depending upon the particular perspective condition, β = .08, SE = .03, p = .011. In the shifted perspective condition, β = −.06, SE = .02, p = .011, d = .06, but not the non-shifted perspective condition, β = .02, SE = .02, p = .30, repetition suppression in the central precuneus predicted greater reductions in emotional intensity during retrieval (see Figure 4A). In contrast, repetition suppression in the angular gyrus did not predict changes in emotional intensity, β = −.03, SE = .03, p = .37. Thus, greater involvement of the central precuneus when adopting an alternative perspective shifting contributed to online changes in memories during retrieval.

Second, we examined whether repetition suppression effects also predicted long-term changes in subsequent memory as reflected by the change in visual perspective of memories from session 1 to session 3 (i.e., pre- vs. post-retrieval). We found that greater repetition suppression in the precuneus predicted changes in perspective from session 1 to session 3 equally across both perspective conditions (see Figure 4B), β = .11, SE = .05, p = .019, d = .06. Thus, the degree of perspective shifting involved during retrieval contributed to persistent changes in the visual perspective of memories. In contrast, and similar to the results observed above, repetition suppression in the angular gyrus did not predict long-term changes in perspective, β = .03, SE = .08, p = .68. Together these findings reveal that the central precuneus supports the ability to adopt a particular egocentric perspective, as well as to update it, consistent with our prediction that perspective shifts during retrieval bias how memories are remembered online and in the future.

1.4 Discussion

Our findings reveal that neural mechanisms that support the ability to take alternative visual perspectives during retrieval contribute to biases in how memories are subsequently remembered. Computational models of spatial memory and imagery predict that egocentric frameworks generated during retrieval from long-term memory are represented and updated during imagined egocentric movements in space (Byrne et al., 2007). Using a repeated retrieval fMRI design, our results provide evidence that the online restructuring of egocentric frameworks that occurs with shifts in visual perspective during retrieval is also retained in subsequent memories. We found that shifting from a dominant to an alternative visual perspective during retrieval of autobiographical memories was supported primarily by posterior parietal cortices. Importantly, the extent of repetition suppression in the precuneus predicted both online and subsequent changes in memories.

Perspective shifting led to greater linear reductions across repetitions in posterior parietal cortex, including the precuneus and right angular gyrus. As a “specialized nexus” the precuneus supports the interaction between frontoparietal and default networks according to task demands (Utevsky et al., 2014), and is thus well situated to support goal directed retrieval processes (Spreng et al., 2010). The involvement of the precuneus in episodic memory retrieval has been primarily attributed to mental imagery processes, but this region has also been linked to other processes such as visuospatial and self-referential tasks (Cavanna and Trimble, 2006). One common denominator may be its role in visual perspective taking. Via its functional connections with the angular gyrus (Margulies et al., 2009) the precuneus may provide an egocentric filter from which to inspect incoming information from long-term memory (Cabeza et al., 2008; Committeri et al., 2015). This viewpoint is in line with the parietal window hypothesis, which holds that a population of neurons in posterior parietal cortices, particularly the precuneus, represents the locations of recalled landmarks from an egocentric viewpoint and manipulates these mental images in the service of planning and navigation (Byrne et al., 2007). For example, Wolbers and colleagues (2008) found that the precuneus supported the manipulation of egocentric representations resulting from navigation in space during the perception of object locations. Our findings converge with this research, and extend it by suggesting that the manipulation of mental images in the precuneus when updating egocentric perspectives also shapes long-term memory retrieval.

Another possible explanation of the findings is that decreased involvement of the precuneus reflects more general differences in difficulty across repetitions in the shifted compared to the non-shifted perspective conditions. This idea is in line with previous research suggesting that precuneus may be modulated by task difficulty (e.g., Gilbert et al., 2012; Leech et al., 2011). However, in the current study we found that maintaining an alternative perspective was subjectively judged to be equally difficult across repetitions of retrieval trials. Thus, repetition suppression effects in the precuneus cannot be readily explained by the more general accounts of decreases in the difficulty of shifting visual perspective with repetition. Moreover, we found that recruitment of precuneus predicted trial-by-trial variation in the success of maintaining a particular visual perspective—irrespective of whether memory retrieval involved a shift or no shift in perspective. Thus, a more parsimonious account is that precuneus modulates processes that support egocentric perspective taking, which are engaged more when memory retrieval involves shifting from a dominant to an alternative visual perspective compared to maintaining a dominant visual perspective.

Shifting visual perspective in autobiographical memories modified the subjective experience of memory retrieval online, and continued to affect subsequent memories even when perspective was not directly manipulated. In line with the previous literature (e.g., Berntsen and Rubin, 2006; Robinson and Swanson, 1993), we found that shifting visual perspective from an own eyes to an observer perspective reduced emotional intensity during retrieval for memories that were initially rated equally. Moreover, the amount of reduction in emotional intensity across repeated retrieval trials was predicted by the amount of repetition suppression in precuneus, suggesting that egocentric processes that support perspective shifting contributed to these behavioral differences. Shifting visual perspective during retrieval also led to subsequent changes in the visual perspective of memories—biasing spontaneous retrieval from an observer perspective for memories that were originally retrieved from a strong own eyes perspective. The behavioral and neural signatures of perspective shifting during retrieval also predicted the amount of overall changes in visual perspective demonstrating that the pattern of findings cannot easily be explained by demand characteristics (also see McIsaac and Eich, 2004). Similar effects of perspective shifting during retrieval on subsequent memories were not found for emotional intensity ratings, as has been sometimes shown (Sekiguchi and Nonaka, 2014). One possible explanation is that the influence of shifting perspective on long-term changes in emotional intensity is modulated by the emotional intensity of memories. In the current study we did not elicit particularly emotional memories, and, coupled with the unusually large number of autobiographical memories generated, this may have led to generally lower levels of emotional intensity. At any rate, our findings suggest that shifting visual perspective can modify some of the phenomenological properties of memories. It is of interest for future research to determine whether shifts in visual perspective contribute to modification in other properties of memories, such as the accuracy or the type of details recalled.

We focused on shifts in visual perspective in one direction only: from own eyes to observer. Since the recent memories used here were more strongly associated with an own eyes perspective (i.e., approximately 72% of memories initially generated) this approach allowed us to robustly test the prediction that shifting from a dominant to an alternative visual perspective would modify memories. This idea is in line with current assumptions regarding the nature of observer perspective in memories, which suggest that it reflects a transformation of memories overtime due to constructive memory processes (e.g., Butler et al., 2016; McDermott et al., 2016). Additionally, previous research suggests that shifting from an observer to an own eyes visual perspective may not confer the same changes in the phenomenology of memory retrieval as we found here. For example, some studies have shown that shifting from an observer to an own eyes perspective does not lead to a similar increase in emotional intensity as the decreases typically found with the reverse perspective shift (Berntsen and Rubin, 2006; Robinson and Swanson, 1993; Sekiguchi and Nonaka, 2014). These asymmetrical effects of visual perspective shifting may reflect a fundamental difference in the nature of memories associated with a dominant observer perspective or on the egocentric processes that operate on such memories. We also note that there are many practical issues with testing shifts in visual perspective from a dominant observer perspective to an alternative own eyes perspective in autobiographical memories in fMRI studies. For example, strong own eyes and strong observer memories will likely differ in memory remoteness and frequency, which makes it difficult to compare a sufficient number of equivalent memories.

Despite these issues, understanding the generalizability of precuneus involvement in reverse shifts in visual perspective during memory retrieval and its effects on memories will be important avenue for future research. Given the flexibility of the precuneus response during visual perspective taking, it is likely that it would supports shifts in visual perspective irrespective of the direction-especially for equally strong own eyes and observer memories. Understanding these issues could also help to explain the mixed findings noted earlier regarding the preferential response of the precuneus when contrasting own eyes and observer perspectives during autobiographical memory retrieval (Eich et al., 2009; Freton et al., 2014; Grol et al., 2017). One important step in this direction will be to control for the initial visual perspective of memories, as well as differences in demands on perspective shifting. Rather than contributing to the ability to adopt one egocentric perspective versus another during memory retrieval, the precuneus likely supports processes that enable us to inspect and manipulate the mental images that arise during remembering, thereby providing an egocentric window onto the past.

In contrast to the posterior parietal cortices, a number of other regions showed common repetition suppression effects. These included many of the regions typically recruited during autobiographical memory retrieval (Cabeza and St Jacques, 2007; Fuentemilla et al., 2014; Spreng et al., 2008; Svoboda et al., 2006), including the medial temporal lobe (hippocampus and parahippocampus), lateral temporal lobe, some regions of the lateral prefrontal cortex, anterior midline and posterior midline regions excluding the precuneus. Surprisingly, previous fMRI studies of autobiographical memory retrieval have not controlled for the visual perspective of memories, or have focused solely on memories retrieved from an own eyes perspective (but see Grol et al., 2017). These considerations may explain why precuneus is not frequently recruited during autobiographical memory retrieval (Svoboda et al., 2006), perhaps reflecting differential demands on perspective shifting.

Interestingly, the hippocampus also showed common repetition suppression effects across the perspective conditions suggesting that modification of memories in the current study was operating primarily on egocentric rather than allocentric (i.e., world-centred frame of reference) representations. Egocentric-related updating processes that occurred during retrieval may have contributed to greater fluency when later translating from allocentric to egocentric frameworks, biasing the reinstatement of similar visual perspectives in the future rather than updating allocentric memory representations (Ekstrom et al., 2014). Although the hippocampus is critical during the formation of novel allocentric representations of space it is less involved when navigating in familiar environments (Moscovitch et al., 2006), and modification of allocentric representations in the hippocampus during retrieval also depends upon the degree of updating required (Byrne et al., 2007). Thus, greater demands on allocentric representations of space during perspective shifting within long-term memory may lead to greater involvement of the hippocampus.

Our findings contribute to the growing literature on retrieval-related mechanisms that support adaptive memory updating processes (Schacter et al., 2011). Theories of memory emphasize the critical role of reactivation on the distributed brain networks that support long-term memory representations (Mcclelland et al., 1995; Winocur and Moscovitch, 2011). Memory reactivation, or the activation of latent memory trace during retrieval, contributes to updating in memories when new information is introduced (Hupbach et al., 2007). Here we show that during memory retrieval shifting visual perspective also signals the presence of new information possibly contributing to reactivation-related changes in memories. In previous work we showed that the quality of memory reactivation modulated the extent of subsequent changes in memories (St Jacques et al., 2013b). In the current study we controlled for initial differences in the quality of memory reactivation by selecting memories that were associated with equivalent levels of reliving, thus, allowing us to target how shifts in visual perspective during retrieval shapes memories. We showed that shifting perspective during retrieval updated the visual perspective of subsequent memories—ultimately biasing the preferred spontaneous visual perspective of memories. This distortion of visual perspective in memories due to explicit retrieval, along with offline reactivation during periods of sleep and awake rest (Oudiette et al., 2013), could contribute to the natural transformation of visual perspective from own eyes to observer perspectives as memories become more remote (Butler et al., 2016). An important question for future research will be to determine whether the transformation of memories due to perspective shifting can be reversed (Butler et al., 2016).

1.5. Conclusions

By manipulating whether people adopted a dominant or alternative visual perspective in autobiographical memories, we showed that shifting visual perspective during retrieval shapes remembering and is supported by neural recruitment in posterior parietal cortices. In particular, precuneus contributed to online reductions in emotional intensity, as well as the distortion of visual perspective in subsequent memories. We suggest that the manipulation of mental images in precuneus when shifting visual perspective during autobiographical memory retrieval can shape and potentially restructure how we remember.

Figure 5.

Neural predictors of online and long-term memory changes. (A) A linear mixed-effects regression model revealed that the amount of repetition suppression (i.e., difference in BOLD response from the first to the third retrieval trial) in the central precuneus predicted the degree of online changes in memories, as reflected by the reduction in emotional intensity across repetitions of individual memories during retrieval in session 2, differently in each perspective condition. (B) A separate linear mixed-effects regression model revealed that the amount of repetition suppression in the central precuneus also predicted greater differences in long-term changes in memories, as reflected by differences in visual perspective from session 1 to session 3, equally in both conditions. For visualization purposes, the predicted values of the difference in emotional intensity (A) or the difference in visual perspective (B) from each regression model are plotted on the y-axis and grand mean centered values of the repetition suppression BOLD response are plotted on the x-axis. Each regression line indicates a separate participant with a variable intercept, and the markers on each line represent values for individual memories.