Abstract
Memory retrieval involves the reactivation of processes that were engaged at encoding. Using a Generalized Linear Model to test for effects of valence, our prior study (Bowen & Kensinger, in press) suggests memory for information previously encoded in a negative context reengages sensory processing regions at retrieval to a greater extent than positive. Here, we used partial least squares analyses of the same dataset to determine whether this valence-specific processing was one of the dominant patterns in the retrieval data. Trials previously paired with a face revealed a strong pattern of emotion that did not vary by valence, but for trials previously paired with a scene, an extensive network of regions was active during recollection of trials paired with negative content. These same regions were negatively correlated with recollection of trials paired with positive content. These results confirm that, despite no emotional content present during the time of retrieval, strong patterns of emotional study context are present in the data. Moreover, at least for trials paired with scenes at encoding, valence-specific networks are linked to episodic memory recollection, providing further support for recapitulation of sensory processing during recollection of negative emotional information.
Keywords: Memory, Emotion, Valence, Functional Connectivity, Reactivation, PLS
Introduction
Several prominent memory theories suggest that when retrieving a memory, the cognitive and neural processes engaged during encoding are reactivated or recapitulated at the time of retrieval [1–3]. Memories for emotional events—negative events in particular—are often accompanied by a feeling of re-experience [4; 5] potentially driven by greater recapitulation of those encoding processes at the time of retrieval. In our prior work [6] we demonstrated that recollected verbal stimuli previously encoded in a negative pictorial context indeed showed more encoding-to-retrieval overlap compared to stimuli encoded in a positive or neutral context. Further, consistent with related work from our lab [7] using degraded line-drawing images of previously encoded full color images, we demonstrated that the valence of the previous encoded context led to differences in regions of the brain that showed this recapitulation. Specifically, stimuli previously encoded in a negative context recapitulated regions in the ventral visual processing stream to a greater extent than positive or neutral and this recapitulation of sensory regions was associated with a subjective memory enhancement such as recollection of visual sensory details.
In this prior work [6; 7] these valence differences were analysed using univariate approaches with specific contrasts to probe for valence effects in memory recapitulation. While these results reflect differences in activation between these conditions, they do not provide information about the dominance of these valence patterns within the data, nor do they provide evidence about the functional connectivity of these neural regions. It is well established that memory retrieval is not localized to specific and separate brain areas, but relies on a distributed network of regions that are functionally connected and interact to support this complex process [8–10]. Instead of focusing on activation differences between task conditions, the current study utilizes a multivariate approach—partial least squares (PLS)—to investigate spatial-temporal whole-brain patterns of network connectivity inherent in the memory retrieval data. PLS identifies latent variables (LVs) that describe the covariance between experimental task conditions and the fMRI time-series across all brain voxels [11–14].
Different PLS analyses can be used to answer different experimental questions. Our first aim was to determine whether PLS would reveal dominant valence patterns during retrieval of neutral stimuli previously encoded in an emotional context, using a dataset that previously revealed valence differences in univariate analyses [6]. Given that no emotional content is re-presented at the time of retrieval, one hypothesis would be that valence effects would be small, or even non-existent. An alternate hypothesis is that the stronger pattern to emerge would be one that characterizes arousal, not differentiating negative and positive valence. Yet our prior results [6; 7] led us to hypothesize that valence-specific retrieval patterns would emerge. Second, we were interested in examining functional connectivity at retrieval, again with a particular interest in whether valence patterns would emerge in this connectivity. Although the same alternate hypotheses existed for connectivity—there might be minimal, or no, effects of emotion, or the dominant effect might be related to arousal—based on our findings of greater recapitulation of visual processing regions during retrieval of stimuli previously encoded in a negative context [6; 7] and in line with tenants of our “NEVER” model of emotional memory [15], we predicted that network connectivity correlated with visual processing regions would vary by emotional valence and would be stronger for stimuli encoded in a negative context.
Methods
All procedures were approved by the Boston College Institutional Review. Protocol details are delineated in Bowen & Kensinger (in press). In brief, the present data are from participants (N = 181; 12 females) who were on average 23.3 (SD = 3.66) years old. After being scheduled for an MRI appointment, participants were directed to an online survey (via survemonkey.com) asking them to rate all the face and scene stimuli (see Bowen & Kensinger, in press for more information). For each participant, 8 negative, 8 positive and 8 neutral images (4 faces, 4 scenes per valence) were selected as experimental stimuli. The ratings ensured each participant had an individualized set of experimental stimuli they deemed emotional.
The task consisted of 4 encoding-retrieval blocks in the fMRI scanner. During encoding, positive, negative and neutral faces or scenes were presented one at a time along with an unrelated 6-letter neutral noun (the contextual valence presented with each word was varied across participants). Participants were asked to make a face or scene judgment (48 trials per block; 24 scenes, 24 faces) but never asked to explicitly associate the image and word, nor asked to respond to the emotional aspects of the stimulus. During retrieval, participants were given only the neutral word cue (48 target, 40 distractor items per block) and asked to make a Remember, Know, or New judgment based on instructions from a prior study [16]. Importantly, retrieval instructions did not explicitly mention bringing back to mind any specific content (e.g., associated face or scene) from the encoding phase, but participants could make a ‘remember’ response so long as they could bring back to mind any episodic detail from encoding. See Figure in Supplemental Digital Content 1 for a schematic of the encoding and retrieval task.
Analysis
The retrieval portion of the task was analysed using two different PLS analyses: 1) Mean-centered PLS: a data-driven analysis correlating whole-brain patterns of activity with experimental design, helpful for exploring the data without specific contrasts or apriori hypotheses and can be used to identify regions of interest or “seed regions” to be used in subsequent analyses with more hypothesis driven questions [17]; 2) Seed-voxel PLS: a technique that provides a measure of task-related functional connectivity by correlating brain activity in a specified seed region(s) and brain activity across the whole brain as a function of experimental conditions [17].
In both analyses, latent variables (LVs) are produced with singular value decomposition using the correlation maps of each condition. LVs indicate the pattern of correlation or connectivity that characterizes each condition across subjects [18]. Each LV consists of a singular value (i.e., the proportion covariance accounted for by the LV), a singular image (i.e., the pattern of brain activity that covaries with seed activity and/or conditions) and a singular profile (i.e., a bar graph representing the correlation between activity in the seed voxel and activity in the brain regions identified in the singular image as a function of the experimental conditions) [12; 17]. Within the singular image, each voxel is assigned a salience value representing how strongly that voxel represents the experimental effect expressed in the singular profile [17]. For example, positive salience (represented as warm colors in the singular image) indicates the voxel expresses the effect and positively correlates with the positive-going condition in the singular profile bar graph. A negative salience (represented as cool colors in the singular image) indicates the voxel expresses the reverse effect and negatively correlates with the conditions with positive-going bars in singular profile while positively correlating with the negative-going bars in the singular profile. The assignment of a condition to positive or negative salience is arbitrary. The significance of each LV was determined using 500 permutations [11] and the reliability of the voxel salience was determined using bootstrap estimates of the standard error using 100 bootstrap samples. Voxels were considered to robustly contribute to the brain pattern identified by the LV if the bootstrap ratio exceeded ±3.0 (equivalent to p <.005 [19]). Bootstrap estimates were also used to derive 95% confidence intervals calculated around the LV correlation profiles to provide a measure of the reliability of the correlation pattern [20]. Overlapping confidence intervals indicate that conditions and/or seeds do not significantly differ from each other. Confidence intervals that overlap with zero indicate that the condition and/or seed is not significantly different from zero, and does not reliably contribute to the pattern of results [19].
Results
For details about fMRI data acquisition and pre-processing steps as well as a description of the behavioral memory results, the reader is referred to Bowen & Kensinger, in press [6]. In short, data were collected at the Harvard Center for Brain Science on a Siemens Tim Trio 3 Tesla scanner with a 32-channel head coil. All functional images were reoriented, realigned, co-registered, spatially normalized to the Montreal Neurological Institute template, (resampled at 3 mm during segmentation and written at 2 mm during normalization), and smoothed using a 4 mm isotropic 12 Gaussian kernel. Global mean intensity and motion outliers were identified using Artifact Detection Tools (ART; available at www.nitrc.org/projects/artifactdetect).
Mean-centered analysis collapsed across faces and scenes
A mean-centered PLS analysis was conducted using activity from the presentation of the neutral word stimulus at retrieval. Similar to analysis approach 2 in our prior study [6]"know” and “new” responses to old items were collapsed together and analyzed separately from “remember/recollection” hits. These two memory classifications were made for each valence (negative, positive, neutral), collapsed across face and scene context, resulting in 6 task conditions (see Figure 1). The results of this analysis yielded one significant latent variable (p < .001) that accounted for 43.93% of the covariance and identified a pattern that differentiated remember responses from the know/new responses, across valence. No other significant patterns emerged. From this analysis, bilateral seeds in the temporal lobe (MNI:×= −60, y = −44, z = −12;×= 62, y = 44, z = −14 [BA 20]) were identified for a subsequent seed-voxel analysis. These seed regions, illustrated in the Figure in Supplemental Digital Content 2, were positively correlated with recollection, but were negatively correlated with the know/new response and showed peak activation in timepoints (i.e., lags) 3 and 4.
Mean-centered analysis separate for faces and scenes
Before conducting the seed analysis, two additional mean-centered PLS analyses were conducted with the same 6 task conditions detailed above separately for face and scene trials. The results of these analyses are not reported as they were only used to extract the BOLD values from the specified seed voxels across 8 timepoints after the presentation of the stimulus to capture activity during retrieval trials. In the extraction, a neighbourhood voxel size of 2 was chosen, which specifies the distance from the seed (in voxels) to be included in the seed analysis. The extracted data for each seed from the peak timepoint (lag 4) was then entered into the seed analysis described below.
Seed-voxel analysis for faces
The extracted BOLD data from the bilateral middle temporal seeds was entered into a seed-voxel functional connectivity analysis to examine patterns of valence and memory for trials that had been previously paired with a face. The analysis revealed two significant LVs. The first LV accounted for 44.6% of the covariance, p < .001, and identified a pattern of functional connectivity that positively correlated with all conditions. Generally, the network included regions known to be involved in memory retrieval [21], including the middle temporal gyrus, as well as medial frontal gyrus, angular gyrus, and posterior cingulate. See the table in Supplemental Digital Content 3, for the peak coordinates for all regions identified in LV1.
The second LV accounted for 12.2% of the covariance, p = .02, and the singular profile presented in Figure 1 identified a main effect of memory and a main effect of emotion. As depicted in warm colors in Figure 1, during lags 3 and 4, the two seeds positively correlated with a network of regions (positive salience in the table, Supplemental Digital Content 3) including the left frontal lobe, right middle frontal gyrus, and areas of the ventral visual stream including bilateral inferior and middle temporal gyrus, precuneus as well as posterior cingulate2. The main effect of memory reflected the greater engagement of this network for recollected trials than non-recollected trials, although this pattern was driven by the recollection of stimuli previously encoded in a negative or positive (not neutral) context. The main effect of emotion reflected the fact that this network was engaged for negative and positive recollected stimuli, and to a lesser extent, non-recollected negative trials (left seed only), while it was not significantly engaged for neutral stimuli.
A later pattern at lags 5 and 6 (depicted in Figure 1 in cool colors) revealed a network of regions (negative salience in the table, Supplemental Digital Content 3) that positively correlated with know/new responses for positive facial context, but that were negatively correlated during recollection of items previously encoded with a positive or negative face. Only the right seed significantly contributed to this network (the confidence intervals in the singular profile representing the left seed and both bars of neutral trials are overlapping with zero). It is worth noting that the third LV from this analysis was a valence pattern such that negative recollection pulled apart from all other conditions, however it failed to reach significance (p = .29).
Seed-voxel analysis for scenes
The same seed-voxel analysis was conducted to examine patterns of valence and memory for trials that had been previously paired with a scene. Like the analysis with faces, the first LV accounted for 39.9% of the covariance, p < .001, and identified a pattern of functional connectivity that correlated with all conditions. The pattern included a network of regions known to be involved in memory retrieval including extensive bilateral frontal and temporal gyri, fusiform gyrus, precuneus, and posterior cingulate. See the table, Supplemental Digital Content 4 for the peak coordinates of the regions identified in LV1.
The second pattern identified in the data accounted for 12.1% of the covariance, p = .05, and as depicted in the singular profile in Figure 1, revealed an Emotion × Memory interaction such that, for negative and positive trials, the pattern of network engagement differed depending on whether the information was recollected or not, whereas for neutral, there was no such memory effect (compare solid and striped bars on Figure 1). More specifically, negative and neutral recollection engaged (see warm colors in Figure 1 and positive salience in the table in Supplemental Digital Content 4) a set of ventral visual processing stream regions including middle and superior temporal gyrus, as well as the precuneus, inferior parietal lobule, and regions in bilateral frontal cortices (at Lags 1 and 2) and the posterior cingulate, angular gyrus, parahippocampal gyrus and bilateral insula (at Lags 5 and 6). However, this pattern was present for all neutral trials (even when they are not recollected), whereas these regions were specifically related to recollection (and not to know/new responses) for the negative trials. Interestingly, these same regions showed a memory-related pattern for positive trials, but the pattern was in the opposite direction as negative trials: They negatively correlated with the recollection of items previously paired with a positive context and positively correlated with know/new responses to those items. The regions that corresponded with recollection of positive trials (cool colors in Figure 1 and negative salience in the table, Supplemental Digital Content 4) were a region in the middle frontal gyrus at lag 1, and a region in right middle frontal gyrus at lag 5; these regions were instead associated with know/new responses to negative trials.
Discussion
Prior research has shown that retrieval data can reveal effects of the emotional study context [23–25] and that such effects can vary according to the emotional valence of the studied item [6; 7]. The present study provides even stronger evidence for effects of emotion on retrieval signatures by demonstrating that—when data are analyzed in a data-driven manner—effects of emotion emerge as one of the most dominant patterns in retrieval data. There has been discussion about whether these emotion effects are implicit [23–25], present regardless of memory success, but the data presented here identified patterns of emotion that were linked to recollection and episodic memory success.
Examining functional correlations with ventral visual processing regions, both faces and scenes showed a significant LV that revealed a pattern of emotion. For faces, the seeds positively correlated with a network that included regions in frontal, temporal and parietal regions, but this was true for recollected items previously encoded in either a negative or positive face context, and did not distinguish between negative and positive valence. For scenes, the seeds positively correlated with other regions in the ventral visual processing stream in valence-specific ways: Most notably, these regions were positively correlated with recollection of negative trials and inversely correlated with recollection of positive trials. Interestingly, although regions of the ventral visual processing stream were engaged for neutral trials, they did not relate to recollective success for those items. These findings are consistent with our prior work [6; 7] and the tenants of our “NEVER” model of emotional memory [15], that sensory processing and reactivation of the sensory regions during retrieval is disproportionately important for negative compared to positive memories, although in these analyses, this pattern was particularly strong when the context was a scene rather than a face.
The present study suggests a few opportunities for future research. First, why scenes and faces3 differed in the presence of valence specificity requires further study. Second, the present results suggest that although ventral visual stream regions are recruited for negative, positive, and neutral information, their relation to memory can differ. For information studied with scene contexts, these regions related positively to recollection for negative trials, related negatively to recollection for positive trials, and showed similar recruitment for neutral trials regardless of whether items were recollected or judged to be known/new. This pattern may suggest that the link between reactivation of sensory processing and a subjective feeling of re-experience can be particularly strong for negative memories—even when negative information is not re-presented at retrieval. A more explicit test of this link to feelings of re-experience is an interesting avenue for future work.
In conclusion, using a data-driven method of analysis, the results confirm that retrieval processes engaged by neutral cues can be strongly affected by the emotional content present during encoding. The data further suggest that not only can arousal of study content be detected during later retrieval processes but so can the valence. Research in the emotion memory literature often focuses on arousal, and collapses across negative and positive valence, but these results indicate that patterns of network activity can differ for negative and positive valence. These data provide more evidence for our “NEVER” model [15] of episodic memory retrieval which goes beyond current models of emotional memory to provide a framework for valence-specific recapitulation effects.
Supplementary Material
Acknowledgments
The authors wish to thank Sarah Scott and Haley DiBiase for help with data collection, as well as Tammy Moran and Dr. Ross Mair from the Harvard Center for Brain Science.
Funding: This work was supported by a National Institute of Health Grant awarded to EAK [R01MH080833].
Footnotes
Conflicts of Interest: None declared.
One participant included in Bowen & Kensinger (in press) was excluded from these analyses due to extreme outlier brain scores in the mean-centered PLS analysis.
See Petrides (2002; [22]) for an interesting discussion of the links between ventrolateral frontal regions and ventral temporal regions during active retrieval of visual context information.
It is possible this difference in connectivity for faces and scenes was due to the specific seed region chosen. However, even when using bilateral fusiform seeds (known to be involved in face processing), the results reported here with the medial temporal seed held and no data patterns consistent with valence effects were revealed for faces. Other seed regions were not explored, and so it remains plausible that valence-related effects would be revealed in some networks. Given that the 3rd LV for faces revealed a (nonsignificant) valence effect, it is also plausible that such effects are present but weaker for the face trials and would reach significance with more power.
References
- 1.Tulving E, Thomson D. Encoding specificity and retrieval process in episodic memory. Psychol. Rev. 1973;80(5):352–373. [Google Scholar]
- 2.Morris CD, Bransford JD, Franks JJ. Levels of processing versus transfer appropriate processing. J. Verbal Learning Verbal Behav. 1977;16(5):519–533. [Google Scholar]
- 3.Buckner RL, Wheeler ME. The cognitive neuroscience of remembering. Nat. Rev. Neurosci. 2001;2(9):624–634. doi: 10.1038/35090048. [DOI] [PubMed] [Google Scholar]
- 4.Phelps EA, Sharot T. How ( and Why ) Emotion Enhances the Subjective Sense of Recollection. 2008;17(2):147–152. doi: 10.1111/j.1467-8721.2008.00565.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Kensinger EA. What Factors Need to be Considered to Understand Emotional Memories? Emot. Rev. 2009;1(2):120–121. doi: 10.1177/1754073908100436. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bowen HJ, Kensinger EA. Recapitulation of emotional source context during memory retrieval. Cortex. doi: 10.1016/j.cortex.2016.11.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kark SM, Kensinger EA. Effect of emotional valence on retrieval-related recapitulation of encoding activity in the ventral visual stream. Neuropsychologia. 2015;78:221–230. doi: 10.1016/j.neuropsychologia.2015.10.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.McIntosh AR. Understanding neural interactions in learning and memory using functional neuroimaging. Ann. N. Y. Acad. Sci. 1998;855:556–571. doi: 10.1111/j.1749-6632.1998.tb10625.x. [DOI] [PubMed] [Google Scholar]
- 9.Nyberg L, Persson J, Habib R, Tulving E, McIntosh aR, Cabeza R, et al. Large scale neurocognitive networks underlying episodic memory. J. Cogn. Neurosci. 2000;12(1):163–173. doi: 10.1162/089892900561805. [DOI] [PubMed] [Google Scholar]
- 10.Mesulam M-M. Large-scale neurocognitive networks and distributed processing for attention, language, and memory. Ann. Neurol. 1990;28(5):597–613. doi: 10.1002/ana.410280502. [DOI] [PubMed] [Google Scholar]
- 11.McIntosh AR, Bookstein FL, Haxby JV, Grady CL. Spatial Pattern Analysis of Functional Brain Images Using Partial Least Squares. Neuroimage. 1996;3(3):143–157. doi: 10.1006/nimg.1996.0016. [DOI] [PubMed] [Google Scholar]
- 12.Burianova H, McIntosh AR, Grady CL. A common functional brain network for autobiographical, episodic, and semantic memory retrieval. Neuroimage. 2010;49(1):865–874. doi: 10.1016/j.neuroimage.2009.08.066. [DOI] [PubMed] [Google Scholar]
- 13.Spaniol J, Bowen HJ, Wegier P, Grady C. Neural responses to monetary incentives in younger and older adults. Brain Res. 2015;1612:70–82. doi: 10.1016/j.brainres.2014.09.063. [DOI] [PubMed] [Google Scholar]
- 14.McLntosh AR, Gonzalez-Lima F. Structural equation modeling and its application to network analysis in functional brain imaging. Hum. Brain Mapp. 1994;2(1–2):2–22. [Google Scholar]
- 15.Bowen HJ, Kark SM, Kensinger EA. NEVER Forget: Negative emotional valence enhances recapitulation. Psychon. Bull. Rev. doi: 10.3758/s13423-017-1313-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Rajaram S. Remembering and knowing: two means of access to the personal past. Mem. Cognit. 1993;21(1):89–102. doi: 10.3758/bf03211168. [DOI] [PubMed] [Google Scholar]
- 17.Habib R, McIntosh AR, Wheeler MA, Tulving E. Memory encoding and hippocampally-based novelty/familiarity discrimination networks. Neuropsychologia. 2003;41(3):271–279. doi: 10.1016/s0028-3932(02)00160-4. [DOI] [PubMed] [Google Scholar]
- 18.McIntosh AR. Mapping cognition to the brain through neural interactions. Memory. 1999;7(5–6):523–548. doi: 10.1080/096582199387733. [DOI] [PubMed] [Google Scholar]
- 19.Grady CL, Grigg O, Ng C. Age differences in default and reward networks during processing of personally relevant information. Neuropsychologia. 2012;50(7):1682–1697. doi: 10.1016/j.neuropsychologia.2012.03.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.McIntosh A, Lobaugh NJ. Partial least squares analysis of neuroimaging data: Applications and advances. Neuroimage. 2004;23(SUPPL. 1):250–263. doi: 10.1016/j.neuroimage.2004.07.020. [DOI] [PubMed] [Google Scholar]
- 21.Rugg MD, Vilberg KL. Brain networks underlying episodic memory retrieval. Curr. Opin. Neurobiol. 2014;23(2):255–260. doi: 10.1016/j.conb.2012.11.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Petrides M. The mid-ventrolateral prefrontal cortex and active mnemonic retrieval. Neurobiol. Learn. Mem. 2002;78(3):528–538. doi: 10.1006/nlme.2002.4107. [DOI] [PubMed] [Google Scholar]
- 23.Jaeger A, Rugg MD. Implicit effects of emotional contexts: An ERP study. Cogn. Affect. Behav. Neurosci. 2012;12(4):748–760. doi: 10.3758/s13415-012-0110-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Maratos EJ, Dolan RJ, Morris JS, Henson RN, Rugg MD. Neural activity associated with episodic memory for emotional context. Neuropsychologia. 2001;39(9):910–920. doi: 10.1016/s0028-3932(01)00025-2. [DOI] [PubMed] [Google Scholar]
- 25.Jaeger A, Johnson JD, Corona M, Rugg MD. ERP correlates of the incidental retrieval of emotional information: Effects of study-test delay. Brain Res. 2009;1269:105–113. doi: 10.1016/j.brainres.2009.02.082. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.