Skip to main content
NCBI home page
Human Brain Mapping logoLink to Human Brain Mapping
. 2021 Jan 26;42(7):2115–2127. doi: 10.1002/hbm.25353

Strong correspondence between prefrontal and visual representations during emotional perception

Doyoung Park 1, Taehyun Kim 1, Sue‐Hyun Lee 1,2,✉
PMCID: PMC8046073  PMID: 33496375

Abstract

Emotion is thought to cause focal enhancement or distortion of certain components of memory, indicating a complex property of emotional modulation on memory rather than simple enhancement. However, the neural basis for detailed modulation of emotional memory contents has remained unclear. Here has been shown that the information processing of the prefrontal cortex differentially affects sensory representations during experience of emotional information compared with neutral information, using functional magnetic resonance imaging (fMRI). It was found that during perception of emotional pictures, information representation in primary visual cortex (V1) significantly corresponded with the representations in dorsolateral prefrontal cortex (dlPFC). This correspondence was not observed for neutral pictures. Furthermore, participants with greater correspondence between visual and prefrontal representations showed better memory for high‐level semantic components but not for low‐level visual components of emotional stimuli. These results suggest that sensory representation during experience of emotional stimuli, compared with neutral stimuli, is more directly influenced by internally generated higher‐order information from the prefrontal cortex.

Keywords: emotional memory, emotional perception, fMRI, prefrontal cortex, sensory representations, visual cortex

We show that the visual representation during perception of emotional stimuli reflects more higher‐order information from the prefrontal cortex than that during perception of neutral stimuli. Furthermore, participants with greater correspondence between visual and prefrontal representations showed better memory for high‐level semantic components but not for low‐level visual components of emotional stimuli. Our results suggest that sensory representation during experience of emotional stimuli compared to neutral stimuli, is more directly influenced by internally generated higher‐order information from the prefrontal cortex.

graphic file with name HBM-42-2115-g001.jpg

1. INTRODUCTION

Memory for emotional experiences is essential to the survival of humans and other animals, because remembering such events enables individuals to avoid potential harm or to increase the likelihood of benefit from a given situation (Nairne, Thompson, & Pandeirada, 2007). Emotional events and stimuli are thought to be more strongly encoded and more likely to be retrieved compared with neutral events and stimuli (Bradley, Greenwald, Petry, & Lang, 1992; Brown & Kulik, 1977; Johansson, Mecklinger, & Treese, 2004; Kensinger & Corkin, 2003; Murty, Ritchey, Adcock, & LaBar, 2010). However, accumulated evidence also shows that not all sensory details, but instead only certain focused aspects of them are enhanced while other aspects are not affected or even distorted (Bookbinder & Brainerd, 2017; Brainerd, Stein, Silveira, Rohenkohl, & Reyna, 2008; Kaplan, Van Damme, Levine, & Loftus, 2016; Kensinger, 2009; Kensinger, Garoff‐Eaton, & Schacter, 2007; Mather, 2007; Mather & Sutherland, 2011; Neisser & Harsch, 1992; Sharot, Delgado, & Phelps, 2004; Talarico & Rubin, 2003). Moreover, different level of processing (e.g., deeper vs. shallower level of processing) could be differently affected by emotion (Hayes et al., 2010; Ritchey, Bessette‐Symons, Hayes, & Cabeza, 2011). These results suggest a complex property of emotional modulation on memory rather than simple enhancement. The effect of emotion can be dissociable for different aspects or processing levels, even for the same event or stimulus. However, the neural basis of the detailed modulation of emotional memory has remained unclear.

While the amygdala is known to play a central role in various emotional effects during memory processes (Canli, Zhao, Brewer, Gabrieli, & Cahill, 2000; Dolcos, Labar, & Cabeza, 2004a; Kensinger & Corkin, 2004; Kensinger & Schacter, 2006; Ritchey, Dolcos, & Cabeza, 2008), research on the elaborate modulation of emotional information emphasizes the role of the prefrontal cortex (PFC) (Dolcos & Denkova, 2014; Dolcos, Iordan, & Dolcos, 2011; Dolcos, Labar, & Cabeza, 2004b; LeDoux & Pine, 2016). It has been suggested that the activity of the prefrontal cortex is related to higher‐order cognitive processes, including semantic processing and memory control during the encoding or retrieval of emotional information (Canli, Desmond, Zhao, & Gabrieli, 2002; Dolcos et al., 2004a; Dolcos & Denkova, 2014). Prior studies also suggest that both enhancing and impairing effects of emotion on memory depend on how the top‐down PFC system interacts with the direct emotional system including the amygdala (Dolcos & Denkova, 2014). Moreover, the PFC has been thought to be a key region for top‐down control (Curtis & D'Esposito, 2003; Funahashi, 2001; Miller & D'Esposito, 2005) during sensory perception. The perception depends on the internally generated top‐down and externally derived bottom‐up interactions (Gilbert & Li, 2013; Gilbert & Sigman, 2007), and the PFC has been considered as the “top” of the top‐down system (Miller & D'Esposito, 2005). Therefore, based on these prior research findings, one plausible scenario is that the top‐down processing of PFC may affect information processing in the sensory system more strongly during the encoding of emotional memory compared with neutral memory, which allows the detailed modulation of sensory representations.

Here we propose that the higher‐order information processing of the prefrontal cortex is more strongly reflected in the sensory representations of emotional information, compared with neutral information. To compare information processing between the dlPFC and the V1 directly, one way is to use representational similarity analysis (RSA) (Kriegeskorte, Mur, & Bandettini, 2008). To do this, we monitored the cortical activity in participants while they viewed emotional and neutral pictures, using functional magnetic resonance imaging (fMRI) (Figure 1a). Additionally, in order to determine whether the differential influence of top‐down processing is specific to the encoding process or can also occur during retrieval, we also collected cortical activity data while the participants retrieved the pictures. Given the more robust and consistent effect of negative emotion across prior studies than positive emotion (Bowen, Kark, & Kensinger, 2018; Kensinger, 2009; Ochsner, 2000), we focus on the perception and retrieval of negative emotional pictures. In the RSA, representational dissimilarity matrices (RDMs), which served to characterize the information carried by a given representation of the corresponding region, were first derived for the dlPFC and the V1 (Kriegeskorte et al., 2008). Subsequently, the correlations between the RDMs of the dlPFC and V1 were calculated as a measure of the representational relationship between these regions (Kriegeskorte et al., 2008). If information processing of the dlPFC is more strongly reflected in the V1, there should be a greater representational relationship between the dlPFC and V1. Accordingly, we tested whether the representations of the dlPFC and V1 carry shared information. Furthermore, we also investigated the relationship between the RSA data and subsequent memory performance outcomes.

FIGURE 1.

FIGURE 1

Open in a new tab

Experimental design and subjective ratings. (a) Cortical activity was monitored while participants perceived or retrieved emotional and neutral pictures inside scanner. On each perception trial, an auditory cue was followed by the visual presentation of a picture. The participants were instructed to view the pictures. The perception trials were interleaved with the retrieval trials. The retrieval trials were identical to the perception trials except that no visual images were presented. The participants were asked to retrieve the specific picture given by the auditory cue. The intertrial interval (ITI) was randomized between 4 and 20 s. (b) Upper panel: Subjective affective ratings to the emotional or neutral pictures. The participants subjectively rated the emotional valence to the emotional or neutral pictures before (prescan) and after (postscan) the scan. The valence rating score scaled from “very pleasant (1)” to “very unpleasant (5).” Lower panel: Subjective vividness ratings to the emotional or neutral pictures. The vividness rating score scaled from “perfectly vivid as viewing (1)” to “remember but no visual information at all (5).” **p < .01. n.s., not significant. Error bars indicate between‐subjects SEM

2. MATERIALS AND METHODS

2.1. Participants

About 25 healthy participants (8 females) with a mean age of 22.96 (SEM = 0.39) participated in the fMRI experiment. Another 21 healthy participants (6 females) with a mean age of 21.81 (SEM = 0.47) participated in the additional behavioral experiment. All participants were native Korean speakers and right‐handed, and they all had normal or corrected to normal vision. All participants provided written informed consent for the procedure in accordance with protocols approved by the KAIST Institutional Review Board.

2.2. Experimental design

2.2.1. Stimuli

About 10 pictures selected from International Affective Picture System (IAPS) were used in the experiment. The IAPS image database provides mean valence scores, which were rated by large groups of participants (Lang, Bradley, & Cuthbert, 2008), for each image, which range from 1 (negative) to 9 (positive). Based on the mean valence scores, negative emotional picture set (mean valence = 2.15, SEM = 0.14) and neutral picture set (mean valence = 5.76, SEM = 0.19) were selected, and the IAPS numbers of the selected pictures were 9433, 6313, 9163, 6821, 2053, 2393, 2396, 2579, 2235, and 2036 listed in order of increasing valence. Every picture of the both negative emotional and the neutral sets contained people in certain events. The picture size was 1,024 × 768 pixel, and the pictures were presented in 8 × 6 degrees of visual angle throughout the experiments. For each picture, a 2‐syllable Korean meaningless pseudoword sound (a woman's voice) was used as a recall auditory cue. We used auditory cues instead of visual cues to minimize contamination of cue information especially in the visual cortex, and the meaningless pseudowords were chosen because real words can evoke any prior knowledge or experiences related to the words. In Figure 1a, the images from the Geneva Affective Picture Database (GAPED) (Dan‐Glauser & Scherer, 2011) and from the Public Health Image Library, licensed under the Creative Commons Zero (CC0) license, are used for illustration purpose only.

2.2.2. Prescan training

Prior to scanning, participants were familiarized with the emotional and neutral pictures and the corresponding auditory cues. During this familiarization, on each trial, each picture was presented for 1 s after an auditory cue delivery for 0.75 s. Between trials, there was a 4‐s intertrial interval (ITI). This familiarization consisted of 5 blocks of 20 trials. After each block except for the last block, the participants were given a forced choice test. In the test, they were asked to choose the picture associated with the presented auditory cue between four choices (two emotional and two neutral familiarized pictures). The participants showed strong performance in the last test (98.40 ± 1.13% for emotional pictures; 99.20 ± 0.82% for neutral pictures). Additionally, the participants also rated their affective valence and the subjective vividness of the retrieved picture for each given cue. The valence rating score scaled from “very pleasant (1)” to “very unpleasant (5),” and the vividness rating score scaled from “perfectly vivid as viewing (1)” to “remember but no visual information at all (5).”

2.2.3. fMRI experiment

The main task consisted of four runs, and each run involved four trial types, emotional perception, neutral perception, emotional retrieval, and neutral retrieval, presented in a fully interleaved event‐related fashion (Figure 1a). Throughout the four runs, participants were instructed to maintain fixation on a central cross. On each trial of the emotional and neutral perception types, the white fixation cross first changed to blue, indicating onset of the perception trial (Figure 1a). After 1.25 s, the participants heard a 0.75‐s long cue immediately followed by the associated image for 2 s. Between trials, there was a variable intertrial interval of 4–20 s. During the emotional perception trials, the associated images were negative emotional pictures while the images were neutral pictures during the neutral perception trials. The participants were simply instructed to passively view the images during both the emotional and neutral perception trials. The retrieval trials were identical to the perception trials except that the white fixation cross first changed to red and no visual images were presented (Figure 1a). The participants were asked to retrieve the specific picture image given by the auditory cue as long as the red fixation cross (2 s) remained on the screen in the absence of any picture. In each run, there were total 40 trials, and every picture occurred in 2 perception and 2 retrieval trials. The order of the trial types and pictures were pseudorandomized and counterbalanced across runs by optseq2 program (https://surfer.nmr.mgh.harvard.edu/optseq/).

2.2.4. Postscan tests

Immediately following the scanning, the participants rated their affective valence and the subjective vividness in the same way as they performed before the scanning. Then, to objectively evaluate participants' picture memory, two postscan tests were conducted: a sentence test and an image test. In the sentence test, participants heard a cue, followed by presentation of a sentence for 2 s. Within 6 s from the onset of the sentence, they had to indicate whether the presented sentence described the cued picture correctly or not (Figure 5a). About eight sentences (true/false × core/surrounding × two difficulty levels) were used for each picture. The less difficult sentences were descriptions related to gist information of people or objects in the scene, such as their actions or postures, while the more difficult sentences were descriptions about more detailed information, such as the clothing of people wore or the numbers of items. In the results, because we found the same tendency for both level sentences, we combined the outcomes from both level tests to increase power. The order of the cues and sentences was randomized. The image test procedure was identical to the sentence test except that a picture fragment was presented instead of a sentence (Figure 5b). Each picture fragment was randomly selected from a set of circles forming a 8 × 6 grid of a picture. The participants had to determine whether the picture fragment belonged to the cued picture or not. For the “false” trials, a picture fragment of a lure image or a noncued picture was presented while a fragment of a cued picture was presented for the “true” trials.

FIGURE 5.

FIGURE 5

Open in a new tab

Memory performance and the similarity between the prefrontal and visual representations. The participants performed two postscan tests: a sentence test (a, c, and e) and an image test (b, d, and f). (a) In the sentence test, participants had to indicate whether the presented sentence correctly described the cued picture. (b) In the image test, the participants had to determine whether the presented portion of a picture belonged to the cued picture. (c) Correct rates in the sentence test. (d) Correct rates in the image test. The dotted lines indicate the chance level (50%). *p < .05, **p < .01. Error bars indicate between‐subjects SEM. (e) Correlation between the accuracy in the sentence test and the similarity between the representations of the dlPFC and the V1 during perception from fMRI data. (f) Correlation between the accuracy in the image test and the similarity between the representations of the dlPFC and the V1 during perception from fMRI data. Each dot indicates the mean value in each participant. Regression line on the scatter plot indicates significant correlation (p < .05, two‐tailed)

2.3. fMRI acquisition

Participants were scanned on the 3 T Siemens MAGNETOM Prisma located in the Center for Neuroscience Imaging Research at the Institute for Basic Science. Echo‐planar imaging (EPI) data were acquired using a 20‐channel head coil, with an in‐plane resolution of 2.5 × 2.5 mm, and 40 × 2.5 mm slices (0.25 mm interslice gap, repetition time [TR] = 2,000 ms, echo time [TE] = 25.0 ms, matrix size 76 × 76, field of view [FOV] = 192 mm). Whole brain volumes were scanned, and slices were oriented approximately parallel to the base of the temporal lobe. Standard MPRAGE (magnetization‐prepared rapid‐acquisition gradient echo) images were collected after the experimental runs in fMRI session for use as a high‐resolution anatomical data for the following fMRI data analysis.

2.4. Follow‐up experiment

To support the result in the sentence test, we conducted an additional behavioral experiment in a different set of participants. The participants were familiarized with the emotional and neutral pictures with the corresponding auditory cues in the same way as the prescan training of the main experiment. Then, they were asked to recall and report the details of the specific picture given by the auditory cue (free recall). They were instructed first to describe the people in the pictures—what they look like and what they are doing—for 1 min and next to describe all the other things in the background for another 1 min. If a participant said everything that he (or she) could remember in less than 1 min, he (or she) could move on to the next step or next trial by pressing the space key. Their reports about the pictures were recorded, and then scored by two examiners. Each specific description of a picture was awarded 1 point, and got −0.5 point for each wrong description. For example, for the IAPS picture 2396, which is one of the neutral pictures we used, “a man is going down the steps” and “the man is wearing a cap” could each be awarded 1 point. The maximum possible points for a picture was 10. The scores from the two examiners were averaged. After the free recall, the participants conducted the sentence test.

2.5. Regions‐of‐interest

The dorsolateral prefrontal cortex (dlPFC), the primary visual cortex (V1), angular gyrus (AnG), and the superior parietal lobule (SPL) were automatically defined by the parcellation of FreeSurfer: “middle frontal gyrus (G_front_middle)” combined with “middle frontal sulcus (S_front_middle)” for the dlPFC, “V1” for V1, “angular gyrus (G_pariet_inf‐Angular)” for the AnG, and “superior parietal lobule (G_parietal_sup)” for the SPL (Destrieux, Fischl, Dale, & Halgren, 2010). To define the central (cV1) and peripheral V1 (pV1), the early visual cortex (EVC) was first determined by a functional localizer scan. In this scan, participants viewed alternating 16 s blocks of a central disk (5°), and an annulus (6–28°). Peripheral and central EVC were localized with the contrast of the responses induced by the two blocks. The cV1 was defined by the overlapping voxels between the anatomical V1 and the central EVC, and the remaining V1 voxels were used to determine the pV1.

2.6. fMRI data analysis

The fMRI data was analyzed using AFNI (https://afni.nimh.nih.gov/), SUMA (AFNI surface mapper), FreeSurfer, and custom MATLAB scripts. Preprocessing included slice timing correction, motion correction, and smoothing (smoothing was performed only for the localizer data, not the event‐related data, with a 5 mm full width at half maximum [FWHM] Gaussian kernel). Then, the percent signal change was calculated for each run of the main task and each participant on a voxel‐by‐voxel basis.

To deconvolve the event‐related BOLD responses during the tasks, we conducted a standard general linear model using the AFNI software package (GAM function of 3dDeconvolve). For each picture perception or retrieval, the β‐value (in percent signal change) and t‐value of each voxel were derived. For the average magnitude of responses across all voxels and stimuli within each ROI, β‐value was used. To reveal common areas of activation in both emotional and neutral perception (or retrieval) trials, voxels with positive mean β‐value in both emotional and neutral perception (or retrieval) trials were set to one and the others to zero for each participant. The heat map was then developed from the sum of all participant data in a standard space (Figure 2).

FIGURE 2.

FIGURE 2

Open in a new tab

The average magnitude of response. (a) The common areas of activation in both emotional and neutral perception trials. The colored areas indicate regions at which more than half of the participants commonly showed positive percent signal change for both emotional and neutral perception. (b) The averaged magnitude of response across voxels in the dlPFC and the V1 ROI during perception. (c) The common areas of activation in both emotional and neutral retrieval trials. (d) The averaged magnitude of response across voxels in the dlPFC and the V1 during retrieval. *p < .05, **p < .01. n.s., not significant. Error bars indicate between‐subjects SEM

For the representational similarity analysis (Haxby et al., 2001; Kravitz, Kriegeskorte, & Baker, 2010; Kriegeskorte, Goebel, & Bandettini, 2006; Lee, Kravitz, & Baker, 2012, 2019), we first divided four event‐related runs for each participant into two halves in all possible three ways: run 1 and 2 versus run 3 and 4, run 1 and 3 versus run 2 and 4, run 1 and 4 versus run 2 and 3. For each of the splits, we estimated the t‐value between each event and baseline in each half of the data. The t‐values were then extracted from the voxels within each ROI, and cross correlated. Before calculating the correlations, the t‐values were normalized separately in each voxel for the encoding and retrieval conditions by subtracting the mean value across all picture conditions (Haxby et al., 2001; Lee et al., 2012). We here used t‐values rather than β‐values as they tend to be slightly more stable (Misaki, Kim, Bandettini, & Kriegeskorte, 2010), though we found nearly identical results from the analysis with β‐values. The correlation coefficients were Fisher's z transformed for further analyses.

To compare the representations in the dlPFC and V1, we derived representational dissimilarity matrices (RDMs), based on the Fisher's z transformed correlation coefficients for each pair of pictures (Kriegeskorte et al., 2008). Subsequently, we computed the correlations between the RDMs of the dlPFC and V1 as a measure of the representational relationship between the regions (Kriegeskorte et al., 2008). In the searchlight analysis (Kriegeskorte et al., 2006), the RDMs were derived based on the patterns within individual spheres (radius 8.75 mm, corresponding to ~123 voxels), and then the correlations between the RDMs of the V1 (seed) and each sphere across the whole brain was calculated. To compute significant clusters across subjects, we additionally conducted the identical analysis using permuted labels (100 per participant), obtained a chance map by averaging the permuted label maps, and then thresholded the searchlight map by comparison with the chance map at p < .01 (Guntupalli, Wheeler, & Gobbini, 2017).

A discrimination index for each picture was calculated by subtracting the average of between‐condition correlations (correlation coefficient comparing each picture with every other picture) from the within‐condition correlations (correlation coefficient comparing each picture with other presentations of the same picture), based on the multivoxel patterns within ROIs.

2.7. Statistical analyses

When we compared behavior, activation, the representational relationship, or decoding results between emotional and neutral conditions, we mainly used paired t‐test (two‐tailed). To verify whether there was significant correspondence between the RDMs of the dlPFC and V1, a permutation test was performed. For this, we randomly shuffled the picture conditions and derived the correlations between the RDMs. We repeated this step 10,000 times, and tested whether the actual correlation falls within the top 5% of the simulated null distribution of correlations (the smallest possible p‐value was .0001) (Kriegeskorte et al., 2008). Activation or decoding results were compared with basal level (zero) by one‐sample t‐test (two‐tailed for activation; one‐tailed for decoding). We also used two‐way ANOVA (tests of within‐subjects effects) to examine the effect of picture type, ROI, and the interaction between them with Greenhouse–Geisser corrections. For correlation analyses, Pearson's correlation was used. We report Cohen's d to show the effect size of t‐test in comparing representational similarity results or behavioral results between emotional and neutral conditions.

3. RESULTS

We monitored cortical activity while the participants viewed or retrieved negative emotional pictures and neutral pictures inside a scanner (Figure 1a). Both the emotional and the neutral pictures contain people in certain events (Figure 1a). Prior to scanning, participants were familiarized with the emotional and neutral pictures with the corresponding auditory cues. Before and after the scans, the participants rated their affective valence and the subjective vividness of each picture retrieved by the given auditory cue (Figure 1b). As expected, the participants expressed more negative feelings for cues associated with emotional pictures compared with the cues paired with neutral pictures in both prescan and postscan test sessions (prescan: t (24) = 16.993, p < .001; postscan: t (24) = 15.092, p < .001) (Figure 1b). However, they reported a comparable level of memory vividness between emotional pictures and neutral pictures for both prescan and postscan test sessions (prescan: t (24) = −0.233, p = .818; postscan: t (24) = −1.744, p = .094) (Figure 1b).

3.1. Correspondence of information processing between the visual and prefrontal cortex

We first examined the magnitude of responses in perception and retrieval for both emotional and neutral pictures (Figure 2). Univariate analysis revealed that the same network of regions including the prefrontal, temporal, parietal, and occipital areas, was engaged in both emotional perception and neutral perception (Figure 2a, Figure S1). To examine the difference between emotional perception and neutral perception in more detail, we focused on the individually defined ROIs, the dlPFC and V1, given that emotion can modulate visual processing even at the early stage (Phelps, Ling, & Carrasco, 2006; Pourtois, Schettino, & Vuilleumier, 2013; Vuilleumier, 2005; Vuilleumier & Driver, 2007). We found that the average magnitude of the responses were comparable between emotional perception and neutral perception in both dlPFC (t (24) = −0.143, p = .887) and V1 (t (24) = −1.678, p = .106) (Figure 2b). A two‐way ANOVA on the average response with Picture Type (emotional pictures vs. neutral pictures) and ROI (dlPFC, V1) as within‐subject factors revealed only a main effect of ROI (F (1,24) = 39.899, p < .001) and no main effect or interaction involving Picture Type (Picture Type: F (1,24) = 1.069, p = .304; Picture Type * ROI: F (1,24) = 0.891, p = .348) (Figure 2b). The same tendency was also observed during retrieval (Figure 2c,d, Figure S1). The involvement of dlPFC (emotional: t (24) = 3.419, p = .002; neutral: t (24) = 4.166, p < .001) and V1 (emotional: t (24) = 2.892, p = .008; neutral: t (24) = 3.942, p < .001), and the equivalence of the levels of activation in dlPFC between emotional and neutral picture conditions (t (24) = −1.271, p = .216) were also found during retrieval although the activation level in V1 was stronger during retrieval of neutral pictures compared with emotional pictures (t (24) = −2.161, p = .041) (Figure 2d). These results suggest that the prefrontal and visual cortical areas are commonly engaged in both emotional and neutral perception as well as emotional and neutral retrieval.

Then, how is the information processing of the prefrontal cortex related to that of the visual cortex in emotional and neutral conditions? To address this, we directly derived the representational relationship between the dlPFC and V1 based on RSA (Kriegeskorte et al., 2008; Figure 3a, “Materials and Methods” section).

FIGURE 3.

FIGURE 3

Open in a new tab

Comparison between visual representations with higher order cortical regions. (a) Example RDMs in the dlPFC and V1 from one participant. The correlations between the RDMs of the dlPFC and V1 were derived as a measure of the representational relationship between the ROIs (red outlined for negative emotional pictures, blue outlined for neutral pictures). (b) The similarity between the RDMs of the V1 and the dlPFC during the perception of emotional or neutral pictures. (c) The similarity between the RDMs of the dlPFC and cV1 (or pV1) during the perception of emotional or neutral pictures. (d) The similarity between the RDMs of the V1 and the dlPFC during the retrieval of emotional or neutral pictures. (e) The similarity between the RDMs of the V1 and the angular gyrus (AnG) or the superior parietal lobule (SPL) during the perception of emotional or neutral pictures. *p < .05, **p < .01. n.s., not significant. Error bars indicate between‐subjects SEM

If the visual cortex shares the same information processing with the prefrontal cortex through the interactions between the prefrontal and visual cortices, significant correspondence between the representations may be observed. Remarkably, we found that the similarity between the RDMs of the dlPFC and V1 was significantly greater during the perception of emotional pictures as compared with that during the perception of neutral pictures (t (24) = 3.651, p = .001, Cohen's d = 0.730) (Figure 3b). This indicates closer correspondence between the prefrontal and visual representations during perception of emotional pictures than that during perception of neutral pictures. We also tested the relatedness of dissimilarity matrices by randomizing the condition labels of the RDMs and computing the correlations between the RDMs (Kriegeskorte et al., 2008). By repeating this method, we found that there was significant correspondence between the RDMs of dlPFC and V1 during the perception of emotional pictures (permutation test, p < .0001) but not during the perception of neutral pictures (permutation test, p = .4511)(Figure 3b). Further, we examined whether the response patterns of central V1 (cV1) and peripheral V1 (pV1) showed the same tendency. Consistent with the whole V1 result, both cV1 and pV1 showed dissimilarity matrices more similar to that of dlPFC during the perception of emotional pictures compared with when neutral pictures were viewed (Figure 3c). During retrieval, the relationship between the RDMs of the dlPFC and V1 was significant for both emotional and neutral pictures (permutation test, p = .0001 for emotional pictures, p = .0006 for neutral pictures), but there was no significant difference between them (t (24) = 0.405, p = .689; Figure 3d).

Since regions in the parietal cortex, which is known to be involved in memory encoding and top‐down attentional control (Chun, Golomb, & Turk‐Browne, 2011; Hopfinger, Buonocore, & Mangun, 2000; Shomstein, 2012; van der Linden, Berkers, Morris, & Fernández, 2017), were also engaged in perception and retrieval of both the emotional and the neutral pictures (Figure 2a,c, Figure S1), we additionally examined the parietal cortical areas, angular gyrus (AnG) and superior parietal lobule (SPL). However, there was no significant difference between the emotional perception and neutral perception in the AnG (t (24) = −0.110, p = .913) or in the SPL (t (24) = 1.645, p = .113)(Figure 3e). Additionally, we did not find any significant difference between the emotional retrieval and neutral retrieval in these regions (AnG: t (24) = 1.442, p = .162; SPL: t (24) = 0.855, p = .401; Figure S2). Thus, these suggest that the differential representational similarity with V1 during emotional perception is not a nonspecific general reflection of the emotional effect on the neural responses in the cortical regions but is specific to certain regions including the prefrontal cortex.

Additionally, we examined whether the visual cortex still shows distinct representations for different pictures even if prefrontal representations are significantly reflected on the visual representations during perception of emotional pictures. We derived discrimination indices as the difference between within‐picture and between‐picture pattern similarities (Kravitz, Peng, & Baker, 2011; Kravitz et al., 2010; Kriegeskorte et al., 2006; Lee et al., 2012), and found that the V1 showed significantly discriminable patterns of response to individual pictures during both emotional and neutral perception (emotional: t (24) = 2.087, p = .024; neutral: t (24) = 2.356, p = .013; Figure S3).

Given that the amygdala is thought to be one of the key areas in emotional processes (Adolphs, Cahill, Schul, & Babinsky, 1997; Canli et al., 2000; Kensinger & Schacter, 2006; LaBar & Cabeza, 2006; Murty et al., 2010), we also assessed the correspondence between the amygdala and prefrontal representations, and between the amygdala and visual representations during the perception of emotional pictures. We found that the RDMs of the amygdala showed greater similarity with the RDMs of the dlPFC or with the RDMs of the V1 during perception of emotional pictures as compared with that during the perception of neutral pictures (dlPFC: t (24) = 2.592, p = .016; V1: t (24) = 2.253, p = .034; Figure S4). These suggest the possibility that the amygdala is involved in the stronger internal processing between the dlPFC and V1 when emotional information is received compared with neutral condition.

To further investigate other regions showing significant representational relationship with V1, we used a whole‐brain searchlight procedure (Kriegeskorte et al., 2006; Figure 4). These searchlights revealed a significant relationship in prefrontal areas during emotional perception but not during neutral perception (thresholded at p < .01 compared with baseline derived from random permutation) (Figure 4a), which is consistent with the results from the ROI analysis (Figure 3). The areas of the parietal cortex showed a significant relationship with V1 during both emotional and neutral perception (Figure 4a). The same tendency was observed during retrieval (Figure 4b).

FIGURE 4.

FIGURE 4

Open in a new tab

Searchlight results for significant representational relationship with the V1 during perception (a) or retrieval (b). In the analysis, the correlations between the RDMs of the V1 (seed) and each sphere across the whole brain was calculated. The colored areas indicate significant clusters above chance (p < .01, corrected) in emotional (red boxed) or neutral (blue boxed) condition

Taken together, these results show that the representation of V1 corresponded much more closely with that in dlPFC during the perception of emotional pictures than during the perception of neutral pictures. These suggest differential information processing across the visual and prefrontal cortical areas in emotional and neutral experiences; visual representations during emotional experiences reflect more higher‐order information especially from the prefrontal cortex than that during neutral experiences.

3.2. Relationship between dlPFC‐V1 representational correspondence and subsequent memory

Next we investigated whether the similarity between the prefrontal and visual representations during emotional picture perception is related to memory enhancement or impairment. To do this, we designed two postscan tests: a sentence test and an image test. In the sentence test, the participants had to indicate whether or not the presented sentence described the cued picture correctly (Figure 5a). In the image test, participants had to determine whether or not the test image (a portion of a picture) belonged to the cued picture (Figure 5b). Thus, while both tests required recall of information pertaining to the cued picture, the image test highlights low‐level visual information whereas the sentence test emphasizes high‐level semantic information that is expressed as written language.

First, we compared the participants' performance on both tests for emotional pictures versus neutral pictures. Given that prior studies have shown that memory for the core element of an emotional event can be processed differently from the surrounding element (Burke, Heuer, & Reisberg, 1992; Kensinger et al., 2007; Waring & Kensinger, 2011), the accuracy levels of the questions corresponding to the core element (or main event) and the surrounding element (or minor event) of the picture were evaluated separately. On the sentence test, the average accuracy was significantly higher for both the core and surrounding elements of emotional pictures than it was for neutral pictures (core: t (24) = 6.871, p < .001, Cohen's d = 1.374; surrounding: t (24) = 5.008, p < .001, Cohen's d = 1.002)(Figure 5c). In contrast, the participants showed comparable performance for both the emotional and neutral pictures in the image test (core: t (24) = −0.886, p = .560, Cohen's d = 0.118; surrounding: t (24) = −1.955, p = .062, Cohen's d = 0.391), (Figure 5d). A two‐way ANOVA on memory accuracy with Picture Type (emotional pictures vs. neutral pictures) and Test (sentence test vs. image test) as within‐subject factors revealed a significant interaction between Picture Type and Test (core: F (1,24) = 18.141, p < .001; surrounding: F (1,24) = 14.168, p < .001). Thus, these results show that while the high‐level semantic components of emotional picture memory are enhanced compared with those of the neutral picture memory, there was no effect or even opposite effect of emotion on the memory performance for the low‐level visual components.

To investigate whether the similarity between the prefrontal and visual representations is related to the performance on the sentence or image test, we directly derived the correlations between the similarity and the performance levels for each test. Because the tendency was similar between the performance for the core part and the surrounding part in each test, we collapsed the accuracy data across the core and surrounding parts for this analysis. We found that the similarity between the RDMs of the dlPFC and the V1 during perception was significantly correlated with the accuracy of the sentence test for emotional pictures (r = .437, p = .029), but not for neutral pictures (r = −.168, p = .423)(Figure 5e). Moreover, no significant correlation was found on the image test for both emotional and neutral pictures (Figure 5f). This suggests that the correspondence between the prefrontal and visual representations during emotional experience reflects the strength of the high‐level semantic components but not the low‐level visual components of memory. These results may reflect that through the more active integration of the PFC information processing in the sensory representations during emotional information encoding, higher‐level aspects rather than lower‐level aspects of input information are mainly affected and enhanced.

Additionally, the memory accuracy in the image test may depend on how similarly the visual features of a picture during perception are reinstated during retrieval. Consistent with this expectation, we found that the neural pattern similarity between perception and retrieval in the V1 was significantly correlated with the level of memory accuracy for the emotional pictures in the image test (r = .515, p = .008 for emotional pictures; r = .346, p = .090 for neutral pictures), but not in sentence test (r = −.010, p = .961 for emotional pictures; r = .054, p = .796 for neutral pictures).

Because memorized higher‐order information might not be fully detected in the performance of the sentence test, we additionally performed a separate behavioral study with a separate group of participants. In this study, the participants were asked to conduct a free recall test in addition to the sentence test of the main experiment. In the free recall test, the participants were instructed to verbally describe the pictures on which they were familiarized before the test. Consistent with the result from the sentence test in the main experiment, these participants showed better performance for emotional pictures than for neutral pictures on both the free recall test (t [20] = 4.294, p < .001) and the sentence test (core: t (20) = 3.1369, p = .005; surrounding: t (20) = 2.4667, p = .023; Figure S5). Thus, this result supports that the high‐level semantic components of the emotional picture memory, which is expressed as language, are enhanced compared with those of the neutral picture memory.

4. DISCUSSION

Our findings highlight a neural basis for specific information processing during emotional experiences. We provide evidence supporting the contention that visual representation during the perception of emotional stimuli reflects more higher‐order information from the prefrontal cortex than that during neutral perception. There was significantly closer correspondence between prefrontal and visual representations during emotional perception than that during neutral perception. Moreover, we found a dissociable emotional effect on memory depending on the level of information processing, and the similarity between the prefrontal and visual representations was positively correlated with the strength of the high‐level semantic components of emotional memory as opposed to low‐level visual components. Taken together, these results suggest that higher‐order information processed in the prefrontal cortex is more strongly integrated into sensory representations during the experience of emotional stimuli, compared with neutral stimuli.

Visual perception reflects the integration of externally driven bottom‐up and internally generated top‐down signals (Albright, 2012; Hsieh, Vul, & Kanwisher, 2010; Kastner, De Weerd, Desimone, & Ungerleider, 1998). Our results directly demonstrate a much stronger integration of bottom‐up and top‐down information during emotional experiences than during neutral experiences. The top‐down information may include internal representations of prior knowledge or expectations. In particular the prefrontal cortex, which is considered to be the main source of top‐down signals in our data, is involved in the process and maintenance of semantic information (Binder & Desai, 2011; Lee, Kravitz, & Baker, 2013; Murray & Ranganath, 2007; Vandenberghe, Price, Wise, Josephs, & Frackowiak, 1996). Thus, through this integration, more semantic information such as interpretation based on prior knowledge may be incorporated into visual representation during the experience of emotional stimuli, resulting in high‐level semantic aspects undergoing greater strengthening. This possibility is supported by the behavioral memory performance results here (Figure 5). We found better performance for emotional pictures than for neutral pictures in the sentence test, while such a tendency was not observed in the image test.

Prior studies of emotional memory suggest that the emotional context during encoding involves both the bottom‐up emotion system, including the amygdala (Kensinger, 2009; McGaugh, 2004), and the top‐down system, including the PFC, which is thought to be involved in a top‐down regulation during emotion processing (Dolcos & Denkova, 2014; Kensinger, 2004; Ritchey et al., 2011). However, how the details of the memory content are modulated has remained unclear. Our results provide the possibility that sensory representation during emotional experiences is modulated by internally generated information processed in the prefrontal regions (Figures 3, 4, 5). Additionally, based on our results showing greater similarity between the amygdala and PFC representations, and between the amygdala and V1 representations (Figure S4), the amygdala also appears to contribute to the stronger internal processing between the dlPFC and V1 during emotional experiences. It will be interesting for future studies to investigate what information is processed between the amygdala, PFC, and V1.

We observed a comparable level of the subjective vividness and the average response magnitude in dlPFC and V1 between emotional and neutral perception, while other studies showed enhanced subjective vividness of emotional memory (Phelps & Sharot, 2008; Rimmele, Davachi, Petrov, Dougal, & Phelps, 2011; Sharot et al., 2004) and higher activity in the prefrontal or visual areas (Sabatinelli et al., 2011; Taylor, Liberzon, & Koeppe, 2000). The relatively short delay between perception and the vividness test could have contributed to the comparable level of vividness between emotional and neutral pictures. Another possibility is that our pre‐scan familiarization and the repetitive exposure to both emotional and neutral stimuli in our experimental protocol may be a cause of the discrepancies in subjective vividness and cortical activity. It may be possible that the subjective vividness and overall cortical activity are mainly influenced by emotional arousal, but in our experimental condition emotional arousal is relatively less strong due to the repetitive exposure to the stimuli (Balderston, Schultz, & Helmstetter, 2011; Vuilleumier, 2005). In future work, it will be interesting to reveal the detailed effects of emotional arousal on the overall activation level in the PFC and V1, and on the information processing between the PFC and V1. Also, since we focused on the neural responses to negative emotional pictures compared with neutral pictures in this study, future research may need to reveal whether neural responses to positive emotional pictures show the same effect.

We found that the similarity between prefrontal and visual representations correlated positively with the accuracy in the semantic test for emotional pictures, but not for neutral pictures (Figure 5e). This result suggests that semantic aspects of emotional stimuli are more strongly encoded by the direct integration of high‐level information into the visual representations. For the correspondence between the prefrontal and visual representations, one concern might be whether the correspondence could be accounted by the possibility that emotional pictures contain more semantic contents. To minimize this possibility and keep the consistency of the contents, we only used the pictures that contain people in certain situations for both emotional and neutral conditions. However, the influence of different amount of semantic contents will need to be further investigated in future work.

While we found a much closer relationship between prefrontal and visual representations during the perception of emotional pictures than during the perception of neutral pictures, this distinction between emotional and neutral perception was not observed during retrieval. Prior research suggests that the retrieval process itself can induce greater similarity between cortical representations compared with perception, as only internally generated top‐down signals are processed without interaction with externally driven bottom‐up signals (Lee et al., 2012). Consistent with this idea, our result showed significant correspondence between prefrontal and visual representations in both emotional and neutral retrieval conditions (Figure 3d). Thus, there is the possibility that the fundamental retrieval property of great correspondence between cortical representations may occlude the differential effect of emotional process on the dlPFC‐V1 representational relationship during retrieval. It will be interesting to investigate whether there are other cortical regions that have differential representational relationships between emotional and neutral retrieval in future work.

In conclusion, our results suggest that the experience of emotional stimuli involves a specific process that directly link information processing in the visual cortex with that in the prefrontal cortex, highlighting the high‐level semantic components of emotional memory and generating more brain interpretations.

CONFLICT OF INTERESTS

The authors declare no conflicts of interest.

Supporting information

Appendix S1. Supporting Information.

Click here for additional data file. (1.5MB, docx)

ACKNOWLEDGMENTS

Thanks to C.I. Baker and H. F. Kim for comments, and members of the Memory and Cognition Laboratory for discussion. This work was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI) funded by the Ministry of Health & Welfare (HI15C3175), the Basic Science Research Program (NRF‐2020R1A2C2007770) and the Brain Research Program (NRF‐2017M3C7A1031333) through the National Research Foundation (NRF), and the Technology Innovation Program (20012464) funded by the Ministry of Trade, Industry & Energy (MOTIE) of Korea.

Park D, Kim T, Lee S‐H. Strong correspondence between prefrontal and visual representations during emotional perception. Hum Brain Mapp. 2021;42:2115–2127. 10.1002/hbm.25353

Funding information Korea Health Industry Development Institute, Grant/Award Number: HI15C3175; Ministry of Trade, Industry and Energy, Grant/Award Number: 20012464; National Research Foundation, Grant/Award Numbers: NRF‐2017M3C7A1031333, NRF‐2020R1A2C2007770

DATA AVAILABILITY STATEMENT

The behavioral and neuroimaging data are available from the corresponding author on reasonable request. Sharing and reuse of data require the expressed written permission of the authors, as well as clearance from the Institutional Review Boards.

REFERENCES

  1. Adolphs, R. , Cahill, L. , Schul, R. , & Babinsky, R. (1997). Impaired declarative memory for emotional material following bilateral amygdala damage in humans. Learning & Memory, 4(3), 291–300. 10.1101/lm.4.3.291 [DOI] [PubMed] [Google Scholar]
  2. Albright, T. D. (2012). On the perception of probable things: Neural substrates of associative memory, imagery, and perception. Neuron, 74(2), 227–245. 10.1016/j.neuron.2012.04.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Balderston, N. L. , Schultz, D. H. , & Helmstetter, F. J. (2011). The human amygdala plays a stimulus specific role in the detection of novelty. NeuroImage, 55(4), 1889–1898. 10.1016/j.neuroimage.2011.01.034 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Binder, J. R. , & Desai, R. H. (2011). The neurobiology of semantic memory. Trends in Cognitive Sciences, 15(11), 527–536. 10.1016/j.tics.2011.10.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bookbinder, S. H. , & Brainerd, C. J. (2017). Emotionally negative pictures enhance gist memory. Emotion, 17(1), 102–119. 10.1037/emo0000171 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bowen, H. J. , Kark, S. M. , & Kensinger, E. A. (2018). NEVER forget: Negative emotional valence enhances recapitulation. Psychonomic Bulletin and Review, 25(3), 870–891. 10.3758/s13423-017-1313-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Bradley, M. M. , Greenwald, M. K. , Petry, M. C. , & Lang, P. J. (1992). Remembering pictures: Pleasure and arousal in memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(2), 379–390. 10.1037/0278-7393.18.2.379 [DOI] [PubMed] [Google Scholar]
  8. Brainerd, C. J. , Stein, L. M. , Silveira, R. A. , Rohenkohl, G. , & Reyna, V. F. (2008). How does negative emotion cause false memories? Psychological Science, 19(9), 919–925. 10.1111/j.1467-9280.2008.02177.x [DOI] [PubMed] [Google Scholar]
  9. Brown, R. , & Kulik, J. (1977). Flashbulb memories. Cognition, 5(1), 73–99. 10.1016/0010-0277(77)90018-X [DOI] [Google Scholar]
  10. Burke, A. , Heuer, F. , & Reisberg, D. (1992). Remembering emotional events. Memory & Cognition, 20(3), 277–290. 10.3758/BF03199665 [DOI] [PubMed] [Google Scholar]
  11. Canli, T. , Zhao, Z. , Brewer, J. , Gabrieli, J. D. , & Cahill, L. (2000). Event‐related activation in the human amygdala associates with later memory for individual emotional experience. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 20(19), RC99–RC99. 10.1523/JNEUROSCI.20-19-j0004.2000 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Canli, T. , Desmond, J. E. , Zhao, Z. , & Gabrieli, J. D. E. (2002). Sex differences in the neural basis of emotional memories. Proceedings of the National Academy of Sciences of the United States of America, 99(16), 10789–10794. 10.1073/pnas.162356599 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Chun, M. M. , Golomb, J. D. , & Turk‐Browne, N. B. (2011). A taxonomy of external and internal attention. Annual Review of Psychology, 62, 73–101. 10.1146/annurev.psych.093008.100427 [DOI] [PubMed] [Google Scholar]
  14. Curtis, C. E. , & D'Esposito, M. (2003). Persistent activity in the prefrontal cortex during working memory. Trends in Cognitive Sciences, 7(9), 415–423. 10.1016/S1364-6613(03)00197-9 [DOI] [PubMed] [Google Scholar]
  15. Dan‐Glauser, E. S. , & Scherer, K. R. (2011). The Geneva affective picture database (GAPED): A new 730‐picture database focusing on valence and normative significance. Behavior Research Methods, 43(2), 468–477. 10.3758/s13428-011-0064-1 [DOI] [PubMed] [Google Scholar]
  16. Destrieux, C. , Fischl, B. , Dale, A. , & Halgren, E. (2010). Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature. NeuroImage, 53(1), 1–15. 10.1016/j.neuroimage.2010.06.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Dolcos, F. , & Denkova, E. (2014). Current emotion research in cognitive neuroscience: Linking enhancing and impairing effects of emotion on cognition. Emotion Review, 6(4), 362–375. 10.1177/1754073914536449 [DOI] [Google Scholar]
  18. Dolcos, F. , Iordan, A. D. , & Dolcos, S. (2011). Neural correlates of emotion‐cognition interactions: A review of evidence from brain imaging investigations. Journal of Cognitive Psychology, 23(6), 669–694. 10.1080/20445911.2011.594433 [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Dolcos, F. , Labar, K. S. , & Cabeza, R. (2004a). Dissociable effects of arousal and valence on prefrontal activity indexing emotional evaluation and subsequent memory: An event‐related fMRI study. NeuroImage, 23(1), 64–74. 10.1016/j.neuroimage.2004.05.015 [DOI] [PubMed] [Google Scholar]
  20. Dolcos, F. , Labar, K. S. , & Cabeza, R. (2004b). Interaction between the amygdala and the medial temporal lobe memory system predicts better memory for emotional events. Neuron, 42(5), 855–863. 10.1016/S0896-6273(04)00289-2 [DOI] [PubMed] [Google Scholar]
  21. Funahashi, S. (2001). Neuronal mechanisms of executive control by the prefrontal cortex. Neuroscience Research, 39(2), 147–165. 10.1016/S0168-0102(00)00224-8 [DOI] [PubMed] [Google Scholar]
  22. Gilbert, C. D. , & Li, W. (2013). Top‐down influences on visual processing. Nature Reviews Neuroscience, 14(5), 350–363. 10.1038/nrn3476 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Gilbert, C. D. , & Sigman, M. (2007). Brain states: Top‐down influences in sensory processing. Neuron, 54(5), 677–696. 10.1016/j.neuron.2007.05.019 [DOI] [PubMed] [Google Scholar]
  24. Guntupalli, J. S. , Wheeler, K. G. , & Gobbini, M. I. (2017). Disentangling the representation of identity from head view along the human face processing pathway. Cerebral Cortex (New York, N.Y.: 1991), 27(1), 46–53. 10.1093/cercor/bhw344 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Haxby, J. V. , Gobbini, M. I. , Furey, M. L. , Ishai, A. , Schouten, J. L. , & Pietrini, P. (2001). Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293(5539), 2425–2430. 10.1126/science.1063736 [DOI] [PubMed] [Google Scholar]
  26. Hayes, J. P. , Morey, R. A. , Petty, C. M. , Seth, S. , Smoski, M. J. , McCarthy, G. , & LaBar, K. S. (2010). Staying cool when things get hot: Emotion regulation modulates neural mechanisms of memory encoding. Frontiers in Human Neuroscience, 4, 230. 10.3389/fnhum.2010.00230 [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hopfinger, J. B. , Buonocore, M. H. , & Mangun, G. R. (2000). The neural mechanisms of top‐down attentional control. Nature Neuroscience, 3(3), 284–291. 10.1038/72999 [DOI] [PubMed] [Google Scholar]
  28. Hsieh, P.‐J. , Vul, E. , & Kanwisher, N. (2010). Recognition alters the spatial pattern of fMRI activation in early Retinotopic cortex. Journal of Neurophysiology, 103(3), 1501–1507. 10.1152/jn.00812.2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Johansson, M. , Mecklinger, A. , & Treese, A.‐C. (2004). Recognition memory for emotional and neutral faces: An event‐related potential study. Journal of Cognitive Neuroscience, 16(10), 1840–1853. 10.1162/0898929042947883 [DOI] [PubMed] [Google Scholar]
  30. Kaplan, R. L. , Van Damme, I. , Levine, L. J. , & Loftus, E. F. (2016). Emotion and false memory. Emotion Review, 8(1), 8–13. 10.1177/1754073915601228 [DOI] [Google Scholar]
  31. Kastner, S. , De Weerd, P. , Desimone, R. , & Ungerleider, L. G. (1998). Mechanisms of directed attention in the human extrastriate cortex as revealed by functional MRI. Science, 282(5386), 108–111. 10.1126/science.282.5386.108 [DOI] [PubMed] [Google Scholar]
  32. Kensinger, E. A. (2004). Remembering emotional experiences: The contribution of valence and arousal. Reviews in the Neurosciences, 15(4), 241–252. 10.1515/REVNEURO.2004.15.4.241 [DOI] [PubMed] [Google Scholar]
  33. Kensinger, E. A. (2009). Remembering the details: Effects of emotion. Emotion Review, 1(2), 99–113. 10.1177/1754073908100432 [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Kensinger, E. A. , & Corkin, S. (2003). Memory enhancement for emotional words: Are emotional words more vividly remembered than neutral words? Memory & Cognition, 31(8), 1169–1180. 10.3758/BF03195800 [DOI] [PubMed] [Google Scholar]
  35. Kensinger, E. A. , & Corkin, S. (2004). Two routes to emotional memory: Distinct neural processes for valence and arousal. Proceedings of the National Academy of Sciences of the United States of America, 101(9), 3310–3315. 10.1073/pnas.0306408101 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Kensinger, E. A. , Garoff‐Eaton, R. J. , & Schacter, D. L. (2007). Effects of emotion on memory specificity: Memory trade‐offs elicited by negative visually arousing stimuli. Journal of Memory and Language, 56(4), 575–591. 10.1016/j.jml.2006.05.004 [DOI] [Google Scholar]
  37. Kensinger, E. A. , & Schacter, D. L. (2006). Amygdala activity is associated with the successful encoding of item, but not source, information for positive and negative stimuli. Journal of Neuroscience, 26(9), 2564–2570. 10.1523/JNEUROSCI.5241-05.2006 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Kravitz, D. J. , Peng, C. S. , & Baker, C. I. (2011). Real‐world scene representations in high‐level visual cortex: It's the spaces more than the places. Journal of Neuroscience, 31(20), 7322–7333. 10.1523/JNEUROSCI.4588-10.2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Kravitz, D. J. , Kriegeskorte, N. , & Baker, C. I. (2010). High‐level visual object representations are constrained by position. Cerebral Cortex, 20(12), 2916–2925. 10.1093/cercor/bhq042 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Kriegeskorte, N. , Goebel, R. , & Bandettini, P. A. (2006). Information‐based functional brain mapping. Proceedings of the National Academy of Sciences, 103(10), 3863–3868. 10.1073/pnas.0600244103 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Kriegeskorte, N. , Mur, M. , & Bandettini, P. (2008). Representational similarity analysis ‐ connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 4. 10.3389/neuro.06.004.2008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. LaBar, K. S. , & Cabeza, R. (2006). Cognitive neuroscience of emotional memory. Nature Reviews Neuroscience, 7(1), 54–64. 10.1038/Nrn1825 [DOI] [PubMed] [Google Scholar]
  43. Lang, P. J. , Bradley, M. M. , & Cuthbert, B. N. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report A‐8. https://csea.phhp.ufl.edu/media.html
  44. LeDoux, J. E. , & Pine, D. S. (2016). Using neuroscience to help understand fear and anxiety: A two‐system framework. American Journal of Psychiatry, 173(11), 1083–1093. 10.1176/appi.ajp.2016.16030353 [DOI] [PubMed] [Google Scholar]
  45. Lee, S. H. , Kravitz, D. J. , & Baker, C. I. (2012). Disentangling visual imagery and perception of real‐world objects. NeuroImage, 59(4), 4064–4073. 10.1016/j.neuroimage.2011.10.055 [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Lee, S. H. , Kravitz, D. J. , & Baker, C. I. (2013). Goal‐dependent dissociation of visual and prefrontal cortices during working memory. Nature Neuroscience, 16(8), 997–999. 10.1038/nn.3452 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Lee, S. H. , Kravitz, D. J. , & Baker, C. I. (2019). Differential representations of perceived and retrieved visual information in hippocampus and cortex. Cerebral Cortex, 29(10), 4452–4461. 10.1093/cercor/bhy325 [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Mather, M. (2007). Emotional arousal and memory binding: An object‐based framework. Perspectives on Psychological Science, 2(1), 33–52. 10.1111/j.1745-6916.2007.00028.x [DOI] [PubMed] [Google Scholar]
  49. Mather, M. , & Sutherland, M. R. (2011). Arousal‐biased competition in perception and memory. Perspectives on Psychological Science, 6(2), 114–133. 10.1177/1745691611400234 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. McGaugh, J. L. (2004). The amygdala modulates the consolidation of memories of emotionally arousing experiences. Annual Review of Neuroscience, 27(1), 1–28. 10.1146/annurev.neuro.27.070203.144157 [DOI] [PubMed] [Google Scholar]
  51. Miller, B. T. , & D'Esposito, M. (2005). Searching for “the top” in top‐down control. Neuron, 48(4), 535–538. 10.1016/j.neuron.2005.11.002 [DOI] [PubMed] [Google Scholar]
  52. Misaki, M. , Kim, Y. , Bandettini, P. A. , & Kriegeskorte, N. (2010). Comparison of multivariate classifiers and response normalizations for pattern‐information fMRI. NeuroImage, 53(1), 103–118. 10.1016/j.neuroimage.2010.05.051 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Murray, L. J. , & Ranganath, C. (2007). The dorsolateral prefrontal cortex contributes to successful relational memory encoding. Journal of Neuroscience, 27(20), 5515–5522. 10.1523/JNEUROSCI.0406-07.2007 [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Murty, V. P. , Ritchey, M. , Adcock, R. A. , & LaBar, K. S. (2010). Reprint of: fMRI studies of successful emotional memory encoding: A quantitative meta‐analysis. Neuropsychologia, 49(12), 3459–3469. 10.1016/j.neuropsychologia.2011.02.031 [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Nairne, J. S. , Thompson, S. R. , & Pandeirada, J. N. S. (2007). Adaptive memory: Survival processing enhances retention. Journal of Experimental Psychology: Learning Memory and Cognition, 33(2), 263–273. 10.1037/0278-7393.33.2.263 [DOI] [PubMed] [Google Scholar]
  56. Neisser, U. , & Harsch, N. (1992). Phantom flashbulbs: False recollections of hearing the news about Challenger. In Winograd E. & Neisser U. (Eds.). Affect and accuracy in recall: Studies of "flashbulb" memories, (pp. 9–31). New York, NY: Cambridge University Press. 10.1017/CBO9780511664069.003. [DOI] [Google Scholar]
  57. Ochsner, K. N. (2000). Are affective events richly recollected or simply familiar? The experience and process of recognizing feelings past. Journal of Experimental Psychology: General, 129(2), 242–261. 10.1037//0096-3445.129.2.242 [DOI] [PubMed] [Google Scholar]
  58. Phelps, E. A. , Ling, S. , & Carrasco, M. (2006). Emotion facilitates perception and potentiates the perceptual benefits of attention. Psychological Science, 17(4), 292–299. 10.1111/j.1467-9280.2006.01701.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Phelps, E. A. , & Sharot, T. (2008). How (and why) emotion enhances the subjective sense of recollection. Current Directions in Psychological Science, 17(2), 147–152. 10.1111/j.1467-8721.2008.00565.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Pourtois, G. , Schettino, A. , & Vuilleumier, P. (2013). Brain mechanisms for emotional influences on perception and attention: What is magic and what is not. Biological Psychology, 92(3), 492–512. 10.1016/j.biopsycho.2012.02.007 [DOI] [PubMed] [Google Scholar]
  61. Rimmele, U. , Davachi, L. , Petrov, R. , Dougal, S. , & Phelps, E. A. (2011). Emotion enhances the subjective feeling of remembering, despite lower accuracy for contextual details. Emotion, 11(3), 553–562. 10.1037/a0024246 [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Ritchey, M. , Bessette‐Symons, B. , Hayes, S. M. , & Cabeza, R. (2011). Emotion processing in the aging brain is modulated by semantic elaboration. Neuropsychologia, 49(4), 640–650. 10.1016/j.neuropsychologia.2010.09.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Ritchey, M. , Dolcos, F. , & Cabeza, R. (2008). Role of amygdala connectivity in the persistence of emotional memories over time: An event‐related fMRI investigation. Cerebral Cortex, 18(11), 2494–2504. 10.1093/cercor/bhm262 [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Sabatinelli, D. , Fortune, E. E. , Li, Q. , Siddiqui, A. , Krafft, C. , Oliver, W. T. , … Jeffries, J. (2011). Emotional perception: Meta‐analyses of face and natural scene processing. NeuroImage, 54(3), 2524–2533. 10.1016/j.neuroimage.2010.10.011 [DOI] [PubMed] [Google Scholar]
  65. Sharot, T. , Delgado, M. R. , & Phelps, E. A. (2004). How emotion enhances the feeling of remembering. Nature Neuroscience, 7(12), 1376–1380. 10.1038/nn1353 [DOI] [PubMed] [Google Scholar]
  66. Shomstein, S. (2012). Cognitive functions of the posterior parietal cortex: Top‐down and bottom‐up attentional control. Frontiers in Integrative Neuroscience, 6, 38. 10.3389/fnint.2012.00038 [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Talarico, J. M. , & Rubin, D. C. (2003). Confidence, not consistency, characterizes flashbulb memories. Psychological Science, 14(5), 455–461. 10.1111/1467-9280.02453 [DOI] [PubMed] [Google Scholar]
  68. Taylor, S. F. , Liberzon, I. , & Koeppe, R. A. (2000). The effect of graded aversive stimuli on limbic and visual activation. Neuropsychologia, 38(10), 1415–1425. 10.1016/S0028-3932(00)00032-4 [DOI] [PubMed] [Google Scholar]
  69. van der Linden, M. , Berkers, R. M. W. J. , Morris, R. G. M. , & Fernández, G. (2017). Angular gyrus involvement at encoding and retrieval is associated with durable but less specific memories. Journal of Neuroscience, 37(39), 9474–9485. 10.1523/JNEUROSCI.3603-16.2017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Vandenberghe, R. , Price, C. , Wise, R. , Josephs, O. , & Frackowiak, R. S. J. (1996). Functional anatomy of a common semantic system for words and pictures. Nature, 383(6597), 254–256. 10.1038/383254a0 [DOI] [PubMed] [Google Scholar]
  71. Vuilleumier, P. (2005). How brains beware: Neural mechanisms of emotional attention. Trends in Cognitive Sciences, 9(12), 585–594. 10.1016/j.tics.2005.10.011 [DOI] [PubMed] [Google Scholar]
  72. Vuilleumier, P. , & Driver, J. (2007). Modulation of visual processing by attention and emotion: Windows on causal interactions between human brain regions. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 837–855. 10.1098/rstb.2007.2092 [DOI] [PMC free article] [PubMed] [Google Scholar]
  73. Waring, J. D. , & Kensinger, E. A. (2011). How emotion leads to selective memory: Neuroimaging evidence. Neuropsychologia, 49(7), 1831–1842. 10.1016/j.neuropsychologia.2011.03.007 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix S1. Supporting Information.

Click here for additional data file. (1.5MB, docx)

Data Availability Statement

The behavioral and neuroimaging data are available from the corresponding author on reasonable request. Sharing and reuse of data require the expressed written permission of the authors, as well as clearance from the Institutional Review Boards.

Articles from Human Brain Mapping are provided here courtesy of Wiley

RESOURCES