Level of processing modulates the neural correlates of emotional memory formation

Abstract

Emotion is known to influence multiple aspects of memory formation, including the initial encoding of the memory trace and its consolidation over time. However, the neural mechanisms whereby emotion impacts memory encoding remain largely unexplored. The present study employed a levels-of-processing manipulation to characterize the impact of emotion on encoding with and without the influence of elaborative processes. Participants viewed emotionally negative, neutral, and positive scenes under two conditions: a shallow condition focused on the perceptual features of the scenes and a deep condition that queried their semantic meaning. Recognition memory was tested 2 days later. Results showed that emotional memory enhancements were greatest in the shallow condition. FMRI analyses revealed that the right amygdala predicted subsequent emotional memory in the shallow more than deep condition, whereas the right ventrolateral prefrontal cortex demonstrated the reverse pattern. Furthermore, the association of these regions with the hippocampus was modulated by valence: the amygdala-hippocampal link was strongest for negative stimuli, whereas the prefrontal-hippocampal link was strongest for positive stimuli. Taken together, these results suggest two distinct activation patterns underlying emotional memory formation: an amygdala component that promotes memory during shallow encoding, especially for negative information, and a prefrontal component that provides extra benefits during deep encoding, especially for positive information.

Memories tinged with emotional salience tend to be remembered better and with greater vividness than emotionally neutral memories (Dolcos, Labar, & Cabeza, 2005; Ochsner, 2000). This influence of emotion on memory has been demonstrated to operate on multiple memory processes, including encoding-- the initial formation of a memory trace (Dolcos, LaBar, & Cabeza, 2004a; Kensinger & Corkin, 2004)-- and consolidation-- the strengthening of this memory trace over time (LaBar & Phelps, 1998). Previous studies of emotional memory have tended to emphasize the role of consolidation processes in mediating these enhancements, driven by neurohormonal interactions between the amygdala and medial temporal lobe (MTL) memory system (McGaugh, 2004). Human neuroimaging has likewise demonstrated the importance of the amygdala and MTL memory system in promoting emotional memory formation. These regions tend to predict subsequent memory accuracy for emotional relative to neutral stimuli (Cahill, et al., 1996; Canli, Zhao, Brewer, Gabrieli, & Cahill, 2000; Dolcos, LaBar, & Cabeza, 2004b; Hamann, Ely, Grafton, & Kilts, 1999; Kensinger & Corkin, 2004), and demonstrate enhanced functional coupling during emotional memory encoding (Dolcos, et al., 2004b; Hamann, et al., 1999; Richardson, Strange, & Dolan, 2004). Consistent with a consolidation account, interactions between the amygdala and MTL memory system increasingly predict emotional memory over time (Ritchey, Dolcos, & Cabeza, 2008).

Despite this emphasis on consolidation processes, there is also human behavioral and neuroimaging evidence that emotion influences other processes during memory encoding, including perceptual, attentional, or semantic processes (Kensinger, 2004; LaBar & Cabeza, 2006). In this way, emotion operates on memory not only through a direct pathway from the amygdala, but also via other regions in the brain that participate in memory encoding. However, the contributions of these alternative mechanisms to emotional memory formation remain underspecified. Exploration into alternative emotional memory pathways thus far has been limited, focusing primarily on the influence of emotion on perception (Vuilleumier & Pourtois, 2007) and how this interaction impacts emotional memory encoding (Kensinger, Garoff-Eaton, & Schacter, 2007).

Another candidate mechanism for emotion-driven encoding benefits is elaborative or semantic processing, thought to be mediated in part by the lateral prefrontal cortices (PFC). Elaborative processing is known to promote successful memory encoding: items encoded with a deep, semantic strategy will tend to be remembered better, under most testing conditions, than those encoded with a shallow, perceptual strategy, a hypothesis known as “levels of processing” theory (Craik & Lockhart, 1972). Within the emotional memory literature, it has been suggested that rapid, preattentive processing of emotional stimuli culminates in increased allocation of controlled resources, and that consequent increases in poststimulus elaboration contribute to enhanced memory for emotional stimuli (Christianson, 1992). According to this hypothesis, under limited encoding resources, emotion confers relatively automatic benefits to memory encoding, and when encoding resources are bountiful, these automatic benefits are compounded with the advantages of deep, elaborative encoding, which may extend to both emotional and neutral material. Behavioral studies using a levels-of-processing approach during emotional and neutral word encoding have supported this hypothesis, finding that emotional memory enhancements were greatest when words were encoded under a shallow versus deep condition (Jay, Caldwell-Harris, & King, 2008; Reber, Perrig, Flammer, & Walther, 1994). These findings indicate that shallow processing amplifies the difference between emotional and neutral words, likely due to automatic arousal mechanisms that distinguish emotional from neutral stimuli in this condition, as well as relative benefits in elaborative processing for neutral stimuli in the deep condition. These findings echo behavioral evidence that dividing attention during encoding impacts memory for arousing stimuli less than for neutral stimuli (Kensinger & Corkin, 2004; Kern, Libkuman, & Otani, 2005).

Although this evidence is intriguing, the link between deep elaborative processing during encoding and emotional memory effects in the brain has not yet been explicitly tested. Of particular interest is the ventrolateral prefrontal cortex (vlPFC), which has a known role in semantic retrieval and elaborative encoding (Prince, Tsukiura, & Cabeza, 2007). If emotional memory encoding recruits deep semantic encoding processes more than neutral memory, one should expect to see greater activity in this region during emotional relative to neutral memory encoding. Consistent with this hypothesis, the left vlPFC has been shown to selectively promote emotional memory (Dolcos, et al., 2004a; Kensinger & Corkin, 2004), an effect that can be driven by emotional valence (how positive or negative a stimulus is) alone. With the hippocampus, the vlPFC may constitute an alternative functional network that supports memory encoding and is modulated by emotion. However, these neuroimaging studies have not manipulated encoding demands to vary reliance on prefrontal mechanisms. Thus, the link between prefrontal activations and increased elaborative encoding of emotional stimuli remains speculative.

Although emotion effects on memory have typically been reported in the left vlPFC, right vlPFC is known to participate in successful memory encoding of visual scenes (Brewer, Zhao, Glover, & Gabrieli, 1998; Kirchhoff, Wagner, Maril, & Stern, 2000). Outside of the memory domain, there is additional evidence that the right vlPFC and amygdala are modulated by the interaction of emotion with levels of processing. For instance, some studies have investigated the perception of angry and fearful faces during affect labeling versus perceptual matching (Hariri, Bookheimer, & Mazziotta, 2000; Lieberman, et al., 2007). In these studies, amygdala activity was greatest during the shallow perceptual matching condition, as well as other control conditions, whereas activity in the right vlPFC was greatest in the relatively deep affect labeling condition. Both studies furthermore identified a negative relationship between the amygdala and right vlPFC activations (Hariri, et al., 2000; Lieberman, et al., 2007), suggesting that the right vlPFC plays a role in dampening the amygdala response during affect labeling. Lieberman et al. (2007) interpreted these results as reflecting, in part, the right vlPFC's participation in complex evaluation of emotional stimuli (Cunningham, Johnson, Gatenby, Gore, & Banaji, 2003). This interpretation is consistent with the hypothesis that right vlPFC may contribute to enhanced elaborative processing of emotional stimuli during memory encoding. Furthermore, arousal-driven responses in the amygdala are strongest when these elaborative processes are absent, suggesting that either shallow processing accentuates the influence of emotional arousal or deep processing promotes inhibitory mechanisms that dampen arousal, or both. Thus, we propose that a levels-of-processing manipulation may reveal two distinct pathways to emotional memory formation: an amygdalar pathway that predicts emotional memory benefits during shallow processing, and a prefrontal pathway that provides additional benefits during deep processing. To our knowledge, however, no neuroimaging studies have tested this hypothesis by varying elaborative encoding demands within a subsequent emotional memory design.

In addition to the general influence of emotional arousal, emotional valence may further modulate the effects of levels of processing during memory formation. It has recently been suggested that negative stimuli tend to benefit from perceptual processing during encoding, whereas positive stimuli tend to benefit from semantic processing (Kensinger & Schacter, 2008; Mickley & Kensinger, 2008). Positive moods may be associated with an expansion of attentional breadth, which serves to diminish attentional selection while increasing access to remote semantic associations (Rowe, Hirsh, & Anderson, 2007). Positive compared to negative words also tend to evoke greater semantic priming effects (Rossell & Nobre, 2004). Consistent with these ideas, negative memory encoding tends to be associated with activations in the temporal and occipital lobes, whereas positive memory encoding tends to recruit prefrontal regions including left and right vlPFC (Mickley & Kensinger, 2008). Thus, we expect that the neural underpinnings of deep emotional encoding may drive memory for positive information more so than negative information. These effects may be furthermore modulated by individual differences in sensitivity to positive information—it has been hypothesized that valence effects on subsequent memory are driven by an individual's tendency to prioritize positive versus negative information, and that these valence effects are dependent on controlled processing (Mather & Knight, 2005). We address this issue by relating valence effects on memory-related activity to individual trait differences in the experience of pleasure.

The present study seeks to fill these gaps in the literature by varying levels of processing during encoding of negative, positive, and neutral pictures. One condition employed a shallow, perceptually-focused encoding strategy and the other condition employed a deep, semantic-focused strategy (Figure 1). Recognition memory was tested 2 days later, and was used to identify regions whose activity during encoding varied with subsequent memory as a function of task and emotion.

Figure 1.

Schematic of the experimental design for a single trial during encoding. Separate lists of 70 negative, 70 neutral, and 70 positive pictures were assigned to deep and shallow conditions. Deep and shallow conditions were blocked across runs.

Taken together, this study has 3 main goals. First, we aim to uncover the influence of emotional arousal on deep versus shallow memory encoding. We expect that emotional memory enhancements will be most pronounced in the shallow condition, when arousal is the main determinant of memory, and that likewise the amygdala will be the strongest predictor of memory during shallow encoding. We predict that, during deep encoding, additional regions will distinguish emotional memory encoding from neutral, including those that support elaborative processes associated with our deep encoding task, such as the vlPFC. Second, we aim to identify the influence of emotional valence on deep versus shallow memory encoding. Consistent with the hypothesis that positive material evokes enhanced elaborative processing, it is predicted that emotional memory-related regions in PFC will be preferentially recruited for positive stimuli. We additionally explore the possibility that individual differences in sensitivity to positive information support valence effects on memory-related activity. Finally, we aim to characterize distinct functional networks linking key regions of interest, including amygdala and vlPFC, with the hippocampus. In particular, we examine how these functional networks may be differentially recruited during the encoding of positive versus negative stimuli.

Methods

Participants

Twenty-one young adults (10 female; mean Age = 23.0, SD = 3.1) participated in the study. Participants were healthy, right-handed, native English speakers, with no disclosed history of neurological or psychiatric episodes. Participants gave written informed consent for a protocol approved by the Duke University Institutional Review Board. Due to excessive head movement, 1 of these participants was excluded from all analyses, and all behavioral and neuroimaging analyses were conducted on the remaining 20 participants (10 female; mean Age = 23.2, SD = 3.1).

Materials

Stimuli consisted of 630 pictures. These were selected from the International Affective Picture System (Lang, Bradley, & Cuthbert, 2001) as well as from an in-house, standardized database that allowed us to better equate the pictures for visual complexity and content (e.g., human presence). Pictures were assigned on the basis of a 9-point normative valence scale to emotionally negative (valence: 1-4), neutral (valence: 4-6), and positive (valence: 6-9) conditions. In accordance with the picture selection procedure, standardized valence scores were lower for negative (M =2.85, SD = .62) than neutral pictures (M = 5.14, SD = .43; t (418) = 43.98, p < .001), and higher for positive (M = 7.02, SD = .54) than neutral pictures (t(418) = 39.85, p < .001). Additionally, arousal scores (1 = calm, 9 = excited) were greater for negative (M = 5.72, SD = 0.49) than neutral pictures (M = 3.51, SD = .49; t (418) = 45.95, p < .001), greater for positive (M = 5.68, SD = .59) than neutral pictures (t (418) = 40.91, p < .001), and did not significantly differ between negative and positive pictures (t (418) = .62, p = .54).

Procedure

Participants encoded pictures in the scanner and their recognition memory for these pictures was tested after 2 days. During encoding, participants viewed 140 negative, 140 positive, and 140 neutral pictures while functional MR images were recorded. The encoding session consisted of 10 functional runs, across which negative, positive, and neutral pictures were evenly divided. Runs alternated between two distinct tasks, deep and shallow, described below. To avoid the induction of long-lasting mood states, the pictures within each block where pseudo-randomized so that no more than three pictures of the same valence were consecutively presented. The assignment of encoding stimulus lists to the deep versus shallow task was counterbalanced across participants.

In the deep task, participants were instructed to carefully analyze each picture for its meaning and interpretation, so that after the picture was taken away, they could choose between two possible descriptions of the picture. In the shallow task, participants were instructed to carefully analyze each picture for its perceptual features, particularly colors and lines, so that after the picture was taken away, they could decide whether there was, for example, more red versus green or more horizontal versus vertical lines in the picture. Critically, participants were cued before each run as to which task was next, so that they were able to tailor their processing of each picture to the current task.

Trial structure was similar between tasks (Figure 1). For each trial a picture was presented for 2 seconds. A jittered fixation interval followed each picture presentation, drawn from an exponential distribution with a mean of 2 seconds. After this interval the participant was instructed to rate the picture for its emotional arousal or intensity on a 4-point scale (1 = calm, 4 = excited). The rating screen remained on-screen for 1 second and was immediately followed by a question screen, which varied by task. In the deep task, the question screen said, “Which word best describes the picture?” Two possible options were presented on-screen, both of which were written for each picture such that both could be related to the picture but only one described the true meaning of the picture. In the shallow task, the question screen said, “Which feature are there more of?” Two possible options were presented on-screen: either two color names or the words horizontal and vertical. The question screen remained for 1 second, followed by another jittered fixation interval (mean = 2 s) before the next trial. Responses were collected until the next picture appeared.

Two days after encoding, participants completed a recognition task for the pictures. An additional 70 emotionally negative, 70 positive, and 70 neutral pictures were presented as distracters. Pictures were each presented for 2 seconds, followed by a jittered fixation interval (mean = 2 s). Participants indicated whether the item was old or new using a 5-point scale, with 1 = definitely new, 2 = maybe new, 3 = maybe old, 4 = definitely old, and 5 = remember. Participants were instructed that a remember response indicated the recollection of a specific detail from when they saw that picture during the encoding period, whereas a definitely old response did not include any specific details. After the retrieval session, participants completed the Temporal Experience of Pleasure Scale (TEPS) (Gard, Gard, Kring, & John, 2006), which includes anticipatory and consummatory subscales, the latter of which indexes trait appreciation of positive stimuli and experiential pleasure. They also completed the Behavioral Inhibition System-Behavioral Activation System (BIS-BAS) scale (Carver & White, 1994), which measures individual differences in aversive and appetitive motivation (results not reported).

Behavioral analyses

Average arousal ratings and question accuracy were calculated separately for each trial type. To measure differences in memory responding between conditions, hit rates, false alarm rates, and d’ scores were evaluated for each trial type. The d’ statistic accounts for both hit rates and false alarm rates, thus providing a measure of true accuracy. Because the effect of emotion on memory tends to be strongest when only highly confident responses or recollection estimates are considered (Dolcos, et al., 2005; Ochsner, 2000), d’ was evaluated with its criterion between 3 (‘maybe old’) and 4 (‘definitely old’). That is, responses of 4 and R were taken as ‘old’ and the rest were taken as ‘new’ responses. Encoding response data and d’ scores were entered into separate repeated-measures ANOVAs with emotion (negative, neutral, positive) and task (deep, shallow) as factors. Subsequent post-hoc statistics consisted of repeated-measures ANOVAs with the corresponding factors and variables of interest.

fMRI Methods

Scanning

Images were collected using a 4T GE scanner. Stimuli were presented using liquid crystal display goggles (Resonance Technology, Northridge, CA), and behavioral responses were recorded using a four button fiber optic response box (Resonance Technology). Scanner noise was reduced with earplugs and head motion was minimized using foam pads and a headband. Anatomical scanning started with a T2-weighted sagittal localizer series. The anterior (AC) and posterior commissures (PC) were identified in the midsagittal slice, and 34 contiguous oblique slices were prescribed parallel to the AC-PC plane. High-resolution T1-weighted structural images were collected with a 24-cm field of view (FOV), a 256² matrix, 68 slices, and a slice thickness of 1.9 mm. Functional images were acquired using an inverse spiral sequence with a 2-sec TR, a 31-msec TE, a 24-cm FOV, a 64² matrix, and a 60° flip angle. Thirty-four contiguous slices were acquired with the same slice prescription as the anatomical images. Slice thickness was 3.8 mm, resulting in 3.75 × 3.75 × 3.8 mm voxels.

fMRI analyses

Preprocessing and data analyses were performed using SPM5 software implemented in Matlab (www.fil.ion.ucl.ac.uk/spm/). After discarding the first 6 volumes, the functional images were slice-timing corrected and motion-corrected, spatially normalized to the Montreal Neurological Institute (MNI) template, spatially smoothed using an 8 mm isotropic Gaussian kernel, and resliced to a resolution of 3.75 × 3.75 × 3.8 mm voxels. For each subject, evoked hemodynamic responses to event types were modeled with a delta (stick) function corresponding to stimulus presentation convolved with a canonical hemodynamic response function within the context of the general linear model, as implemented in SPM5. Six main event types were modeled, representing all possible combinations of emotion (negative, neutral, positive) and encoding task (deep, shallow). An additional regressor modeled the separate effects of the arousal rating and question period, but this regressor was not included in any analyses. Thus, all effects reflect activity during the picture period only. Confounding factors (head motion, magnetic field drift) were also included in the model.

The subsequent memory paradigm (Paller & Wagner, 2002) was adapted to identify regions reflecting parametric difference in memory (Dm) effects; that is, regions whose activity increased as a function of recognition response. Linear parametric regressors indexing the subsequent recognition response (1 = definitely new, 2 = maybe new, 3 = maybe old, 4 = definitely old, and 5 = remember) were included for each of the six main trial types. Estimates for the Dm regressors were generated for each participant, and then entered into a group-level, repeated-measures ANOVA with factors for emotion (negative, neutral, positive) and task (deep, shallow). The main effect of Dm was evaluated by contrasting all Dm regressors versus implicit baseline at p < .0025, extent threshold = 5 voxels. This analysis was conducted solely to characterize the network associated with encoding overall, prior to interrogating the effects of emotion and task. Planned contrasts were used to evaluate main effects and interactions within the ANOVA framework, with an emphasis on the effects of arousal (negative and positive > neutral) and valence (negative > positive and positive > negative). Main effects of emotion on Dm were taken as the conjunction of each task effect (e.g., negative and positive > neutral for deep inclusively masked with negative and positive > neutral for shallow), with a joint probability < .0025, extent threshold = 5 voxels, to ensure the presence of a significant effect within each task. Interactions were evaluated within the ANOVA framework via interaction contrasts (e.g., negative and positive > neutral for deep and neutral > negative and positive for shallow) at p < .005, extent threshold = 5. To verify the directionality of the effect, these interactions were inclusively masked with the corresponding within-task effect (e.g., negative and positive > neutral for deep) at p < .05. In this way, the interactions allowed us to isolate those regions that predicted arousal or valence benefits to subsequent memory for one task more than the other. Effects within the amygdala, our a priori region of interest, were taken at p < .05, extent threshold = 5. Although this within-ROI threshold is liberal, there is a wealth of evidence supporting the important role of the amygdala in predicting emotional memory benefits (see LaBar & Cabeza, 2006 for a review) and strong a priori predictions for its role in the present study. To verify the presence of emotion effects in each task independent of subsequent memory, one-sample t-tests evaluating overall effects of emotion (negative and positive > neutral) were conducted for each task. These results are presented in Table 2 and will not be discussed further.

Table 2.

Emotion effects.

	BA	Hem	Talairach Coordinates			t	voxels
			x	y	z
Negative and Positive > Neutral, Deep
Middle/ Inferior Temporal Gyrus	39, 37	L	-48	-72	7	10.67	367
Inferior/ Middle Temporal Gyrus	37, 39	R	48	-70	-6	10.42	509
Precentral Gyrus	3	L	-56	-17	29	7.38	528
Superior Parietal Lobule	7	L	-33	-38	40	6.62	--
Inferior Frontal Gyrus	44	R	56	5	31	6.88	121
Superior Parietal Lobule	7	R	30	-52	52	6.59	243
Midbrain		L	-4	-11	-3	5.86	152
Middle Frontal Gyrus	6	R	26	-1	53	5.84	82
Medial Frontal Gyrus	9, 10	L	-11	52	15	5.8	141
Insula	47	R	30	25	-8	5.65	81
Amygdala		R	23	-4	-13	4.4	--
Inferior Frontal Gyrus	44	L	-48	5	24	5.6	63
Inferior Frontal Gyrus	47	L	-37	25	-4	5.58	107
Precuneus	7	L	-19	-78	42	5.55	21
Midcingulate Gyrus	24	L	-4	-2	28	5.2	56
Inferior Frontal Gyrus	45	R	48	30	9	5	20
Amygdala/ Parahipppocampal Gyrus		L	-30	-4	-13	4.15	10
Insula		R	33	-8	-9	3.96	7
Negative and Positive > Neutral, Shallow
Inferior Temporal Gyrus	37	R	48	-69	-3	10.06	301
Midbrain			0	-29	-2	6.88	109
Medial Frontal Gyrus	10	L	-4	59	18	6.79	104
Middle Temporal Gyrus	39	L	-48	-72	11	6.47	138
Postcentral Gyrus	1	R	59	-20	33	6.24	32
Precentral Gyrus	6	L	-30	-12	53	5.56	111
Amygdala/ Parahippocampal Gyrus		R	33	-1	-19	5.02	18
Peri-Amygdaloid Region	25	L	-22	10	-16	4.75	7
Posterior Cingulate	23	L	-7	-50	23	4.59	56
Superior Frontal Gyrus	8	R	4	17	55	4.57	15
Cerebellum		R	22	-52	-20	4.45	18
Superior Parietal Lobule	7	R	30	-52	52	4.44	20
Inferior Frontal Gyrus	47	L	-45	33	-5	4.36	9
Orbitofrontal Gyrus	11		0	43	-18	4.12	7

Note. BA = Brodmann Area; Hem = Hemisphere; L = Left; R = Right.

Functional network analyses

Across-subject multiple regressions were used to evaluate the relationships among the amygdala, right vlPFC, and hippocampus under each task condition. Anatomical ROIs were derived from the Anatomical Automatic Labeling atlas, as implemented in the WFU Pickatlas, for left and right amygdala, left and right hippocampus, and right inferior frontal gyrus (pars triangularis), the portion of vlPFC that most closely approximated the region identified by the SPM analyses. For each subject, contrast estimates corresponding to the negative deep, positive deep, negative shallow, and positive shallow Dm regressors were extracted from these four ROIs, and their means were entered into separate linear regressions for each condition with right or left hippocampus regressing on the contralateral amygdala and right vlPFC. Because amygdala and vlPFC were included in the same model, beta coefficients reflect the amount of unique variance explained by each. Amygdala regions were chosen to be contralateral rather than ipsilateral to the hippocampus to minimize inflation of the amygdala-hippocampal relationship as a byproduct of spatial smoothing. Thus, each regression measured the degree to which memory-related activity in the hippocampus could be predicted as a function of memory-related activity in the amygdala and right vlPFC, separately for each of the 4 trial types of interest. Because ROIs were derived on an anatomical rather than functional basis, results from the regression analysis provide novel information regarding the relationships between these ROIs, above and beyond the SPM results. Linear regressions were evaluated within SPSS version 15.0 (SPSS Inc., Chicago IL), and standardized beta coefficients and p-values are reported.

Results

Behavioral Analyses

Encoding response data

Average arousal ratings and question accuracy scores were entered into separate repeated-measures ANOVAs with emotion (negative, neutral, positive) and task (deep, shallow) as factors (Table 1). For the arousal ratings, there was a significant main effect of emotion, F(2, 38) = 206.31, p < .001, η_p² = .92. Follow-up tests revealed that negative pictures were rated as more arousing than neutral, F(1, 19) = 335.38, p < .001, or positive, F(1, 19) = 37.63, p < .001, pictures. Positive pictures were also rated as more arousing than neutral pictures, F(1, 19) = 185.47, p < .001. Critically, there was no main effect of task, F(1, 19) < 1, p > .1, η_p² = .02, or interaction of emotion and task, F(2, 38) < 1, p > .1, η_p² = .04, indicating that our task manipulation did not alter the participants’ perceived emotional responses to the stimuli. For the question accuracy scores, there were main effects of task, F(1, 19) = 317.42, p < .001, η_p² = .94, and emotion, F(2, 38) = 10.45, p < .001, η_p² = .36, but only a marginally significant interaction, F(2, 38) = 2.64, p = .08, η_p² = .12. Main effects reflected that encoding accuracy was higher for the deep than shallow task, and for positive than negative, F(1, 19) = 21.22, p < .001, or neutral stimuli, F(1, 19) = 13.80, p = .001. Encoding accuracy in each condition was significantly above chance.

Table 1.

Behavioral results.

Task	Emotion Type	Arousal	Hit Rate	FA Ratea	d′
Deep	Negative	2.79 (.38)	0.64 (0.20)	0.03 (0.03)	2.47 (0.53)
	Neutral	1.39 (.20)	0.55 (0.22)	0.03 (0.02)	2.15 (0.55)
	Positive	2.43 (.42)	0.58 (0.16)	0.04 (0.04)	2.04 (0.53)
Shallow	Negative	2.79 (.43)	0.56 (0.22)	0.03 (0.03)	2.22 (0.52)
	Neutral	1.39 (.20)	0.41 (0.20)	0.03 (0.02)	1.74 (0.54)
	Positive	2.39 (.44)	0.51 (0.17)	0.04 (0.04)	1.84 (0.55)

Note. Data are reported as mean (SD). Arousal refers to individual arousal ratings on a scale from 1 = calm to 4 = excited. FA = False Alarm.

^aFA rates are common to both tasks.

Recognition memory data

Hit rates, false alarm rates, and d’ scores were evaluated for each participant (Table 1). D’ scores were entered into separate repeated-measures ANOVAs with emotion (negative, neutral, positive) and task (deep, shallow) as factors. There was a main effect of emotion, F(2, 38) = 14.63, p < .001, η_p² = .44, indicating that negative pictures were better remembered than neutral, F(1, 19) = 19.52, p < .001, and positive, F(1, 19) = 33.93, p < .001, pictures. There was no difference between positive and neutral pictures, p > .05. There was also a main effect of task, demonstrating the LOP effect, with deeply-encoded pictures being better remembered than shallowly-encoding pictures, F(1, 19) = 87.10, p < .001, η_p² = .82. Critically, the interaction between emotion and task was also significant, F(2, 38) = 4.52, p = .02, η_p² = .19, reflecting a greater difference between emotional and neutral pictures in the shallow task than in the deep task. Although positive memory did not differ from neutral memory, further inspection of the results reveals that this effect may be driven by a trend toward a higher false alarm rate to positive pictures than neutral pictures, F(2, 38) = 2.91, p = .07, η_p² = .13. Indeed, positive hit rates are significantly higher than neutral hit rates in the shallow condition, F(1, 19) = 18.45, p < .001. Thus, when considering only those items represented in the encoding session (i.e., hits and misses), emotion enhanced subsequent memory for both negative and positive pictures.

fMRI Analyses

Overall memory-related activity

Parametric regressors indexing subsequent memory performance were entered into a group-level ANOVA with emotion and task as factors, and planned contrasts were evaluated within this ANOVA framework. To validate the sensitivity of our design to elucidate memory effects, all trial types were first included to identify Dm effects across all conditions, regardless of emotion and task. This contrast yielded the standard memory encoding network, including bilateral MTL clusters spanning hippocampus and parahippocampal gyrus, bilateral fusiform gyrus, and bilateral vlPFC, in addition to bilateral amygdala (Table 3).

Table 3.

Overall Dm Activity.

	BA	Hem	Talairach Coordinates			t	voxels
			x	y	z
Peri-amygdala/ Hippocampus		L	-11	-8	-6	6.45	591
Fusiform Gyrus	37	R	41	-52	-17	5.99	--
	19	R	37	-70	-9	4.74	--
Amygdala/ Parahippocampal Gyrus	34	R	11	-8	-9	5.62	--
Inferior Temporal Gyrus	37	R	52	-62	-3	5.42	--
Parahippocampal Gyrus	35	R	30	-26	-18	3.66	--
Amygdala		R	26	-5	-19	3.28	--
Middle Temporal Gyrus	39	R	48	-69	10	3.11	--
Fusiform Gyrus	37	L	-37	-48	-14	5.7	316
	19	L	-37	-66	-13	5.14	--
Superior Colliculus			-4	-29	-2	4.37	--
Inferior Occipital Gyrus	18	L	-45	-73	-6	3.85	--
Parahippocampal Gyrus	35	L	-19	-37	-5	3.08	--
Superior Frontal Gyrus	9	L	-7	56	18	4.27	53
	9	L	-11	52	29	3.92	--
Medial Frontal Gyrus	9	R	4	52	25	3.1	--
Inferior Frontal Gyrus	45	R	48	37	2	4.81	50
Inferior Frontal Sulcus	45/46	R	48	30	13	4.43	--
Middle Occipital Gyrus	19	R	30	-79	25	3.76	21
	19	R	41	-79	14	3.63	--
Inferior Frontal Gyrus	45	L	-48	22	2	3.68	15
	45	L	-52	23	16	3.33	--
Medial Frontal Gyrus	10	R	7	59	11	3.5	5
Middle Temporal Gyrus	21	R	52	-1	-22	3.35	5

Note. All subpeaks at least 12 mm apart are reported. BA = Brodmann Area; Hem = Hemisphere; L = Left; R = Right

Common and task-specific effects of emotional arousal on memory-related activity

Effects of emotional arousal, regardless of task, were identified by evaluating the negative and positive > neutral contrast for each task and then taking their conjunction (Table 4). The left peri-amygdaloid region was the only region to show Dm effects driven by emotional arousal in both tasks (Figure 2a).

Table 4.

Influence of Emotional Arousal on Dm Activity.

	BA	Hem	Talairach Coordinates			t	voxels
			x	y	z
Negative Dm and Positive Dm > Neutral Dm, Both Deep and Shallow
Peri-Amygdaloid Region		L	-30	6	-26	2.33	8
Negative Dm and Positive Dm > Neutral Dm, Shallow > Deep
Amygdala		R	30	-1	-19	1.99	8
Negative Dm and Positive Dm > Neutral Dm, Deep > Shallow
Inferior Frontal Gyrus	45	R	37	26	16	2.98	5
Posterior Cingulate	31		0	-65	14	2.97	6
Paracentral Lobule	5	R	15	-34	47	2.96	6
Precuneus	7	R	7	-44	58	2.94	5

Note. BA = Brodmann Area; Hem = Hemisphere; L = Left; R = Right. For the conjunction, coordinates refer to the peak associated with the larger t-value.

Figure 2.

Common and task-specific effects of emotional arousal on Dm activity. Activations are overlaid on a T1 template and mean contrast estimates within the activated regions are plotted for each condition to illustrate the effect. a) Left peri-amygdaloid region showing common effects. b) Right amygdala region showing shallow task-specific effects. c) Right vlPFC region showing deep task-specific effects. Error bars denote standard error.

To identify task-specific effects of emotional arousal on memory-related activity, we looked for regions that predicted negative and positive versus neutral Dm more in the shallow task than in the deep task, and vice versa (Table 4). Only the right amygdala showed greater arousal-driven Dm effects in the shallow than in the deep task (Figure 2b). For the converse interaction, a network of regions including the right vlPFC, posterior cingulate, and precuneus exhibited greater arousal-driven Dm effects in the deep task than in the shallow task (Figure 2c). These results are consistent with our predictions that emotional memory encoding in the shallow condition would be primarily supported by the amygdala, whereas deep emotional memory encoding would additionally benefit from memory-related activity in the vlPFC.

Common and task-specific effects of emotional valence on memory-related activity

Effects of emotional valence, regardless of task, were identified by evaluating the positive > negative Dm, and vice versa, contrast for each task and then taking their conjunction (Table 5). Several regions demonstrated stronger Dm effects for positive than negative encoding, including retrosplenial cortex, precuneus, prefrontal cortex, and regions in primary visual cortex. Interestingly, many of these activations, including retrosplenial cortex, were driven by deactivations associated with memory for negative stimuli (Figure 3a). No regions were more active for negative versus positive Dm in both tasks.

Table 5.

Influence of Emotional Valence on Dm Activity.

	BA	Hem	Talairach Coordinates			t	voxels
			x	y	z
Positive Dm > Negative Dm, Both Deep and Shallow
Cuneus/ Lingual Gyrus	17, 18	L	-15	-91	5	3.02	87
Lingual Gyrus	19	L	-15	-58	-3	2.82	10
Retrosplenial Cortex	23		0	-39	19	2.57	25
Superior Frontal Gyrus	6	R	26	6	45	2.4	13
Middle Frontal Gyrus	8	L	-22	13	41	2.3	13
Precuneus	7	L	-11	-64	38	1.9	7
Positive Dm > Negative Dm, Deep > Shallow
Superior Colliculus		L	-4	-26	-2	3.09	7
Caudate Nucleus		L	-15	8	10	2.85	5

Note. BA = Brodmann Area; Hem = Hemisphere; L = Left; R = Right. For the conjunction, coordinates refer to the peak associated with the larger t-value.

Figure 3.

a) Common effects of emotional valence on Dm activity. Retrosplenial cortex activations are overlaid on a T1 template and mean contrast estimates within this activated region are plotted for each condition to illustrate the effect. b) Deep task-specific effects of emotional valence on Dm activity. c) Scatterplot depicting the correlation between each individual's Dm activity within the caudate nucleus during the deep positive condition, and individual scores on the consummatory subscale of the Temporal Experience of Pleasure Scale (TEPS). Error bars denote standard error.

To identify task-specific effects of emotional valence on memory-related activity, we looked for regions that predicted valence effects in the shallow task but not the deep task and vice versa (Table 5). The caudate nucleus and superior colliculus showed positive versus negative Dm effects in the deep task but not in the shallow task (Figure 3b). All other interactions were null. Within this region of caudate nucleus, mean Dm effects in the deep positive condition significantly correlated with individual scores on the consummatory subscale of the TEPS, indicating that the more an individual tends to appreciate positive experiences, the more that region tends to predict memory for deeply-encoded positive images (Figure 3c). Notably, this correlation was not significant for any of the other trial types.

Multiple regression analyses

We further investigated the functional relationships between key regions of interest, focusing on interactions of the amygdala and vlPFC with the hippocampus during memory formation. In particular, we were interested in assessing whether these pathways were differentially recruited for negative versus positive information. For each of the 4 conditions of interest (negative deep, negative shallow, positive deep, and positive shallow), linear multiple regression models measured the degree to which left or right hippocampal Dm effects covaried with Dm effects in the contralateral amygdala and right vlPFC across participants (Figure 4a). Note that although amygdala and vlPFC were similarly activated for both negative and positive Dm, these network analyses assess the degree to which each region influences memory-related activity in the hippocampus, i.e., the relative strength of the pathway. Standardized beta coefficients are presented in Figure 4b.

Figure 4.

a) Schematic of the functional network and equation used in the multiple regression analyses. b) Plots of the standardized beta coefficients as a function of valence for left and right hippocampus. Results are plotted separately for the deep (top) and shallow (bottom) conditions. Asterisks denote the significance of the corresponding coefficient in each condition, p < .05. HC = hippocampus, Amyg = Amygdala, Std = Standardized.

For the left hippocampus, amygdala Dm predicted hippocampal Dm effects for the deep negative (p = .041) and shallow negative (p < .001) and positive (p = .011) conditions. Right vlPFC Dm, on the other hand, predicted hippocampal Dm for the deep positive (p = .016) and shallow positive (p < .001) conditions. For the right hippocampus, amygdala Dm predicted hippocampal Dm effects for the deep negative (p < .001) and shallow negative (p < .001) conditions. Right vlPFC Dm was not a positive predictor of hippocampal Dm for any condition, but was a negative predictor in the shallow negative condition, p = .01. Taken together, the results indicate that, regardless of task, valence modulates the strength of these functional relationships: negative memory tends to be supported more strongly by the amygdala-hippocampal association, whereas positive memory tends to be supported more strongly by the vlPFC-hippocampal association.

Discussion

With respect to the previously outlined goals, the results of this study yield 3 main findings. First, the influence of emotional arousal on memory is modulated by level of processing, with the amygdala predicting emotional memory enhancements best in the shallow condition and the right vlPFC additionally supporting these enhancements in the deep condition. Second, the interaction of valence with level of processing affects memory-related activity in the caudate nucleus, but not in PFC as originally expected. Specifically, the caudate nucleus differentially supports positive memory in the deep condition, an effect related to individual differences in sensitivity to positive experiences. Finally, the amygdala and right vlPFC may participate in distinct functional networks with the hippocampus that are differentially weighted by valence.

Most neuroimaging studies of emotional memory have focused on how the amygdala predicts emotional memory enhancements, presumably by modulating memory consolidation in the MTL memory system (LaBar & Cabeza, 2006; McGaugh, 2004). Consistent with this literature, the present results indicate that the amygdala predicts memory for negative and positive stimuli better than for neutral stimuli. Whereas most previous studies have employed a single encoding task, we expand these findings to compare the role of the amygdala during deep versus shallow encoding. Interestingly, although the results show that the left amygdala predicts emotional memory across both tasks, the right amygdala preferentially predicts emotional memory in the shallow condition. This finding maps onto the behavioral finding that emotional memory benefits are accentuated for those items encoded in the shallow condition. Taken together, these results imply that stimulus-induced arousal and corresponding amygdala activation are the critical determinants of subsequent memory during shallow encoding, when other encoding resources are minimized.

These findings are reminiscent of the results of previous studies comparing perceptual matching versus affect labeling, which can be taken as shallow and deep tasks, respectively. These studies found the amygdala response to fearful and angry faces was weakest in the affect labeling condition (Hariri, et al., 2000; Lieberman, et al., 2007). The present experiment adds the novel finding that the right amygdala likewise predicts emotional memory best in the shallow condition. Shifts in laterality from common to task-specific emotional Dm effects may additionally reflect differences in encoding demands—the left amygdala has been associated with cognitive representations of arousal, which may play a stronger role during deep encoding, whereas the right amygdala has been associated with automatic or conditioned shifts in arousal (Glascher & Adolphs, 2003; Phelps, et al., 2001).

A few studies have attempted to clarify the role of the PFC in predicting emotional memory enhancements, finding that the left vlPFC promotes emotional memory better than neutral (e.g., Dolcos, et al., 2004a; Kensinger & Corkin, 2004). The present study improves upon this literature by contrasting deep versus shallow encoding, thereby isolating emotional memory mechanisms that emerge only when encoding resources are high. The right vlPFC, in addition to regions within parietal cortex, follows this pattern, indicating its role in elaborative processes during encoding that are sensitive to emotional arousal. Although the previous literature on emotional memory has typically highlighted the left vlPFC, the right vlPFC has been shown to participate in successful memory encoding, especially for visual scenes (Brewer, et al., 1998; Kirchhoff, et al., 2000).

The right vlPFC has also been identified as being more activated during affect labeling than perceptual matching or other shallow conditions (Hariri, et al., 2000; Lieberman, et al., 2007). Because this region was inversely related to amygdala activity in these studies, the authors concluded that right vlPFC likely plays a role in dampening the amygdala response during affect labeling. One account for right vlPFC during affect labeling is that this region is involved in symbolic thought about emotional stimuli (Lieberman, et al., 2007). Supporting this idea, this region is activated during evaluative valence judgments, particularly when these judgments are ambivalent or ambiguous, increasing the complexity of the decision (Cunningham, et al., 2003; Nomura, et al., 2003). Alternatively, right vlPFC activation may index inhibitory processes, which may be preferentially recruited during deep processing. Indeed, the right vlPFC has been associated with inhibitory processes across a wide variety of tasks (Aron, Robbins, & Poldrack, 2004), and this region has also been identified during tasks that require the inhibition of emotional responses (Dolcos & McCarthy, 2006). These two accounts may be compatible with each other—in the context of emotional stimuli, inhibitory processes may be supported by evaluative mechanisms, consistent with ideas regarding emotion regulation strategies such as reappraisal. Reappraisal involves re-conceptualizing emotional information, a process known to engage right vlPFC while dampening arousal responses (Ochsner, et al., 2004). Furthermore, reappraisal strategies during encoding have been shown to promote memory for emotional stimuli (Dillon, Ritchey, Johnson, & LaBar, 2007). Taken together, our finding that right vlPFC predicts emotional memory benefits in the deep condition may reflect recruitment of emotion evaluation or regulation processes that in term promote deep memory encoding.

This account may also help reconcile the present results with previous studies of emotional arousal effects on memory encoding, which have typically found emotional Dm effects in both amygdala and vlPFC within the same task. These previous studies have traditionally employed encoding tasks consisting of simple valence ratings (e.g., Dolcos et al., 2004a,b) or binary decisions about the content of the stimulus (e.g., Kensinger & Corkin, 2004; Ritchey et al., 2008). These encoding tasks may have fallen somewhere in between the present shallow and deep tasks in terms of how much they relied on elaborative encoding processes. In the present shallow task, participants were required to conduct careful perceptual analysis of the stimuli, perhaps interfering with any meaning-based processing beyond that required for an arousal rating. In the present deep task, however, participants were required to deeply analyze the meaning of each picture so that they could answer any question about its meaning, a task more demanding of elaborative resources than a binary decision. Thus, in the present study, increased vlPFC Dm effects in the deep condition may have resulted from this condition's heightened demands on elaborative encoding. Another consequence of these demands may have been preferential recruitment of regulatory processing, resulting in relative inhibition of the amygdala Dm effects which predominate in the shallow condition.

In addition to these main findings driven by emotional arousal, our results suggest that emotional valence also interacts with memory encoding. Across both tasks, a network of regions predicted memory for positive more than negative stimuli, although this pattern was primarily driven by inverse relationships with Dm in the negative condition. Some of these regions, including retrosplenial cortex and precuneus, are part of the default network and thought to be important for self-referential processing (Buckner, Andrews-Hanna, & Schacter, 2008). Although we did not anticipate this finding, one possible interpretation for these data is that deactivating the default network benefits memory for negative stimuli, but does not influence memory for positive stimuli. These results hint at possible differences in the kinds of processes that support each form of memory encoding. Additional research is needed to investigate how default network activity differentially impacts valenced memories, which could have important implications for understanding memory biases in psychiatric populations.

We also identified a task-specific valence effect: the caudate nucleus predicts positive versus negative memory in the deep but not shallow task. Furthermore, deep positive Dm effects in this region correlate with individual scores on the consummatory subscale of the TEPS. These results imply that the more an individual is likely to experience consummatory pleasure in response to positive stimuli, the more the caudate nucleus predicts memory for deeply-encoded positive stimuli. These results provide preliminary evidence that individual differences in valence-driven memory biases, both in the normal population and in special populations like depressed patients (Dalgleish & Watts, 1990) or older adults (Mather & Knight, 2005), might be partially underscored by individual differences in the experience of pleasure and the affiliated reward circuitry. Although there has been increasing interest in the relationship between reward processes and memory (Adcock, Thangavel, Whitfield-Gabrieli, Knutson, & Gabrieli, 2006; Wittmann, et al., 2005), little attention has been paid to how reward processes may contribute to memory for positive scenes within a standard emotional memory paradigm (but see Wittmann, Schiltz, Boehler, & Düzel, 2008). It would be worthwhile to unite these literatures by further examining the contribution of the caudate and other parts of the striatum, known to be involved in reward processing (Delgado, 2007), during emotional memory formation.

Consistent with prior suggestions that positive memory tends to be preferentially supported by elaborative encoding, we had expected to find valence interactions in vlPFC. However, these predictions emerged not as activation differences, but as network differences. Results from the multiple regression analysis indicate the presence of functional networks linking Dm activity in the hippocampus with Dm activity in the amygdala and right vlPFC. Task does not seem to influence the strength of these relationships despite task modulation of activity within right vlPFC and amygdala. This may indicate common functional pathways that simply become more or less activated in each task. However, the relative strength of these networks does vary by valence, in that the amygdala pathway tends to be strongest for negative memory and right vlPFC pathway tends to be strongest for positive memory, particularly for the left hippocampus. Although there were no activity differences according to valence within these regions, these network analyses may reflect the degree to which each pathway mediates memory-related changes in the hippocampus. That is, although memory-related activations in the amygdala and vlPFC are recruited by emotional arousal, valence determines the relative influence of these activations on memory-related activity in the hippocampus.

A few caveats are worth noting with regard to the interpretation of the present results. First, although we interpret the differences between negative and positive encoding as arising from their opposing valence, we cannot rule out the possibility that some of the present results are driven by self-reported differences in arousal between negative and positive rather than valence. Under this interpretation, the network results would be consistent with evidence linking left vlPFC to the encoding of non-arousing negative words (Kensinger & Corkin, 2004), although this previous study did not compare negative to positive stimuli. Second, because positive items tended to evoke higher false alarm rates during retrieval, indicating increased memory bias for these items, positive Dm effects may not always reflect processes supporting true memory accuracy. This may not be a problem for the present data since encoding processes such as semantic elaboration may predict subsequent memory bias as well as accuracy (Kim & Cabeza, 2007). Furthermore, it is unclear how bias could elicit task differences in encoding activity. Finally, we interpret task differences as emerging from varying degrees of deep, elaborative processing during encoding. However, we cannot rule out the possibility that these tasks may have facilitated other forms of processing that may have contributed to the present results. For example, the shallow task emphasized perceptual features of the stimuli, which may have in turn emphasized their emotionally-salient details. Similarly, the deep task may have encouraged gist-based processing of the stimuli. These possible differences are consistent with the idea that the features attended to during encoding, such as meaning, can dynamically shift the networks supporting successful memory for emotional stimuli. A fascinating question for future research is whether level of processing at encoding differentially influences gist-based versus detailed emotional memory.

Although much is known regarding the role of the amygdala in promoting emotional memory consolidation, the link between emotion, elaborative encoding, and their neural correlates has remained underexplored. By contrasting deep and shallow encoding tasks, this study isolated those components of emotional memory formation that proceed in the face of limited encoding resources, as well as those that are recruited when elaborative encoding resources are high. The present findings indicate that under shallow encoding, arousal and consequent changes in amygdala activity are the best predictors of subsequent memory, whereas under deep encoding, the right vlPFC additionally predicts subsequent emotional memory benefits. Although these activation patterns are not modulated by valence, valence does influence the relative contributions of these regions to memory-related activity in the hippocampus: amygdala-hippocampal links are strongest during negative memory encoding, whereas prefrontal-hippocampal links are strongest during positive memory encoding. The results suggest two distinct pathways to emotional memory formation: an automatic amygdalar pathway that promotes emotional memory during shallow encoding, especially for negative stimuli, and a prefrontal pathway that provides extra benefits during deep encoding, especially for positive stimuli. Through these systems, emotion influences both the quality of the information being transmitted to the MTL memory system as well as the likelihood that that information will be consolidated into a long-term memory trace.