Affective recognition memory processing and event-related brain potentials

Abstract

Recognition memory was examined for visual affective stimuli using behavioral and event-related brain potential (ERP) measures. Images from the International Affective Picture System (IAPS) that varied systematically in arousal level (low, high) and valence direction (unpleasant, pleasant) were first viewed passively. Then, during a response phase, the original images were intermixed with an equal number of new images and presented, and participants were instructed to press a button to indicate whether each stimulus picture was previously viewed (target) or new (foil). Participants were more sensitive to unpleasant- than to pleasant-valence stimuli and were biased to respond to high-arousal unpleasant stimuli as targets, whether the stimuli were previously viewed or new. Response times (RTs) to target stimuli were systematically affected by valence, whereas RTs to foil stimuli were influenced by arousal level. ERP component amplitudes were generally larger for high than for low arousal levels. The P300 (late positive component) amplitude was largest for high-arousal unpleasant target images. These and other amplitude effects suggest that high-arousal unpleasant stimuli engage a privileged memory-processing route during stimulus processing. Theoretical relationships between affective and memory processes are discussed.

Affect neuropsychology

The quantitative study of memory originally involved stimuli without personal relevance or affect in order to promote objectivity, because reason and emotion were hypothesized to occur separately in the neocortex and limbic systems, respectively (MacLean, 1952). However, contemporary neuroimaging has demonstrated that affect produces neural activity in both limbic and neocortical structures (Bechara, Damasio, & Damasio, 2000; Bush, Luu, & Posner, 2000; Philips, Young, Scott, Caler, Andrew, Giampietro, et al., 1998). For example, a review of neural activity for affect found activation in the anterior cingulate cortex and the medial prefrontal cortex, areas involved in both cognitive and affective processing (Phan, Wager, Taylor, & Liberzon, 2004). One aspect of such activation is reflected in prior affective experiences that influence the current emotional state, which is modulated by both arousal and valence (Lang, 1995; Lang, Bradley, & Cuthbert, 1990; Lang, Greenwald, Bradley, & Hamm, 1993). Arousal reflects how intensely (low, high) motivational systems are activated by stimuli. Valence reflects the type of motivational system response to a stimulus: either a pleasant appetitive/approach reaction, activated in response to desire (procreation, sustenance), or an unpleasant aversive/withdrawal reaction, activated in response to threat (anxiousness, fearfulness). These motivational systems originate from brain networks that promote survival through appetitive and defensive behavior—that is, from the autonomic and somatic neural pathways underlying attention and action (Bradley & Lang, 2007; Davis & Lang, 2003; Lang, 2010; Lang & Bradley, 2010; Lang, Bradley, & Cuthbert, 1998).

The affective stimuli that elicit these reactions benefit from rapid detection and attentional biasing by engaging more intense processing than is engaged by neutral stimuli (Bargh, Chaiken, Govender, & Pratto, 1992; Kensinger, 2009; Kitayama, 1990). Attentional biasing regulates sensory inputs, which facilitates the interruption of current activity to permit focusing on salient emotional information, even when these stimuli are unexpected (Cahill, Haier, Fallon, Alkire, Tang, Keator, et al., 1996; Davis & Whalen, 2001; Dolan & Vuilleumier, 2003; Globisch, Hamm, Esteves, & Ohman, 1999; Hamann, 2001; Merikle, Smilek, & Eastwood, 2001; Mogg, Bradley, De Bono, & Painter, 1997). The amygdala is hypothesized to contribute to affective attentional biasing through reentrant projections to all levels of the visual processing stream (Lane, Chua, & Dolan, 1999). Indeed, activity in the amygdala correlates positively with enhanced recall of visual detail for emotional stimuli (Kensinger, Garoff-Eaton, & Schacter, 2006). Preferentially attended affective stimulus processing is driven by amygdala activity that covaries with hippocampal activation for correctly remembered high-arousal unpleasant stimuli and by interactions of prefrontal cortex with the hippocampus (Dolcos, LaBar, & Cabeza, 2004; Kensinger & Corkin, 2004; Morris, Friston, & Dolan, 1997; Richardson, Strange, & Dolan, 2004). Thus, the amygdala and the surrounding medial temporal lobe, occipital, and prefrontal structures all contribute to affective memory.

These brain mechanisms interact to produce neurocognitive processes that enhance memory for emotional stimuli. As affective stimuli elicit an attentional bias that focuses attention from nonaffective peripheral details to the central emotional content (Easterbrook, 1959; Loftus, 1979), the internal representational overlap with neutral items is decreased, and elaboration of the affective stimuli is strengthened (Schacter & Wiseman, 2006; Schmidt, 1994). In addition, the motivationally relevant nature of affective stimuli increases physiological arousal, thereby increasing cognitive appraisal and encoding strength, augmenting memory for affective events (Christianson & Engelberg, 1999; LeDoux, 2000; Rogers, Kuiper, & Kirker, 1977; Symons & Johnson, 1997). Indeed, both elaboration in the prefrontal cortex and increased cognitive appraisal through amygdala–hippocampal interactions have been found to contribute to enhanced memory for affective stimuli (Dolcos et al., 2004; Morris et al., 1997; Richardson et al., 2004; Steinmetz & Kensinger, 2009).

Distinct neural pathways influence affective memory based on stimulus arousal strength, since memory for arousing stimuli produces activation in the amygdala and hippocampus, whereas memory for nonarousing stimuli yields activation only in the hippocampus (Kensinger & Corkin, 2004). Brain damage to the amygdala alters memory for relatively high-arousing stimuli, but not for low-arousing stimuli (LeBar & Phelps, 1998), and amygdala activation correlates more with stimulus arousal level than with valence direction (Garavan, Pendergrass, Ross, Stein, & Risinger, 2001; Hamann, Ely, Grafton, & Kilts, 2002). In this framework, high-arousal stimuli gain their memory benefit from an automatic focus driven by an amygdala–hippocampal circuit, whereas nonarousing stimuli benefit from elaboration in prefrontal cortex–hippocampal connection (Kensinger, 2009).

Although memory is enhanced for affective stimuli, these stimuli also produce increased false memories. The relatedness of affective semantic constructs facilitates the production of retrieval cues to promote recall (Hunt & McDaniel, 1993; Talmi & Moscovitch, 2004), but this relatedness can lead to faulty retrieval. Related affective items are more likely to be misremembered—especially for items with an unpleasant valence, such as threatening stimuli (Dougal & Rotello, 2007; Sharot & Phelps, 2004; Windmann & Kutas, 2001). Misrecognition can therefore mitigate the benefits of affective memory, especially if semantic relatedness and rapid processing lead to increased but false confidence during retrieval.

Affect and ERPs

Event-related brain potential (ERP) techniques have been used to evaluate affective stimuli, with component amplitude modulations related to emotional stimulus attributes (for a review, see Olofsson, Nordin, Sequeira, & Polich, 2008). Early components (P1, N1, P2) are associated with attentional responses to valence direction (unpleasant, pleasant), whereas later potentials (N2, P3, slow wave) are thought to index cognitive resources responsive to fluctuations stemming from stimulus arousal level (low, high). It is noteworthy that this temporal distinction for affective content is consonant with ERP cognitive timing of attention as occurring with early components of stimulus processing and memory with later components (Azizian & Polich, 2007; Clark & Hillyard, 1996; Vogel & Luck, 2000; Voss & Paller, 2008).

ERP affect studies initially focused on how stimulus valence altered early components (100–350 ms); these amplitude changes were interpreted as reflecting automatic and rapid processing, fostered by fundamental systemic reactions to environmentally and personally relevant stimuli (Bradley, Greenwald, Petry, & Lang, 1992; Esteves, Parra, Dimberg, & Öhman, 1994; Öhman, Flykt, & Esteves, 2001; Öhman & Soares, 1998). The relatively fast responses produced by the salience of high-arousal stimuli (appetitive, aversive) induced attention capture, especially for high-arousal unpleasant items that generated a so-called negativity bias (Cacioppo & Berntson, 1994; Carretié, Mercado, Tapia, & Hinojosa, 2001), wherein threatening stimuli quickly produced valence-related ERP modulations (Cacioppo, Gardner, & Berntson, 1999; Crawford & Cacioppo, 2002).

Long-latency affective ERP amplitude modulations (350 to 1,000 ms) are sometimes described as an aggregate dubbed the “late positive component” (LPC), which can be elicited by contrasting low- and high-arousing stimuli (Cuthbert, Schupp, Bradley, Birbaumer, & Lang, 2000; Delplanque, Silvert, Hot, & Sequeira, 2005; Delplanque, Silvert, Hot, Rigoulot, & Sequeira, 2006; Hajcak, MacNamara, & Olvet, 2010; Rozenkrants & Polich, 2008; Schupp et al., 2000). The larger LPC amplitudes found with high-arousal stimuli are robust, are observed across paradigms, and vary in strength as a function of physical changes in the perceptual factors (Bradley, Hamby, Low, & Lang, 2007; Cano, Class, & Polich, 2009; Codispoti, Ferrari, De Cesarei, & Cardinale, 2006; Delplanque, N’diaye, Scherer, & Grandjean, 2007; Rozenkrants, Olofsson, & Polich, 2008). These arousal influences are thought to originate from attentional resources engaged by motivational factors related to encoding and memory (Polich & Kok, 1995; Schupp, Junghöfer, Weike, & Hamm, 2004), which likely also contribute to the LPC negativity bias (Conroy & Polich, 2007; Delplanque et al., 2006; Delplanque et al., 2005). Thus, ERP components are sensitive to both early (valence) and late (arousal) influences from visual affective stimuli.

Several ERP studies have attempted to directly assess the relationship between emotional stimuli and memory. Increased component amplitudes for affective relative to neutral pictures at parietal electrodes for middle-latency potentials suggest a privileged access to processing resources (Dolcos & Cabeza, 2002), as do the stable memory effects produced by arousal over time (Bradley et al., 1992). In comparisons of words with unpleasant versus neutral meanings, increased false alarm rates are produced for the unpleasant relative to the neutral words. Smaller ERP amplitudes were observed over Fz, suggesting that the selection criterion for unpleasant words was relaxed (Maratos, Allan, & Rugg, 2000; Windmann & Kutas, 2001). Taken together, these ERP results support the relationship between affective processing and memory outcomes. Affective processing differences between pictures and words, high false alarm rates, and the use of neutral stimulus conditions have made the interpretation of these past studies problematic.

The present study

The present study investigated affect and recognition memory interactions using International Affective Picture System (IAPS) images highly normalized for characteristics of low and high arousal as well as negative and positive valence. The goal was to independently manipulate arousal and valence effects and to use the high temporal resolution of ERPs, coupled with an immediate recognition memory task, to characterize the early time course of visual affective and memory processing.

Given the affect and memory findings reviewed above, high-arousal stimuli were hypothesized to elicit larger ERP amplitudes for middle to late latency components, relative to low-arousal stimuli, regardless of valence. Although high-arousal stimuli may generate more false positives, the privileged memory circuit for high-arousal stimuli should result in enhanced memory for high-arousal unpleasant stimuli that would benefit from a “negativity bias” (Dolan & Vuilleumier, 2003; Kensinger, 2009; Kensinger & Corkin, 2004). Early affective processing may be influenced by arousal such that memory for arousing stimuli produces activation in the amygdala and hippocampus, whereas memory for nonarousing stimuli yields activation only in the hippocampus (Kensinger & Corkin, 2004). High-arousing items would therefore benefit from enhanced memory, due to amygdala–hippocampal interactions. Low-arousing items were expected to lack this memory benefit, due to the longer time course necessary for prefrontal–hippocampal elaboration (Kensinger, 2009).

Method

Participants

A total of 24 female undergraduates served as participants for course credit (mean age = 21.1 years, SD = 2.1). Only females were assessed in order to avoid sex difference confounds, because females rate unpleasant content as more unpleasant/arousing than do males, and also demonstrate an aversive bias toward defensive reactivity (Bradley & Lang, 2007). Females also remember more affective details of events than males do (Canli, Desmond, Zhao, & Gabrieli, 2002; Seidlitz & Diener, 1998), perhaps because of sex differences in the amygdala and hippocampus (Goldstein et al., 2001; Jazin & Cahill, 2010). All of the participants were right-handed nonsmokers, had normal or corrected-to-normal vision, were free of neurological/psychiatric disorders, and provided written informed consent after being shown several appetitive and aversive stimulus examples.

Recording conditions

Neuroelectric (EEG) activity was recorded from the Fz, Cz, Pz, Fp1/2, F3/4, F7/8, C3/4, T7/8, P3/P3, and O1/O2 recording sites, referenced to linked earlobes, with a forehead ground and impedances of 10 kΩ or less. Additional electrodes were placed at the outer canthi, as well as above and below the left eye, to assess electro-ocular (EOG) activity. The bandpass was 0.02–50 Hz (3 dB/octave), and the EEG was sampled at 256 Hz for 1,000 ms, with a 100-ms prestimulus baseline. The waveforms were averaged offline, and individual trials on which the EEG or EOG exceeded ±100 μV were rejected. All recordings were made with eyes open, and scheduled rest periods were provided. The participants were instructed to sit upright with 75 cm between the front edge of their chair and the plane of the computer screen.

Stimuli and procedure

Figure 1 illustrates the distribution of the arousal and valence stimulus ratings (9-point scales, females only) from the IAPS stimuli employed, as compared to all of the available IAPS images. The Appendix lists the specific IAPS image numbers used, and examples can be found in previous publications (Cano et al., 2009; Rozenkrants et al., 2008; Rozenkrants & Polich, 2008).

Fig. 1.

Scatterplot of valence and arousal ratings (1–9 scales) for all available International Affective Picture System pictures. The darker points denote the images used in this study

Appendix.

International Affective Picture stimulus numbers of selected images for each stimulus category

Arousal and valence categories	IAPS Images selected
Low-arousal, unpleasant	2205, 2276, 2375.1, 2455, 2750, 2900.1, 3300, 9000, 9220, 9265, 9280, 9290, 9320, 9330, 9342, 9561, 9421, 9435, 3181, 2141, 9007, 9830, 2900.1, 9432, 6311, 9471, 9331, 9041, 9001, 2753, 2590, 2490
High-arousal, unpleasant	2730, 3030, 3400, 6230, 6250, 6260, 6300, 6313, 6350, 6370, 6510, 6530, 6550, 8485, 9250, 9600, 3530, 6821, 6360, 9050, 6312, 6315, 9921, 6560, 9635.1, 3150, 9810, 9252, 6540, 3071, 6250.1, 3500
Low-arousal, pleasant	1463, 1500, 1510, 1590, 1721, 1920, 2080, 2224, 2331, 2395, 2530, 5594, 5820, 7280, 7480, 8461, 2391, 7430, 1340, 1540, 4614, 8497, 2650, 1600, 2341, 1999, 4622, 7580, 2311, 2170, 5982, 5831
High-arousal, pleasant	8080, 8180, 8185, 8186, 8370, 8400, 8490, 8501, 5460, 7502, 8210, 5470, 8161, 5626, 8170, 8300, 8470, 4689, 4800, 8190, 4659, 8200, 4220, 4290

The pictures were selected such that 96 low-arousal and 96 high-arousal images were employed, with each group containing 48 unpleasant and 48 pleasant images. From each category of 48 images, 24 were selected as targets and 24 as foils; specific images were consistently employed as either a target or foil to minimize subsequent recognition variability. All stimuli measured 20 × 15 cm and were centrally displayed on a light gray background at a normal luminance level on a computer monitor.

Figure 1 indicates that the specific stimuli were chosen to reflect the extremes of the arousal and valence ratings, purposefully excluding any images within the neutral range in order to maximize the arousal and, especially, the valence differences. The mean ratings for the arousal/valence categories were low-arousal/unpleasant, M = 4.6/2.8 (SD = 2.1/1.6); high-arousal/unpleasant, M = 6.8/2.1 (SD = 2.1/1.6); low-arousal/pleasant, M = 4.4/7.2 (SD = 2.3/1.6); and high-arousal/pleasant, M = 6.4/7.2 (SD = 2.1/1.6). A two-factor (2 arousal × 2 valence) analysis of variance was applied to the rating values for each image. Low-arousal images were rated lower than were high-arousal images, F(1, 60) = 390.0, p < .001, and likewise, unpleasant-valence images were rated as more unpleasant than were pleasant-valence images, F(1, 60) = 2,211.2, p < .001. No interactions were obtained.

The image stimuli were arranged for passive viewing in three sequences that were constructed using 32 target images in each, with 8 images from each of the four affective categories and with no more than 3 images from the same affective category displayed consecutively. One minute after the viewing sequence of pictures, a response sequence was presented, using the 32 target images plus an additional 32 foil images intermixed. Participants were instructed to indicate whether the stimuli presented in the same manner were previously seen. Each stimulus was presented for a duration of 1,000 ms, followed by a blank screen for 1,000 ms, and the next stimulus was then presented. With 8 target stimuli from each category viewed per sequence, there were 24 target stimuli for each affect group condition. Each IAPS picture occurred in only one sequence, and the presentation order of the three sequences was counterbalanced across participants.

Participants were instructed to press the keyboard “S” key with the index finger of their left hand if the image had been presented previously (i.e., was a target) and to press the “H” key with the index finger of their right hand if the image had not been presented previously (i.e., was a foil). All of the right-handed, female participants responded in this fashion in order to preclude response variability. Participants were instructed to respond to each stimulus and were encouraged to guess if they thought that the image was a target, even if they were unsure. This approach was adopted and applied uniformly in order to facilitate obtaining a sufficient number of acceptable ERP trials. Error rate and response time (RT) data were recorded.

Results

Behavioral data

Figure 2 illustrates the means (+ SEMs) of d′ (detection sensitivity) and c′ (response bias), as well as RTs for the target and foil stimuli from each category (Macmillan & Creelman, 1991). Table 1 summarizes the analyses of variance (2 arousal × 2 valence), with F ratios (dfs given in parentheses), probabilities, and partial-η² values presented for each behavioral variable. Unpleasant images produced larger d′ values than did pleasant images and therefore were detected more readily. Unpleasant images also yielded smaller c′ values than did pleasant images, with high-arousal unpleasant images yielding a smaller c′ value than all other conditions, producing an interaction between the arousal and valence affective categories. These results imply that high-arousal unpleasant affective pictures elicited a more biased, stronger response than did stimuli from the other categories.

Fig. 2.

Behavioral d′, c′, and response times for targets and foils as a function of unpleasant and pleasant valence for each arousal condition (see Table 1)

Table 1.

Behavioral data: Summaries of the two-factor (2 arousal × 2 valence) repeated measures analyses of variance applied to each of the dependent measures

	Target detection (d′)		Response bias (c′)		Target response time		Foil response time
	F	η2	F	η2	F	η2	F	η2
Arousal (1, 23)	—	–	11.12**	0.32	–	–	53.93***	0.71
Valence (1, 23)	15.11***	0.39	7.9**	0.24	15.89***	0.41	33.87***	0.59
A × V (2, 46)	–	–	12.65***	0.38	6.21*	0.21	11.63**	0.33

Partial η² indexes the proportion of variance accounted for by each factor.

^*p < 0.05,

^**p < 0.01,

^***p < 0.001

RTs from the target stimuli were longer for unpleasant than for pleasant images but were not reliably different for high-arousal stimuli across valences. An interaction between the factors suggested that recognition memory may have been engaged more quickly for relatively low-arousal and pleasant stimuli, since valence level accounted for considerable variability. RTs from the foils were longer for high- than for low-arousal images and longer for unpleasant than for pleasant stimuli. An interaction between the two factors suggested that stimulus arousal level influenced foil judgment responses more than did valence level. Thus, arousal level contributed to response bias tendencies and to foil RTs, whereas valence affected all behavioral measures.

ERP data

ERP averages for each participant and stimulus condition were obtained after excluding both individual trials containing artifacts (using a criterion of ±100 μV) and trials producing an incorrect response. These averages ranged from 70.1% to 77.9% for the view-sequence stimuli and from 81.5% to 88.1% for the response-sequence stimuli across affect categories. A two-factor (2 arousal × 2 valence) analysis of variance found no differences for the number of view- or response-sequence trials (p > .15 and p > .65, respectively). The mean number of target stimulus averages was 20.0 (SD = 3.4), and the mean of the foil stimulus averages was 20.7 (SD = 5.5), with no statistical differences between the two conditions (p > .65).

Figure 3 illustrates the grand average ERPs from the midline electrodes, with the low- and high-arousal conditions overlapped for the unpleasant and pleasant stimuli from the passive-view, target stimulus, and foil stimulus conditions. Mean amplitude areas were obtained by summing the voltage within the latency windows for each component, as indicated in the statistical summary tables (see below). For the passive-viewing condition, slow-wave amplitude was assessed across the entire time window. For the target and foil stimulus conditions, slow-wave amplitude was assessed using two time windows designed to capture different processes that have been observed previously (e.g., Olofsson & Polich, 2007; Rozenkrants & Polich, 2008). Hemispheric amplitude differences were also assessed but did not produce results appreciably different from the midline electrodes and will not be considered further. Component peak latencies yielded no significant effects for affect category and also will not be considered further.

Fig. 3.

Grand averages of event-related potentials, with arousal conditions overlapped for pictures with unpleasant (left) and pleasant (right) valences in the passive-view, target stimulus, and foil stimulus experimental conditions

Tables 2, 3, and 4 summarize the three-way repeated measures (2 arousal × 2 valence × 3 electrodes) analyses of variance, with F ratios (dfs in parentheses), probabilities, and partial-η² values presented for each ERP component from the passive-view, target stimulus, and foil stimulus conditions. Greenhouse–Geisser corrections to the dfs were applied when needed, and the corrected probabilities are reported. Electrode location produced strong significant main effects, since the individual components were consistently at maximum centrally and parietally. These topographic outcomes will be considered only in the context of interactions with the arousal and valence factors.

Table 2.

Passive view: Summaries of the three-factor (2 arousal × 2 valence × 3 electrode) repeated measures analyses of variance applied to area measure amplitude data from each component latency window (in milliseconds)

	P1 (100–200)		N1 (200–350)		N2 (300–450)		P2 (400–550)		P3 (500–750)		Slow Wave (700–900)
	F	η2	F	η2	F	η2	F	η2	F	η2	F	η2
Arousal (1, 23)	–	–	–	–	7.61**	0.24	10.15**	0.31	5.91*	0.21	–	–
Valence (1, 23)	–	–	–	–	–	–	–	–	–	–	–	–
Electrode (2, 46)	29.09***	0.55	25.56***	0.52	27.95***	0.54	29.16***	0.55	27.23***	0.54	18.01**	0.43
A × V (1, 23)	–	–	–	–	–	–	–	–	8.38**	0.26	7.78**	0.25
A × E (2, 46)	–	–	–	–	–	–	–	–	–	–	–	–
V × E (2, 46)	–	–	–	–	4.33*	0.15	6.42**	0.21	7.18	0.23	5.77**	0.21
A × V × E (2, 46)	–	–	–	–	–	–	–	–	–	–	–	–

Partial η² indexes the proportion of variance accounted for by each factor.

^*p < 0.05,

^**p < 0.01,

^***p < 0.001

Table 3.

Target stimuli: Summaries of the three-factor (2 arousal × 2 valence × 3 electrode) repeated measures analyses of variance applied to area measure amplitude data from each component latency window (in milliseconds)

	P1 (100–200)		N1 (200–300)		N2 (300–400)		P2 (350–450)		P3 (450–650)		SW, Early (650–750)		SW, Late (750–900)
	F	η2	F	η2	F	η2	F	η2	F	η2	F	η2	F	η2
Arousal (1, 23)	–	–	11.5**	0.33	20.8***	0.47	29.4***	0.56	14.4***	0.38	13.4***	0.36	10.9**	0.32
Valence (1, 23)	–	–	6.2*	0.21	8.2**	0.26	–	–	4.5*	0.16	–	–	–	–
Electrode (2, 46)	20.8***	0.47	13.7***	0.37	12.9***	0.35	9.9***	0.31	14.7***	0.39	11.7***	0.338	3.7*	0.13
A × V (1, 23)	–	–	–	–	–	–	–	–	–	–	–	–	–	–
A × E (2, 46)	10.1***	0.31	7.3***	0.24	6.6**	0.22	21.9***	0.48	17.2***	0.42	15.6***	0.41	13.0***	0.36
V × E (2, 46)	–	–	–	–	14.9***	0.39	12.1***	0.34	12.6***	0.35	12.3***	0.34	6.3**	0.21
A × V × E (2, 46)	–	–	–	–	–	–	–	–	8.1***	0.26	5.7**	0.19	3.5*	0.13

Partial η² indexes the proportion of variance accounted for by each factor. SW, slow wave.

^*p < 0.05,

^**p < 0.01,

^***p < 0.001

Table 4.

Foil stimuli: Summaries of the three-factor (2 arousal × 2 valence × 3 electrode) repeated measures analyses of variance applied to area measure amplitude data from each component latency window (in milliseconds)

	P1 (100–200)		N1 (200–300)		N2 (300–400)		P2 (350–450)		P3 (450–650)		SW, Early (650–750)		SW, Late (750–900)
	F	η2	F	η2	F	η2	F	η2	F	η2	F	η2	F	η2
Arousal (1, 23)	–	–	16.5***	0.41	50.2***	0.68	42.1***	0.64	21.71***	0.48	24.194***	0.51	19.51***	0.45
Valence (1, 23)	–	–	–	–	–	–	–	–	–	–	–	–	–	–
Electrode (2, 46)	22.8***	0.49	15.6***	0.41	13.6***	0.37	15.1***	0.39	17.03***	0.42	15.59***	0.41	9.61***	0.29
A × V (1, 23)	–	–	–	–	–	–	–	–	–	–	–	–	–	–
A × E (2, 46)	–	–	–	–	–	–	–	–	–	–	5.02**	0.17	–	–
V × E (2, 46)	–	–	–	–	–	–	–	–	–	–	–	–	–	–
A × V × E (2, 46)	–	–	–	–	–	–	–	–	8.61***	0.27	–	–	–	–

Partial η² indexes the proportion of variance accounted for by each factor. SW, slow wave.

^*p < 0.05,

^**p < 0.01,

^***p < 0.001

Figure 4 illustrates the mean (+ SEM) P3 component amplitudes as a function of midline electrodes for the passive-view, target stimulus, and foil stimulus conditions. The statistical patterns are organized in the three tables for the different processing events, and each stimulus condition was evaluated in terms of how the arousal and valence factors contributed to each of the ERP measures. The P3 results for the separate conditions will be used in the figures to guide the overview.

Fig. 4.

Mean P3 amplitudes (+ SEMs) as a function of the midline electrodes, with arousal conditions compared in each panel, unpleasant-valence pictures on the top row, and pleasant-valence pictures on the bottom row, for the passive-view, target stimulus, and foil stimulus experimental conditions (see Tables 2, 3, and 4)

Passive viewing (Table 2)

High-arousal stimuli produced larger amplitudes across valence levels for the middle to late portions of the waveform. However, high-arousal stimuli demonstrated appreciably larger amplitudes for unpleasant than for pleasant stimuli for the later components, yielding significant interactions between arousal and valence level. Unpleasant stimuli demonstrated stronger topography effects than did pleasant stimuli for the middle and later components, such that amplitudes generally increased from the frontal to the parietal electrodes, producing significant interactions between the valence and electrode factors. These results are striking, in that simply viewing affectively distinct stimuli, in the absence of a response, produced main effects of arousal level that interacted with valence level, which itself reflects the topography influences of electrode location.

Target stimuli (Table 3)

High-arousal stimuli yielded larger amplitudes than did low-arousal stimuli for the early, middle, and late latency components, although the largest proportions of variance accounted for were produced by the middle P2 and N2 potentials. Unpleasant-valence stimuli demonstrated larger amplitudes than did pleasant-valence stimuli for the early N1 and P2 components as well as for the later P3 potential, with about the same proportion of variance accounted for by each component. These outcomes indicate that recognition memory processing was altered by both affective factors, but at different times.

The effects of arousal and valence level on mean amplitudes increased across electrode locations, with both variables accounting for more variance from the later N2, P3, and SW components. The three-way interactions appear to originate primarily from P3 potential changes, which accounted for the largest proportion of variance. Taken together, the target stimulus ERPs from the recognition memory task delineate the relative importance of arousal level on the entire waveform, with the primary valence effects occurring earlier, as is the case for tasks that do not directly assess memory (Delplanque et al., 2005, 2006; Olofsson et al., 2008).

Foil stimuli (Table 4)

In a manner similar to the target stimuli, but with even more experimental variance accounted for, high-arousal foil stimuli yielded larger amplitudes than low-arousal foil stimuli for the early, middle, and late latency components. No main effects of valence were obtained. However, high-arousal stimuli did not increase in amplitude as much as did low-arousal stimuli, yielding a reliable interaction between the arousal and electrode factors for the SW early component, which was actually somewhat stronger for the P3 component when valence level contributed to the three-way interaction among all three variables. This P3 result was virtually identical to that obtained for the targets, suggesting that the memory processing effects engendered were less pronounced and more focused than those for the target stimuli.

Discussion

Affective recognition memory and behavioral findings

The behavioral outcomes demonstrated that recognition memory is influenced by both the arousal and valence content of a stimulus. Arousal level modulated behavioral measures less strongly than did valence, with arousal influencing response bias and foil RTs, whereas valence influenced detection sensitivity and RTs for all stimulus categories. The findings imply that these two emotion dimensions contribute to recognition memory independently and can strongly interact, especially for high-arousal unpleasant images relative to other affective stimulus groups.

Valence affected sensitivity to targets such that unpleasant images elicited recognition superior to that for pleasant images—not an unexpected outcome, given an inherent negativity bias (Cacioppo & Berntson, 1994; Carretié, Mercado, Tapia, & Hinojosa, 2001). Participants also were predisposed to respond to unpleasant high-arousal foils as if they were targets, which is consistent with previous reports that unpleasant-valence images are subject to misrecognition (Dougal & Rotello, 2007; Sharot & Phelps, 2004). Despite their worse recognition, pleasant images elicited shorter RTs than did unpleasant images, possibly because pleasant stimuli have been reported to elicit a feeling of familiarity, in contrast to the vivid detail evoked by unpleasant images, and this feeling of familiarity has been associated with shorter RTs (Bless & Schwartz, 1999; Duarte, Ranganath, Winward, Hayward, & Knight, 2004; Ochsner, 2000).

Exactly how memory strength alters RTs is unclear, because dissenting findings have suggested that familiarity can produce longer RTs (Dewhurst, Holmes, Brandt, & Dean, 2006; Wixted, 2009). High-arousal foil stimuli demonstrated longer RTs relative to low-arousal foil stimuli, again suggesting that familiar, low-arousal items elicit quicker responses than do vivid, high-arousal stimuli. Unpleasant high-arousal stimuli, which are possibly the most vivid, were observed to have the longest RTs. The two dimensions produced reliable interactions stemming from the high-arousal unpleasant stimuli, interactions possibly driven by the threatening nature of these stimuli (Bradley & Lang, 2007; Lang et al., 1998). Overall, these results imply that valence increases memory storage probability and slows recognition RTs for unpleasant relative to pleasant stimuli, and perhaps this is the basis that underlies the negativity bias in emotional processing and regulation (Crawford & Cacioppo, 2002; Hajcak & Nieuwenhuis, 2006).

Motivationally relevant emotional stimuli produce a memory benefit as compared to less relevant neutral stimuli, which suggests that arousal level and motivational valence direction determine the cognitive imprint from affect stimuli (Christianson & Engelberg, 1999; LeDoux, 2000). In the present study, the superior detection rates and longer RTs for unpleasant images may have been related to the often threatening or anxiety-producing nature of these stimuli. Since they are important to avoid, they perhaps elicit additional cognitive appraisal during stimulus evaluation, relative to less temporally pressing pleasant stimuli. Correctly avoiding a threat is more important to survival than quickly approaching an appetitive goal, so that greater attention and cognitive appraisal may focus on unpleasant stimuli (Bradley, 2010; Cuthbert et al., 2000; Lang et al., 1990).

The importance of reacting to threats is highlighted by the response pattern for high-arousal unpleasant stimuli used as foils. When viewing these stimuli, attention is quickly captured by the central threatening stimulus feature and away from more peripheral details (Bradley, 2010; Codispoti et al., 2006). The longer RTs for high-arousal foils, especially for unpleasant high-arousal foils, suggest difficulty in rejecting stimuli from this category as targets, since attention is captured by the threatening central features rather than by the overall stimulus content. Because threatening details are semantically related, focusing on them instead of the overall stimulus content can lead to misrecognition (Dougal & Rotello, 2007; Sharot & Phelps, 2004; Windmann & Kutas, 2001). This view is consistent with findings that the unpleasant character of such pictures stems from vivid details that contribute to the negativity bias (Cacioppo et al., 1999; Carretié et al., 2001), whereas pleasant pictures are associated with a feeling of familiarity (Bless & Schwartz, 1999; Ochsner, 2000). Furthermore, females may be more responsive to unpleasant stimuli than are males, who may process affective stimuli differently because of fundamental brain structure differences (Bradley & Lang, 2007; Cahill, 2006; Seidlitz & Diener, 1998).

Affective recognition memory and ERPs

During passive viewing, larger amplitudes were obtained for high-arousal than for low-arousal stimuli in middle to late components, whereas during the response condition, high-arousal images elicited larger ERP amplitudes over the entire time course, particularly for the middle latency components. The consistent larger amplitudes for highly arousing stimuli indicates that more attentional resources were devoted to these stimuli (Dolcos & Cabeza, 2002; Polich & Kok, 1995), even though these images did not display a recognition memory benefit. These amplitude findings are in agreement with nonmemory affect tasks, however, indicating the disconnect between strong ERPs and weak behavioral effects for arousal level suggests that the larger P3/LPC component may be an intrinsic processing signature of highly arousing stimuli (Bradley, 2010; Olofsson et al., 2008).

When passively viewing the stimuli, high-arousal unpleasant images elicited a greater increase across midline electrodes during the middle and late components than did the high-arousal pleasant stimuli, producing an interaction between valence and scalp distribution. That these variations occurred when viewing as well as when responding support the hypothesis that perceptual qualities interact with memory processes to contribute to ERP morphology (Garavan et al., 2001; Hamann et al., 2002). Several reports have begun to isolate the contributions of the structural properties of visual stimuli to ERPs, with a general finding that image complexity affects LPC size, such that picture content interacts with arousal level (e.g., sports pictures—Weinberg & Hajcak, 2010). Thus, the effects of image structure on overall attentional and affective processing are being characterized, but with a strictly emotional interpretation not always supported (Bradley, 2010; Cano et al., 2009; Codispoti et al., 2006).

Although ERP valence main effects were not found, in contrast to the strong main effects of valence in the behavioral data, valence did interact with scalp amplitude topography. When passively viewing and when responding to target images, unpleasant stimuli elicited larger amplitudes than did pleasant images across the midline during the middle to late latency components. Given the increased amplitude of ERP components associated with the assignment of cognitive resources and memory during the learning of the stimuli, this outcome may account for the increased sensitivity to unpleasant stimuli (Azizian & Polich, 2007; Clark & Hillyard, 1996). Indeed, when participants were instructed to attend to the nonsalient details of negative pictures, or when they attended to “exciting” sports pictures that communicated no threat, reduced amplitudes for late components were obtained (Dunning & Hajcak, 2009; Weinberg & Hajcak, 2010). Such a response may originate from the need to process the salient details associated with threats, since high-arousal unpleasant stimuli elicited larger ERP amplitudes, reflecting statistical interactions among arousal, valence, and scalp topography for the middle to late latency components, which index cognitive resource allocation (Polich, 2007).

Theoretical implications and conclusions

The neural activity associated with these behavioral outcomes is influenced by both arousal and valence in different ways and at different times (Dolan & Vuilleumier, 2003; Kensinger et al., 2006; Richardson et al., 2004). High-arousal images elicit larger ERP amplitudes in middle to late latency components, even when passively viewing visual stimuli (Cuthbert et al., 2000; Delplanque et al., 2005). When an active response is required that engages attentional resources, arousal-level effects are found for the early, middle, and late components. In this context, it is noteworthy that ERP modulation from valence direction has been observed for early potentials, but amplitude changes are found inconsistently for later components (Bradley et al., 1992; Öhman et al., 2001). These ERP patterns suggest that valence influences early and middle latency components when responding to target stimuli but inconsistently alters amplitude when images are non-targets or are viewed passively. The variable nature of ERP valence effects indicates that affective images engage an attentional default mechanism that reacts to arousal but not to valence when the images are viewed passively or as nontargets (Schupp et al., 2000, 2004). The obtained valence × topography interaction, which reflects greater allocation of attention resources to unpleasant images, may underlie the valence-related behavioral effects, whereas the attentional default mechanism for arousal explains the strong ERP arousal modulation but the weaker behavior effects.

High-arousal images may all share a common neural circuit, but the early processing of high-arousal stimuli can vary appreciably with respect to valence direction (Hamann et al., 2002; Kensinger & Corkin, 2004). The amplitude and topography differences found for high-arousal unpleasant stimuli suggest a possible valence-related disassociation with respect to how high-arousal images are processed. The need to quickly and correctly identify, evaluate, and respond to threats leads to a greater allocation of cognitive resources to unpleasant than to pleasant high-arousal images. Once the immediate threat is analyzed, later processing of high-arousal stimuli may be assessed similarly to pleasant stimuli, thereby reflecting arousal’s dual importance to propagating and protecting life (Davis & Lang, 2003; Lang et al., 1990).

Arousal and valence contribute to emotional stimulus processing, both independently and interactively. Recognition ERP memory measures index different arousal levels and valence directions to produce distinct neural activation patterns based on the intrinsic visual properties of affective stimuli. High-arousal unpleasant stimuli are also processed in a unique manner, relative to other categories, possibly because of their perceived menacing details. Thus, an early focus on unpleasant stimuli is caused by stimulus properties indicating threat and enhances subsequent memory, almost regardless of the overriding processing task.