Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2007 Jan 31;27(5):1082–1089. doi: 10.1523/JNEUROSCI.3223-06.2007

Selective Visual Attention to Emotion

Harald T Schupp 1,✉, Jessica Stockburger 1, Maurizio Codispoti 2, Markus Junghöfer 3, Almut I Weike 4, Alfons O Hamm 4
PMCID: PMC6673176  PMID: 17267562

Abstract

Visual attention can be voluntarily directed toward stimuli and is attracted by stimuli that are emotionally significant. The present study explored the case when both processes coincide and attention is directed to emotional stimuli. Participants viewed a rapid and continuous stream of high-arousing erotica and mutilation stimuli as well as low-arousing control images. Each of the three stimulus categories served in separate runs as target or nontarget category. Event-related brain potential measures revealed that the interaction of attention and emotion varied for specific processing stages. The effects of attention and emotional significance operated additively during perceptual encoding indexed by negative-going potentials over posterior regions (∼200–350 ms after stimulus onset). In contrast, thought to reflect the process of stimulus evaluation, P3 target effects (∼400–600 ms after stimulus onset) were markedly augmented when erotica and mutilation compared with control stimuli were the focus of attention. Thus, emotion potentiated attention effects specifically during later stages of processing. These findings suggest to specify the interaction of attention and emotion in distinct processing stages.

Keywords: attention, emotion, ERP, posterior negative potential, P3, late positive potential

Introduction

The preferential processing of high-priority stimuli in the environment is an essential function of selective attention. The neural signature of explicitly directed attention toward visual features, objects, and higher-order semantic categories has been assessed in recent cognitive neuroscience studies. When attending to specific stimulus features, functional magnetic resonance imaging (fMRI) studies revealed accentuated activations in occipital and inferior temporal cortical regions preferentially engaged by specific stimulus attributes (Kastner and Ungerleider, 2000). Detailing the temporal characteristics of attentional selection, event-related potential (ERPs) studies revealed the operation of selective attention in specific processing stages. Attention to distinct stimulus features or higher-order categorizations defined by semantic meaning is associated with a broadly distributed negative ERP difference over temporo-occipital regions occurring ∼150–350 ms after stimulus onset. This occipital negativity effect reflects a transitory processing period at which task-relevant stimuli are presumably selected for elaborate processing (Potts and Tucker, 2001; Delorme et al., 2004; Codispoti et al., 2006a). Subsequently, target compared with nontarget stimulus processing is associated with an enlarged P3 wave, representing the most explored ERP index in selective attention paradigms. The P3 is typically observed as centroparietal positivity occurring between 300 and 700 ms after stimulus onset, serving as a measure of stimulus evaluation (Johnson, 1988). Thus, ERP components with distinct polarity, topography, and timing reveal selective attention effects at the level of fleeting processing stages.

In addition to voluntary attention, emotional stimuli may command attentional resources and priority processing because of their intrinsic stimulus significance (Vuilleumier, 2005). fMRI studies revealed increased activations by emotionally arousing pictures in occipital, parietal, and inferior temporal cortex (Bradley et al., 2003; Junghöfer et al., 2005, 2006; Sabatinelli et al., 2005). Furthermore, ERP research determined that the processing of high-arousing pleasant and unpleasant compared with low-arousing control images is associated with a negative-going potential over temporo-occipital sensor regions ∼150–300 ms after stimulus onset, followed by increased positive potentials over centroparietal regions (Junghöfer et al., 2001; Keil et al., 2002; Schupp et al., 2003). Accordingly, processing of emotional cues is associated with similar ERP effects as observed for explicitly directed attention.

What will happen when both processes coincide and emotional stimuli become the focus of attention? Given their significance for survival, reproduction, and procreation, explicit attention effects might be accentuated for the processing of emotionally significant stimuli. Providing evidence for a synergistic relationship, emotion potentiated the effects of attention on low-level visual processing in stimulus-driven attention (Phelps et al., 2006). Alternatively, as observed with explicit spatial attention manipulations (Keil et al., 2004), emotion and attention may operate simultaneously but independently from each other. Finally, the effects of paying attention to emotion may vary for specific processing stages. To assure that high-priority stimuli gain access to capacity-limited processes associated with focused attention and conscious recognition, emotion may potentiate selective attention particularly during later stages of stimulus processing (Öhman et al., 2000). To determine the relationship of emotional significance and directed attention, participants viewed a rapid and continuous stream of erotica, mutilations, and low-arousing people images, while each stimulus category served in separate runs as target and nontarget category.

Materials and Methods

Participants

Participants were 16 (eight females) right-handed introductory psychology students from the University of Greifswald. They received course credits toward their research requirements and could receive a maximum of 9 Euro with perfect behavioral performance in the counting task. Participants were between the ages of 19 and 32 years [mean (M), 25]. The participants provided written informed consent for the protocol approved by the Review Board of the University of Greifswald.

Stimulus materials and task

Participants viewed 30 pictures from the International Affective Picture System (IAPS), a collection of standardized photographic materials (Lang et al., 2005). The three picture categories consisted of 10 picture contents showing heterosexual couples or males/females alone in erotic postures (erotica), 10 images depicting scenes of mutilation and injury (mutilation), and 10 pictures showing couples and individual people in everyday life situations (people). Stimulus selection was based on previous studies establishing that images of erotica and mutilation compared with people are associated with pronounced differences across emotional response channels including measures of physiological arousal (e.g., skin conductance) and defensive reflex activity (Bradley et al., 2001), fMRI–blood oxygenation level-dependent signal changes in inferotemporal cortex and amygdala (Sabatinelli et al., 2005), and the ERP components, which are the subject matter in the present study (cf. Schupp et al., 2003, 2006b). According to normative ratings (Lang et al., 2005), mutilation pictures were rated as unpleasant compared with pictures of erotica and people (valence M: 1.7, 6.6, and 6.5, respectively). Furthermore, erotica and mutilation pictures were associated with strongly increased ratings of stimulus arousal compared with people pictures (arousal M: 6.4, 6.2, and 3.8, respectively). All images showed humans (alone or in small groups) and had simple figure/ground distinctions.

The experiment consisted of four separate conditions: participants were asked to either silently count the (1) erotic, (2) people, or (3) mutilation stimulus category or (4) simply view the stimuli without any explicit task instruction. The order of the four experimental conditions was balanced across participants. During the attention tasks, participants were asked at irregular intervals how many target pictures they have identified. Within each condition, the set of 30 pictures was repeated 20 times in a perceptually random order according to the valence of the stimuli. Recent studies determined the near absence of habituation in selective emotion processing indexed by the negative-going potential over posterior regions and the late positive potential suggesting that stimulus novelty is not critical for these emotional ERP modulations (Codispoti et al., 2006b; Schupp et al., 2006a).

To increase perceptual demands, pictures were shown as a continuous stream without perceivable interstimulus gaps and presentation times of 333 ms of each individual exemplar (rapid serial visual presentation).

ERP recordings and analysis

Brain and ocular scalp potential fields were measured with a 129 lead geodesic sensor net, on-line bandpass filtered from 0.01 to 100 Hz, and sampled at 250 Hz using Netstation acquisition software and Electrical Geodesics (EGI; Eugene, OR) amplifiers. Electrode impedance was kept below 30 kΩ, as recommended for this type of electroencephalogram (EEG) amplifier by EGI guidelines. Data were recorded continuously with the vertex sensor as the reference electrode. A 30 Hz digital low-pass filter was applied off-line to the continuous EEG data. Stimulus synchronized epochs lasting from 100 ms before until 800 ms after picture onset were extracted. Data editing and artifact rejection were based on an elaborate method for statistical control of artifacts, specifically tailored for the analyses of dense-sensor ERP recordings (Junghöfer et al., 2000). The reported data were baseline corrected and converted to an average reference. Finally, separate average waveforms were calculated for the six experimental cells (three picture categories by target and nontarget conditions, respectively) for each sensor and participant.

Early differential ERP activity: posterior negative potentials

In selective attention studies, the effect of task relevance associated with target compared with nontarget processing is best observed as differential ERP activity in a time interval between 150 and 350 ms (Hillyard and Anllo-Vento, 1998). Across widely differing overall ERP topographies, the signature of this early selective attention effect appeared invariably as negative difference potential over temporo-occipital sensor regions, also referred to as selection negativity or N2 (Smid et al., 1999; Potts and Tucker, 2001; Codispoti et al., 2006a). Rather than exploring the functional significance of selected ERP peaks, this approach seeks to determine the ERP difference associated with attention and task relevance while assuring that the same stimuli serve as the target and nontarget category (cf. Luck, 2005). A similar pattern of results has been observed in studies examining emotional stimuli. Emotionally arousing IAPS pictures as well as emotional facial expressions are associated with a negative difference potential in a time interval between 150 and 350 ms compared with neutral control stimuli, although the overall ERP wave appeared rather different across stimulus materials and experimental protocols (cf. Junghöfer et al., 2001; Schupp et al., 2004b, 2006b). Visual inspection of the ERP waveforms revealed the modulation of the ERP potential over temporo-occipital regions as a function of both attention and emotion (compare Fig. 1). For the purpose of statistical analysis, single-sensor waveform and conventional repeated-measures ANOVAs were calculated. In single-sensor waveform analyses, 2 (attention) × 3 (emotion) ANOVAs were calculated for each time point after picture onset separately for each individual sensor. To control for spurious findings in the waveform analysis, significant effects were considered meaningful only when the effects were observed for at least eight continuous data points (32 ms) and two neighboring sensors (cf. Schupp et al., 2003). Results revealed highly significant main effects of attention and emotion in the time interval from 200 to 350 ms and the expected regional topographies. However, no significant interaction of attention and emotion was observed within the first 350 ms after stimulus onset. To further characterize these effects, mean amplitudes from representative temporo-occipital sensor clusters identified by both visual inspection and waveform analyses were averaged over a time interval from 200 to 350 ms (EGI sensor numbers of the left cluster: 56, 63, 64, 65, 66, 69, 70, 71, 74, 75; of the right cluster: 83, 84, 85, 89, 90, 91, 95, 96, 100, 108). Because of volume conductor effects, task and emotion effects over occipito-temporal areas are observed with reversed polarity over anterior sites (cf. Schupp et al., 2003; Codispoti et al., 2006b). Exploring these centrofrontal sensor sites mirrored the effects observed for the temporo-occipital clusters. For brevity, this analysis will not be reported here.

Figure 1.

Figure 1.

Open in a new tab

Early differential ERP activity: posterior negative potentials. A, B, Grand-averaged ERP waveforms for a selected right occipital sensor (number 90), difference scalp potential maps, and L2-minimum-norm estimates (L2-MNE) serve to illustrate the main effects of attention (A) and emotion (B). C, ERP waveforms for sensor number 90 and difference scalp potential maps (target − nontarget) for erotica, people, and mutilation picture contents illustrate the additive relationship of attention and emotion. Paying attention to emotion had similar effects for high- and low-arousing stimulus contents. The maps display a right view.

Late differential ERP activity: the P3 wave

As expected, inspection of the ERP waveform revealed increased positive potentials ∼350–650 ms after stimulus onset for target (vs nontarget) and emotionally arousing (vs people) pictures, also referred to as P3b or late positive potential (compare Fig. 2). Again, the outcome of the single-sensor waveform and conventional ANOVAs were used in concert to secure statistical significance attributed to attention and emotion. Waveform analyses confirmed the significance of the increased positive potentials over centroparietal regions associated with target and emotion processing. Furthermore, these main effects were qualified by a significant interaction of attention and emotion. To further characterize these effects, mean amplitudes from representative temporo-occipital sensor clusters identified by both visual inspection and waveform analyses were averaged over a time interval from 400 to 600 ms (EGI sensor numbers of the left cluster: 7, 13, 31, 32, 38, 43, 53, 54, 61; of the right cluster: 79, 80, 81, 87, 88, 94, 106, 107, 113). The P3 effect occurred at a latency at which the target images were already replaced by the following stimuli. Thus, representing a possible confound, P3 effects may be attributed to subsequent picture processing. However, this interpretation appears unlikely for several reasons. Previous studies already obtained P3 waves with the expected topography and latency in paradigms using rapid visual presentations (Vogel et al., 1998; Kranczioch et al., 2003; Schupp et al., 2004a; Sergent et al., 2005). In addition, considering the target and emotion P3 effect observed in the present study, P3 appeared with similar polarity, topography, and latency as in studies without stimulus overlap (Schupp et al., 2003; Codispoti et al., 2006a). Furthermore, erotic, people, and mutilation pictures were presented in a perceptually randomized stimulus sequence equating the probability for the subsequent stimuli according to emotional valence. Thus, neither the target nor the emotion effect is likely contaminated by systematic differences in terms of the emotional valence of the subsequent pictures. Finally, because the emotion and target P3 effect developed at ∼300 ms and continued for several hundred milliseconds, the effect ascribed to the subsequent picture would have been apparent with short latency. Clearly, this hypothesis is not plausible considering a latency of ∼150–200 ms for target and emotion effects with these stimulus materials.

Figure 2.

Figure 2.

Open in a new tab

P3 effects. A, Grand-averaged ERP waveforms for a selected right parietal sensor (number 80) as a function of attention and emotion. B, Difference ERP waves (target − nontarget) collapsed across the left and right P3 sensor clusters for high-arousing erotica and mutilation and low-arousing people picture contents illustrate the synergistic relationship of attention and emotion. Attention effects were potentiated for erotica and mutilation target pictures. C, Display of the scalp potential difference (target − nontarget) separately for the three picture categories (top view). D, L2-minimum-norm estimates (right view) of the scalp potential difference separately for high-arousing emotional (erotica and mutilation) and low-arousing people picture categories.

For effects involving repeated measures, the Greenhouse–Geisser procedure was used to correct for violations of sphericity.

Results

Behavioral data

Overall, behavioral performance was good with an average of only 5.3% errors across the three picture categories. A repeated-measures ANOVA of the numbers of errors revealed significant differences across the three stimulus categories (F(2,30) = 7.4; p < 0.01). Post hoc tests revealed inferior performance when counting low-arousing people images (M, 7.9; SD, 4.7) compared with mutilation (M, 3.2; SD, 2.2) and erotic (M, 4.9; SD, 4.7) images, reaching significance only for the former comparison (F(1,15) = 14.0; p < 0.01).

ERP data

Early differential ERP activity: posterior negative potentials

The selective processing of target stimuli is usually referred to as negative difference potential over temporo-occipital sensor sites contrasting experimental conditions in which the same stimuli serve as target and nontarget stimuli (Hillyard and Anllo-Vento, 1998). As shown in Figure 1A, the attention effect appeared in the present study as broadly distributed negativity over posterior sensor regions in a time window from 200 to 350 ms. Furthermore, differential processing of high-arousing emotional compared with low-arousing images is captured as relative negativity over posterior regions (cf. Schupp et al., 2006b). Replicating these findings, the markedly pronounced emotion effect is illustrated in Figure 1B. Of most interest, Figure 1C suggests an additive relationship of attention and emotion (i.e., paying attention to emotional target stimuli has comparable effects as attending to neutral target stimuli).

To test for statistical significance, a repeated-measures ANOVA was calculated including factors of attention (target vs nontarget), emotion (erotica vs people vs mutilations), and laterality (left vs right). As expected, target stimulus processing was associated with an increased relative negativity compared with the processing of the same stimuli serving as nontarget items (ΔM = −0.74 μV; attention: F(1,15) = 49.2; p < 0.0001). Of most interest, this effect was not further qualified by higher-order interactions including attention and emotion (Fs < 1). The broadly distributed negative potential difference for target compared with nontarget stimuli was similarly pronounced for the three stimulus categories (erotica, ΔM = −0.82; people, ΔM = 0.62; mutilation, ΔM = 0.79 μV).

As shown in Figure 1B, the present data also replicated previous findings regarding the negative ERP difference over temporo-occipital regions associated with the processing of emotionally arousing picture contents (emotion: F(2,30) = 73.1; p < 0.0001). Post hoc tests revealed that erotica and mutilations were associated with an increased negativity (ΔM = −4.0 and −0.8 μV, respectively) relative to low-arousing people stimuli (ts(15) = 9.4 and 3.7, respectively; p < 0.0001). In addition, as in previous studies (Schupp et al., 2006a), the effect appeared more pronounced for erotica compared with mutilation pictures (t(15) = 8.4; p < 0.002).

Furthermore, no higher-order interactions of laterality involving attention or emotion were observed, whereas the potential was generally more negative over the right compared with the left posterior region (laterality: F(1,15) = 5.2; p < 0.05).

Additionally, L2-minimum-norm solutions were calculated to provide an estimate of the generator sources of the emotion and attention effects. The L2-minimum-norm estimate enables enhanced resolution of brain activations generating the scalp-recorded field potentials without a priori assumptions regarding the location and number of current sources (Hämäläinen and Ilmoniemi, 1994). Calculation of the L2-minimum-norm was based on a four-shell spherical head model with evenly distributed 3 (radial, azimuthal, and polar direction) × 350 dipoles as a source model. A shell radius of 8 cm was chosen as best tradeoff between depth sensitivity and spatial resolution (Hauk, 2004). As shown in Figure 1B, selective processing of erotica and mutilation contents was modeled by sources over occipito–temporal–parietal cortical regions. Interestingly, albeit less pronounced, similar posterior regions reflected effects of explicit attention to target pictures (Fig. 1A).

Late differential ERP activity: the P3 wave

A hallmark finding in the cognitive neuroscience of attention are increased P3 waves to target compared with nontarget stimuli over centroparietal sensor sites (Johnson, 1988). In addition, the processing of high-arousing emotional compared with low-arousing images is associated with enlarged positive potentials (Schupp et al., 2003). To show these effects, a representative right parietal sensor is displayed in Figure 2A contrasting target and nontarget processing in separate panels for the three stimulus categories. Thus, increased P3 amplitudes for target compared with nontarget pictures become readily visible, whereas the pronounced effect of emotional arousal is only observed by comparing the three panels. Interestingly, and in contrast to the early differential ERP activity, Figure 2, A–C, reveals an interaction of attention and emotion. Specifically, developing at ∼350 ms and lasting for several hundred milliseconds, the centroparietal P3 target effect was potentiated when emotionally arousing cues were the focus of attention.

A repeated-measures ANOVA including the factors of attention, emotion, and laterality served to test for statistical significance. As expected, the P3 wave was enlarged for target (M, 1.1 μV) compared with nontarget (M, −0.3 μV) pictures (attention: F(1,15) = 32.5; p < 0.0001). Interestingly, this effect was qualified by a significant interaction of emotion and attention (F(2,30) = 7.7; p < 0.01). Although separate post hoc tests revealed significantly enlarged P3 amplitudes for target compared with nontarget stimuli for low-arousing people stimuli (t(15) = 2.9, p < 0.02), the effect appeared more pronounced for high-arousing erotic images (t(15) = 6.8; p < 0.0001) and high-arousing mutilation contents (t(15) = 6.9; p < 0.0001). To statistically confirm that the P3 target effect was amplified for high-arousing emotional categories, the P3 difference between target and nontarget processing was contrasted for the three stimulus categories. The P3 target effect was significantly enlarged for erotica (ΔM = 1.5 μV) and mutilations (ΔM = 1.75 μV) compared with the people category (ΔM = 0.9 μV; ts(15) = 2.6 and 3.6, respectively; p < 0.02). Furthermore, the target P3 effect was comparable for high-arousing erotica and mutilation categories (t(15) = −1.2; NS).

As in previous studies, positive potentials varied for the three stimulus categories (emotion: F(2,30) = 62.5; p < 0.0001). Specifically, positive potentials were markedly enlarged for erotica (M, 0.9 μV) and mutilations (M, 0.6 μV) compared with low-arousing people images (M, −0.4 μV; ts(15) = 9.4 and 9.6, respectively; p < 0.0001). In addition, although representing a small effect in terms of amplitude difference (ΔM = 0.3 μV), positive potentials to pictures of erotica were more pronounced compared with mutilations (t(15) = 2.7; p < 0.02). Given the significant interaction of attention and emotion, separate post hoc tests confirmed that emotional compared with low-arousing control pictures were associated with increased positive potentials for target and nontarget processing (ts(15) > 5.6; p < 0.0001), whereas the difference between erotica and mutilation was significant only during nontarget picture processing (ΔM = 0.4 μV; t(15) = 5.0; p < 0.0001).

Finally, although the positive potential was more pronounced over the left compared with the right hemisphere (laterality: F(1,15) = 13.5; p < 0.01), no higher-order interactions of laterality involving attention or emotion were observed.

Additionally, as for the early ERP component, L2-minimum-norm solutions were calculated to provide an estimate of the generator sources of the P3 findings. To highlight the significant interaction of attention and emotion, Figure 2D illustrates the difference of target and nontarget processing separately for high-arousing emotional and low-arousing control pictures. As shown, occipital, parietal, and inferior temporal generator strength was more pronounced for high-arousing emotional target stimuli.

Interference effects

The interaction of attention and emotion has been explored in other research from the perspective of competition. Specifically, presenting emotional stimuli and nonemotional task stimuli concurrently, recent fMRI and ERP studies revealed a diminished capacity for selective emotion processing when performing demanding cognitive tasks (Vuilleumier et al., 2001; Pessoa et al., 2002a; Anderson et al., 2003; Holmes et al., 2003; Pourtois et al., 2004). The present study enabled a different test of interference among attention and emotion processes. Specifically, attending to emotional target stimuli in the rapid stream of pictures might interfere with the capacity to discriminate among high- and low-arousing nontarget stimuli. To explore this hypothesis, additional analyses compared the processing of erotic, mutilation, and people stimuli during the attention tasks (nontarget stimuli) with the processing of these images in the passive viewing condition. The 2 (condition) × 3 (emotion) × 2 (laterality) ANOVAs revealed no interactions involving condition for the early differential ERP activity, even when tailoring the area measure (200–300 ms) to capture the emotion differentiation (Fs < 1.6; p > 0.2). As shown in Table 1, rather than revealing interference, the negative-going ERP potential associated with selective emotion processing appeared somewhat more accentuated during the attention tasks compared with the passive viewing condition.

Table 1.

Analysis of interference effects comparing nontarget picture processing in the active task with passive viewing as the control condition

Erotica–people
 Mutilation–people
Control Task Control Task
Posterior negative potentials Δ−4.2 Δ−4.5 Δ−0.9 Δ−1.3
Late positive potential Δ 1.3 Δ 0.8 Δ 0.7 Δ 0.5
Open in a new tab

Data are difference scores (high- vs low-arousing pictures) in microvolts collapsed across left and right sensor clusters.

In contrast, analysis of the P3 wave revealed interference effects of selective emotion processing when performing the attention tasks compared with passive viewing (condition × emotion: F(2,30) = 4.4; p < 0.05). To follow up this interaction, difference scores of the positive potential were calculated between pictures high (erotica and mutilation) and low (people) in emotional arousal for active task (nontarget stimuli) and passive viewing conditions (Table 1). The 2 (condition) × 2 (erotica–people, mutilation–people) ANOVAs revealed the significantly attenuated emotional differentiation of the positive potential during the attention tasks compared with passive viewing (condition: F(1,15) = 7.4; p < 0.05). However, although attenuated in the active task condition, positive potentials were still significantly increased for erotica and mutilation compared with control pictures (ts(15) = 6.4 and 4.1, respectively; p < 0.001).

Discussion

Voluntary attention may often be directed to emotional stimuli given their importance for survival, reproduction, and procreation. To examine the effects of paying attention to emotion, attention was directed to stimuli categories varying in emotional intensity. Providing a millisecond time resolution of brain activity, ERP measurements revealed that the relationship between emotion and attention varied across different processing periods. Explicit attention and emotional significance effects operated additively during an early period of perceptual encoding at which stimuli are presumably selected for enhanced processing (∼200–350 ms after stimulus onset). Synergistic effects of implicit emotion and explicit attention emerged at a later processing stage (∼400–600 ms after stimulus onset) thought to reflect the process of stimulus evaluation. The critical comparison for the interaction of attention and emotion was based on the processing of the same stimuli serving as target and nontarget stimuli. Thus, the interaction effect cannot be driven by physical differences in the stimulus material. Together, the findings of the present study implicate to consider the interaction of attention and emotion at the level of specific cognitive subprocesses.

Selective attention effects brought out by either intrinsic emotional significance or explicit task relevance have been established previously in separate lines of research. In the present study, target stimuli were defined according to their emotional valence, and therefore both avenues to induce selective attention were assessed within the same experimental context. Notably, target stimulus processing was associated with a relative negativity over posterior regions and increased P3 potentials with similar topography and latency as in studies defining target stimuli based on higher-order semantic meaning (Codispoti et al., 2006b; Delorme et al., 2004). Furthermore, replicating previous findings (cf. Junghöfer et al., 2001; Schupp et al., 2004a, 2006a), picture contents high in emotional arousal elicited a negative-going temporo-occipital potential and increased positive potentials compared with low-arousing picture contents, and this effect appeared most pronounced for erotic stimuli. Establishing these cortical indices of selective attention attributable to implicit emotional and explicit task-defined significance provided the foundation to meaningfully interpret the observed interplay of attention and emotion.

The main finding of the present study was that attention effects were potentiated for emotionally arousing target stimuli in a later stage of processing indexed by the P3. This synergistic effect of the two facets of motivational significance, intrinsic emotion and explicit task relevance, appeared sizable. Contrasting target and nontarget processing, the P3 target effect almost doubled when attending to erotic and mutilation compared with low-arousing control pictures. Acknowledging limitations in spatial resolution of source analysis, results suggest that the P3 wave elicited by visual cues represents multiple cortical generator structures in occipital, inferior temporal, temporoparietal, and lateral prefrontal regions (Sergent et al., 2005). The L2-minimum-norm analyses of the target P3 concur with the finding of widespread neural generator sources in multiple cortical association regions. Interestingly, although the pattern of generator sources appeared similar across target categories, high-arousing emotional cues were associated with increased generator strength, in particular over inferior temporal, parietal, and occipital regions. Overall, selective attention to emotionally significant cues potentiated the P3 target effect compared with control stimuli, presumably reflecting a later processing stage involving increased activity in distributed networks of cortical association areas (cf. Sabatinelli et al., 2006). Future studies need to delineate whether these findings generalize to other emotional stimuli including the study of discrete emotion categories (i.e., fearful and threatening facial expressions) (cf. Öhman and Mineka, 2001).

In addition to motivational significance, the P3 amplitude is also sensitive to stimulus probability and rate of target occurrence (Courchesne et al., 1977; Croft et al., 2003). Although the P3 amplitude might be generally reduced in rapid serial presentation paradigms, stimulus probability appears unlikely to account for the differential P3 findings of this study. Regarding emotion modulation, the three stimulus categories had the same probability of occurrence. Furthermore, the finding that the P3 varied with emotional arousal was opposite to a stimulus probability account (i.e., high-arousing stimuli were twice as frequent as low-arousing images) (cf. Cuthbert et al., 2000). With regard to task relevance, target stimuli had a lower probability than nontarget stimuli, presumably enhancing the P3 target effect compared with a design equating the target/nontarget ratio (Johnson, 1988). Importantly, the significant interaction of emotion and attention is not comprised by stimulus probability because the target/nontarget ratio was the same for counting erotica, mutilations, and neutral contents. Accordingly, the synergistic P3 effect most likely reflects the coincidence of two facets of motivational significance (i.e., paying selective attention to highly arousing emotional stimuli). However, it might be instructive to determine in future studies to what extent the observed findings are sensitive to variations in stimulus probability and presentation parameters (Luck and Hillyard, 2000).

Considering the functional significance of the P3 findings, several lines of research and theories are relevant (Donchin and Coles, 1988; Johnson, 1988; Nieuwenhuis et al., 2005). Many studies suggest that the P3 reflects the operation of a capacity-limited processing system. For instance, dual-task studies revealed a reciprocal relationship of the P3 amplitudes evoked by two concurrent tasks (Wickens et al., 1983). In addition, signal detection and recognition studies provide additional evidence linking the P3 amplitude to quality and accuracy of stimulus processing. Specifically, a linear relationship was observed between P3 amplitude, and the participants' felt a degree of confidence in their perceptual decision (Hillyard et al., 1971). More recently, attentional blink studies demonstrate that the P3 amplitude systematically varied with conscious recognition (Vogel et al., 1998). Specifically, seen target stimuli (presented in the attentional blink interval) elicited a P3 wave, which was absent for unseen target stimuli (Kranczioch et al., 2003; Sergent et al., 2005). Considered from the perspective of two stage models of stimulus processing (Chun and Potter, 1995), enlarged P3 amplitudes to motivationally significant stimuli may reflect enhanced stimulus consolidation in capacity-limited second stage processing related to conscious recognition and elaborate processing of significant stimuli (Öhman, 1986; Sergent et al., 2005; Schupp et al., 2006b). However, enhanced memory processes may be brought out by heightened selective attention, and it has been suggested that the P3 amplitude reflects increased attentional resources devoted to motivationally significant stimuli “… such that the probability is increased that the detection of incoming sensory signals is appropriately acted on via selective response facilitation” (Nieuwenhuis et al., 2005, p 522). Overall, target P3 amplitudes were potentiated when attending to emotionally arousing stimuli, presumably reflecting the engagement of a capacity-limited processing system associated with conscious recognition, and the preparation and organization of efficient behavioral responses.

Preceding the P3, a transitory ERP component is also sensitive to motivational significance (i.e., emotional significance and task-defined attention). Explicit attention studies revealed that the onset of the occipital negativity effect provides a high-resolution measure of the time at which stimuli possessing target features are discriminated and selectively processed (Hillyard and Anllo-Vento, 1998). Interestingly, an increased occipital negativity has been shown in attentional blink studies to discriminate among seen and unseen target stimuli (Sergent et al., 2005). According to two stage models of stimulus perception (Chun and Potter, 1995; Öhman, 1986), mandatory processes of stimulus identification and detection (stage 1) may be followed by a transitory processing period at which motivationally significant stimuli are being “tagged” for preferential processing. Furthermore, rather than reflecting the enhancement of obligatory ERP peaks as observed in spatial attention studies, the negative ERP difference appears as endogenous brain potential modulation (i.e., additional brain activity) with different scalp topographies for different types of feature selections (Hillyard et al., 1995; Hillyard and Anllo-Vento, 1998). L2-minimum-norm source analyses of the target and emotion effect are consistent with this hypothesis revealing a similar pattern of posterior neural activity. Thus, the negative-going potential over posterior regions may reflect facilitated processing of significant stimuli in perceptual representation regions intermediate to later processing stages involving distributed networks of cortical association areas (Potts and Tucker, 2001; Schupp et al., 2003; Sergent et al., 2005). Interestingly, at this transitory stage of processing, task and emotional relevance operated additively but independent from each other.

Previous studies assessed the interference of emotion processing by introducing a competing spatial attention task (Pessoa et al., 2002b). The present study entailed a different test of interference by comparing the selective emotion processing of nontarget stimuli with the passive viewing condition. Interference effects were limited to the P3 showing a complementary pattern of findings as observed for the case of cooperation of emotion and attention. Specifically, while attending to one class of emotional targets, the positive potential difference between high- and low-arousing nontarget stimuli was attenuated compared with the viewing condition. More recent studies revealed an attenuation of selective emotion processing indexed by the differential posterior negativity using a competing feature-based attention task (Schupp et al., 2006b). Thus, diverting attentional resources by nonspatial attention tasks seem to comprise the selective processing of emotional stimuli providing complementary evidence to spatial attention studies. Overall, accumulating empirical evidence suggests to detail the interaction of attention and emotion (i.e., at the level of specific cognitive subprocesses and with regard to different attentional mechanisms) (LeDoux, 2000; Luck and Hillyard, 2000).

Footnotes

This work was supported by the German Research Foundation.

References

  1. Anderson AK, Christoff K, Panitz D, De Rosa E, Gabrieli JD. Neural correlates of the automatic processing of threat facial signals. J Neurosci. 2003;23:5627–5633. doi: 10.1523/JNEUROSCI.23-13-05627.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bradley MM, Codispoti M, Cuthbert BN, Lang PJ. Emotion and motivation I: defensive and appetitive reactions in picture processing. Emotion. 2001;1:276–299. [PubMed] [Google Scholar]
  3. Bradley MM, Sabatinelli D, Lang PJ, Fitzsimmons JR, King W, Desai P. Activation of the visual cortex in motivated attention. Behav Neurosci. 2003;117:369–380. doi: 10.1037/0735-7044.117.2.369. [DOI] [PubMed] [Google Scholar]
  4. Chun MM, Potter MC. A two-stage model for multiple target detection in rapid serial visual presentation. J Exp Psychol Hum Percept Perform. 1995;21:109–127. doi: 10.1037//0096-1523.21.1.109. [DOI] [PubMed] [Google Scholar]
  5. Codispoti M, Ferrari V, Junghöfer M, Schupp HT. The categorization of natural scenes: brain attention networks revealed by dense sensor ERPs. NeuroImage. 2006a;32:583–591. doi: 10.1016/j.neuroimage.2006.04.180. [DOI] [PubMed] [Google Scholar]
  6. Codispoti M, Ferrari V, Bradley MM. Repetitive picture processing: autonomic and cortical correlates. Brain Res. 2006b;1068:213–220. doi: 10.1016/j.brainres.2005.11.009. [DOI] [PubMed] [Google Scholar]
  7. Courchesne E, Hillyard SA, Courchesne RY. P3 waves to the discrimination of targets in homogeneous and heterogeneous stimulus sequences. Psychophysiology. 1977;14:590–597. doi: 10.1111/j.1469-8986.1977.tb01206.x. [DOI] [PubMed] [Google Scholar]
  8. Croft RJ, Gonsalvez CJ, Gabriel C, Barry RJ. Target-to-target interval versus probability effects on P300 in one- and two-tone tasks. Psychophysiology. 2003;40:322–328. doi: 10.1111/1469-8986.00036. [DOI] [PubMed] [Google Scholar]
  9. Cuthbert BN, Schupp HT, Bradley MM, Birbaumer N, Lang PJ. Brain potentials in affective picture processing: covariation with autonomic arousal and affective report. Biol Psychol. 2000;52:95–111. doi: 10.1016/s0301-0511(99)00044-7. [DOI] [PubMed] [Google Scholar]
  10. Delorme A, Rousselet GA, Mace MJ, Fabre-Thorpe M. Interaction of top-down and bottom-up processing in the fast visual analysis of natural scenes. Cogn Brain Res. 2004;19:103–113. doi: 10.1016/j.cogbrainres.2003.11.010. [DOI] [PubMed] [Google Scholar]
  11. Donchin E, Coles MG. Is the P300 component a manifestation of context updating? Behav Brain Sci. 1988;11:357–427. [Google Scholar]
  12. Hämäläinen MS, Ilmoniemi RJ. Interpreting magnetic fields of the brain: minimum norm estimates. Med Biol Eng Comput. 1994;32:35–42. doi: 10.1007/BF02512476. [DOI] [PubMed] [Google Scholar]
  13. Hauk O. Keep it simple: a case for using classical minimum norm estimation in the analysis of EEG and MEG data. NeuroImage. 2004;21:1612–1621. doi: 10.1016/j.neuroimage.2003.12.018. [DOI] [PubMed] [Google Scholar]
  14. Hillyard SA, Anllo-Vento L. Event-related brain potentials in the study of visual selective attention. Proc Natl Acad Sci USA. 1998;95:781–787. doi: 10.1073/pnas.95.3.781. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Hillyard SA, Squires K, Bauer J, Lindsay P. Evoked potential correlates of auditory signal detection. Science. 1971;172:1357–1360. doi: 10.1126/science.172.3990.1357. [DOI] [PubMed] [Google Scholar]
  16. Hillyard SA, Mangun GR, Woldorff MG. Neural systems mediating selective attention. In: Gazzaniga MS, editor. Cognitive neurosciences. Cambridge, MA: MIT; 1995. pp. 665–681. [Google Scholar]
  17. Holmes A, Vuilleumier P, Eimer M. The processing of emotional facial expression is gated by spatial attention: evidence from event-related brain potentials. Cogn Brain Res. 2003;16:174–184. doi: 10.1016/s0926-6410(02)00268-9. [DOI] [PubMed] [Google Scholar]
  18. Johnson R. The amplitude of the P300 component of the event-related potential: review and synthesis. In: Ackles PK, Jennings JR, Coles MGH, editors. Advances in psychophysiology. Vol 3. Greenwich, CT: JAI; 1988. pp. 69–138. [Google Scholar]
  19. Junghöfer M, Elbert T, Tucker DM, Rockstroh B. Statistical control of artifacts in dense array EEG/MEG studies. Psychophysiology. 2000;37:523–532. [PubMed] [Google Scholar]
  20. Junghöfer M, Bradley MM, Elbert TR, Lang PJ. Fleeting images: a new look at early emotion discrimination. Psychophysiology. 2001;38:175–178. [PubMed] [Google Scholar]
  21. Junghöfer M, Schupp HT, Stark R, Vaitl D. Neuroimaging of emotion: empirical effects of proportional global signal scaling in fMRI data analysis. NeuroImage. 2005;25:520–526. doi: 10.1016/j.neuroimage.2004.12.011. [DOI] [PubMed] [Google Scholar]
  22. Junghöfer M, Sabatinelli D, Bradley MM, Schupp HT, Elbert TR, Lang PJ. Fleeting images: rapid affect discrimination in the visual cortex. NeuroReport. 2006;17:225–229. doi: 10.1097/01.wnr.0000198437.59883.bb. [DOI] [PubMed] [Google Scholar]
  23. Kastner S, Ungerleider LG. Mechanisms of visual attention in the human cortex. Annu Rev Neurosci. 2000;23:315–341. doi: 10.1146/annurev.neuro.23.1.315. [DOI] [PubMed] [Google Scholar]
  24. Keil A, Bradley MM, Hauk O, Rockstroh B, Elbert T, Lang PJ. Large-scale neural correlates of affective picture processing. Psychophysiology. 2002;39:641–649. doi: 10.1017.S0048577202394162. [DOI] [PubMed] [Google Scholar]
  25. Keil A, Moratti S, Sabatinelli D, Bradley MM, Lang PJ. Additive effects of emotional content and spatial selective attention on electrocortical facilitation. Cereb Cortex. 2004;15:1187–1197. doi: 10.1093/cercor/bhi001. [DOI] [PubMed] [Google Scholar]
  26. Kranczioch C, Debener S, Engel AK. Event-related potential correlates of the attentional blink phenomenon. Brain Res Cogn Brain Res. 2003;17:177–187. doi: 10.1016/s0926-6410(03)00092-2. [DOI] [PubMed] [Google Scholar]
  27. Lang PJ, Bradley MM, Cuthbert BN. Technical Report A-6. Gainesville, FL.: University of Florida; 2005. International affective picture system (IAPS): digitized photographs, instruction manual and affective ratings. [Google Scholar]
  28. LeDoux J. Cognitive-emotional interactions: listen to the brain. In: Lane RD, Nadel L, Ahern G, editors. Cognitive neuroscience of emotion. New York: Oxford UP; 2000. pp. 129–155. [Google Scholar]
  29. Luck SJ. An introduction to the event-related potential technique. Cambridge, MA: MIT; 2005. [Google Scholar]
  30. Luck SJ, Hillyard S. The operation of selective attention at multiple stages of processing: evidence from human and monkey electrophysiology. In: Gazzaniga MS, editor. The new cognitive neurosciences. Cambridge, MA: MIT; 2000. pp. 687–710. [Google Scholar]
  31. Nieuwenhuis S, Aston-Jones G, Cohen JD. Decision making, the P3, and the locus coeruleus-norepinephrine system. Psychol Bull. 2005;131:510–532. doi: 10.1037/0033-2909.131.4.510. [DOI] [PubMed] [Google Scholar]
  32. Öhman A. Face the beast and fear the face: animal and social fears as prototypes for evolutionary analyses of emotion. Psychophysiology. 1986;23:123–145. doi: 10.1111/j.1469-8986.1986.tb00608.x. [DOI] [PubMed] [Google Scholar]
  33. Öhman A, Mineka S. Fears, phobias, and prepardness: toward an evolved module of fear and fear learning. Psychol Rev. 2001;108:483–522. doi: 10.1037/0033-295x.108.3.483. [DOI] [PubMed] [Google Scholar]
  34. Öhman A, Flykt A, Lundqvist D. Unconscious emotion: evolutionary perspectives, psychophysiological data and neuropsychological mechanisms. In: Lane RD, Nadel L, editors. Cognitive neuroscience of emotion. New York: Oxford UP; 2000. pp. 296–327. [Google Scholar]
  35. Pessoa L, McKenna M, Gutierrez E, Ungerleider LG. Neural processing of emotional faces requires attention. Proc Natl Acad Sci USA. 2002a;99:11458–11463. doi: 10.1073/pnas.172403899. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Pessoa L, Kastner S, Ungerleider LG. Attentional control of the processing of neutral and emotional stimuli. Brain Res Cogn Brain Res. 2002b;15:31–45. doi: 10.1016/s0926-6410(02)00214-8. [DOI] [PubMed] [Google Scholar]
  37. Phelps EA, Ling S, Carrasco M. Emotion facilitates perception and potentiates the perceptual benefits of attention. Psychol Sci. 2006;17:292–299. doi: 10.1111/j.1467-9280.2006.01701.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Potts GF, Tucker DM. Frontal evaluation and posterior representation in target detection. Cogn Brain Res. 2001;11:147–156. doi: 10.1016/s0926-6410(00)00075-6. [DOI] [PubMed] [Google Scholar]
  39. Pourtois G, Grandjean D, Sander D, Vuilleumier P. Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cereb Cortex. 2004;14:619–633. doi: 10.1093/cercor/bhh023. [DOI] [PubMed] [Google Scholar]
  40. Sabatinelli D, Bradley MM, Fitzsimmons JR, Lang PJ. Parallel amygdala and inferotemporal activation reflect emotional intensity and fear relevance. NeuroImage. 2005;24:1265–1270. doi: 10.1016/j.neuroimage.2004.12.015. [DOI] [PubMed] [Google Scholar]
  41. Sabatinelli D, Lang PJ, Keil A, Bradley MM. Emotional Perception: Correlation of Functional MRI and Event-Related Potentials. Cereb Cortex. 2006 doi: 10.1093/cercor/bhl017. in press. [DOI] [PubMed] [Google Scholar]
  42. Schupp HT, Junghöfer M, Weike AI, Hamm AO. Emotional facilitation of sensory processing in the visual cortex. Psychol Sci. 2003;14:7–13. doi: 10.1111/1467-9280.01411. [DOI] [PubMed] [Google Scholar]
  43. Schupp HT, Junghöfer M, Weike AI, Hamm AO. The selective processing of briefly presented affective pictures: an ERP analysis. Psychophysiology. 2004a;41:441–449. doi: 10.1111/j.1469-8986.2004.00174.x. [DOI] [PubMed] [Google Scholar]
  44. Schupp HT, Öhman A, Junghöfer M, Weike AI, Stockburger J, Hamm AO. The facilitated processing of threatening faces: an ERP analysis. Emotion. 2004b;4:189–200. doi: 10.1037/1528-3542.4.2.189. [DOI] [PubMed] [Google Scholar]
  45. Schupp HT, Stockburger J, Codispoti M, Junghöfer M, Weike AI, Hamm AO. Stimulus novelty and emotion perception: the near absence of habituation in the visual cortex. NeuroReport. 2006a;17:365–369. doi: 10.1097/01.wnr.0000203355.88061.c6. [DOI] [PubMed] [Google Scholar]
  46. Schupp HT, Flaisch T, Stockburger J, Junghöfer M. Emotion and attention: event-related brain potential studies. Prog Brain Res. 2006b;156:31–51. doi: 10.1016/S0079-6123(06)56002-9. [DOI] [PubMed] [Google Scholar]
  47. Sergent C, Baillet S, Dehaene S. Timing of the brain events underlying access to consciousness during the attentional blink. Nat Neurosci. 2005;8:1391–1400. doi: 10.1038/nn1549. [DOI] [PubMed] [Google Scholar]
  48. Smid HG, Jakob A, Heinze HJ. An event-related brain potential study of visual selective attention to conjunctions of color and shape. Psychophysiology. 1999;36:264–279. doi: 10.1017/s0048577299971135. [DOI] [PubMed] [Google Scholar]
  49. Vogel EK, Luck SJ, Shapiro KL. Electrophysiological evidence for a postperceptual locus of suppression during the attentional blink. J Exp Psychol Hum Percept Perform. 1998;24:1656–1674. doi: 10.1037//0096-1523.24.6.1656. [DOI] [PubMed] [Google Scholar]
  50. Vuilleumier P. How brains beware: neural mechanisms of emotional attention. Trends Cogn Sci. 2005;9:585–594. doi: 10.1016/j.tics.2005.10.011. [DOI] [PubMed] [Google Scholar]
  51. Vuilleumier P, Armony JL, Driver J, Dolan RJ. Effects of attention and emotion on face processing in the human brain: an event-related fMRI study. Neuron. 2001;30:829–841. doi: 10.1016/s0896-6273(01)00328-2. [DOI] [PubMed] [Google Scholar]
  52. Wickens C, Kramer A, Vanasse L, Donchin E. Performance of concurrent tasks: A psychophysiological analysis of the reciprocity of information-processing resources. Science. 1983;221:1080–1082. doi: 10.1126/science.6879207. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES