Slow biasing of processing resources in early visual cortex is preceded by emotional cue extraction in emotion–attention competition

Abstract

In our previous studies on competition for attentional processing resources in early visual cortex between a foreground task and distracting emotional background images we found that emotional background images withdraw attentional resources from the foreground task after about 400 ms. Costs in behavioral data and a significant reduction of the steady state visual evoked potential (SSVEP) amplitude that was elicited by the foreground task lasted for several hundred milliseconds. We speculated that the differential effect in SSVEP amplitudes is preceded by the extraction of the emotional cue. Event related potential (ERP) studies to emotional and neutral complex images identified an early posterior negativity (EPN) as a robust neural signature of emotional cue extraction. The late positive potential (LPP) was related to in‐depth processing of the emotional image. We extracted ERPs that were evoked by the onset of background images concurrently with the SSVEP that was elicited by the foreground task. Emotional compared to neutral background pictures evoked a more negative EPN at about 190 ms and a more positive LPP at about 700 ms after image onset. SSVEP amplitudes became significantly smaller with emotional background images after about 400 ms lasting for several hundred ms. Interestingly, we found no significant correlations between the three components, indicating that they act independently. Source localizations resulted in nonoverlapping cortical generators. Results suggest a cascade of perceptual processes: Extraction of the emotional cue preceded biasing of attentional resources away from the foreground task towards the emotional image for an evaluation of the picture content. Hum Brain Mapp 35:1477–1490, 2014. © 2013 Wiley Periodicals, Inc.

INTRODUCTION

In everyday life, we are confronted with complex environmental settings that require highly adaptive interactions. Pivotal for adaptive behavior is to extract and process relevant sensory information while one has to ignore all other information that is not relevant for the task at the given moment. A basic assumption of all attention accounts is that sensory processing is linked to a strictly limited resource, which must be allocated by selective attention [Kahneman et al., 1983]. The postulate of limited resources consequences that neuronal processing of multiple stimuli requires competition for a limited pool of processing resources [cf., Desimone and Duncan, 1995; Duncan et al., 1997; Kastner and Ungerleider, 2000]. This competition among stimuli for neural representation can be biased, both by bottom‐up sensory driven mechanisms such as stimulus salience or attentional top‐down signals [Desimone and Duncan, 1995]. Emotional stimuli relative to other visual stimuli are assumed to have competitive advantages due to their intrinsic stimulus significance that are related to a bottom‐up driven bias of attentional resources [Lang et al., 1997b; Öhman et al., 2001; Vuilleumier and Driver, 2007; Bradley, 2009; Lang and Bradley, 2010].

In three previous studies [Müller et al., 2008; Hindi Attar et al., 2010a; Müller et al., 2011], we investigated the time course of competitive interactions for attentional resources in early visual cortex between a foreground task and emotional background pictures from the International Affective Picture Set [IAPS, Lang et al., 1997a] that served as distractors. In our competition paradigm, we present flickering squares at a certain frequency eliciting the steady state visual evoked potential (SSVEP). In electroencephalographic (EEG) recordings, the SSVEP is a continuous oscillatory brain response with the same frequency as the driving stimulus [Regan, 1989] having its main generators in early visual areas [Hillyard et al., 1997; Müller et al., 1998a; Müller et al., 2006; Di Russo et al., 2007; Andersen et al., 2008; Andersen and Müller, 2010]. Most importantly, the amplitude of the SSVEP increases significantly when a certain stimulus was attended compared to when that stimulus had to be ignored [cf., Morgan et al., 1996; Müller et al., 1998a; Müller and Hübner, 2002; Müller et al., 2003; Andersen et al., 2008]. Thus, frequency‐tagging of stimuli has been proven to be a highly effective method to investigate the distribution of attentional resources in multielement stimulus displays [for using flickering IAPS images to investigate attentional resource allocation to emotional compared to neutral images see Keil et al., 2003, 2005, 2009; Hajcak et al., 2013]. Furthermore, given that the SSVEP is a continuous neural response, it also serves as a powerful tool to investigate temporal neural dynamics of the deployment of attentional resources to a particular stimulus within such multielement stimulus displays [Müller et al., 1998b; Müller, 2008; Andersen and Müller, 2010]. Although these studies used endogenous cues to measure the time course of neural facilitation of a certain stimulus in early visual cortex in voluntary attention, the present distraction paradigm makes use of the proposed stimulus driven (bottom‐up) influence of emotional stimuli in biasing attentional resources (see above).

In our previous studies on the time course of emotion–attention competition in early visual cortex, subjects were instructed to attend to the flickering squares to detect and to respond to rare coherent motion events. A consistent finding was, that the presentation of an emotional compared to a neutral IAPS image in the background resulted in a significantly greater decrease in SSVEP amplitude that started about 400 ms after the presentation of the background images. That decrease of SSVEP amplitude was seen as a consequence of the biasing of attentional resources away from the flickering stimuli that served as foreground task towards the emotional background images what was also mirrored by a significant reduction in hit rates [Müller et al., 2008; Hindi Attar et al., 2010a]. Interestingly, the onset of differential SSVEP amplitude reduction as a function of stimulus valence is somewhat in the middle between what we found for spatial [Müller et al., 1998b] and feature‐based [Andersen and Müller, 2010] SSVEP amplitude modulations with top‐down guidance of attention. The question emerged whether this was due to the fact that the images remained visible for several seconds, or whether this delay is something like an “inherent” shifting time for biasing attentional resources in early visual cortex.

To find an answer to that question in a subsequent study [Müller et al., 2011], background images were presented for 133 ms only, followed by a mask. That short presentation time was chosen to allow for one fixation [Christianson et al., 1991] but presentation time was sufficient for emotional content categorization [Thorpe et al., 1996; Schupp et al., 2004]. In contrast to the first two studies in which the background image remained visible for several seconds, brief presentation and masking is more suitable for assessing the immediate affective impact of a stimulus without further elaborative and (emotional) regulating processes inherent to longer presentation times [Larson et al., 2005]. The short image presentation resulted in a similar latency of the reduction in SSVEP amplitude with emotional background images. Importantly, the onset of the competitive interactions in early visual cortex started about 275 ms after the image was already masked and no longer visible on the screen. Obviously, it seems to be the case that once this shifting mechanism is triggered it seems to continue even when the emotional distractor is no longer visible. The relatively constant latency of SSVEP amplitude reduction with emotional IAPS background distractors across our previous studies gave rise to the speculation that competitive interactions for processing resources in early visual areas—as measured with SSVEP amplitude reductions—are (necessarily) preceded by the extraction of the emotional content.

One way to test that greater SSVEP amplitude reductions with emotional compared to neutral background images follow emotional content extraction is to concurrently analyze the visual evoked potential (VEP) that is elicited by the onset of the respective background image and the time course of SSVEP amplitudes related to the flickering dots that serve as the foreground task [Müller and Hillyard, 2000]. In previous studies that extracted the VEP to emotional and neutral IAPS images, two components were consistently reported [Schupp et al., 2000, 2004; Bradley et al., 2007; Flaisch et al., 2008; Sabatinelli et al., 2013]. First, an early posterior negativity (EPN) that occurs between 150 and 300 ms after picture onset, with its maximum amplitude at occipitotemporal electrodes. The EPN exhibits a more negative deflection for emotional compared to neutral IAPS pictures. Schupp et al. 2004 identified the EPN as the first neuronal activity that signifies the extraction of an emotional cue in complex IAPS pictures. However, some other studies reported of much earlier VEP modulations. In the context of emotional conditioning the conditioned stimulus modulated the C1 component that has its generator in primary visual cortex (V1), Such effects were reported for conditioning using emotional IAPS images as unconditioned stimuli and gratings as conditioned stimuli [Stolarova et al., 2006] or odors and images of faces [Steinberg et al., 2012], just to mention two examples. However, conditioning seems to be a highly specific case that is different from watching IAPS images without preconditioning. In experiments that investigated the VEP to IAPS image onset, the earliest modulation as a function of valence was the P1 component with a latency of about 120 ms [cf., Smith et al., 2003; Carretie et al., 2004).1 The majority of studies, however, reported of a modulation of the N1 component with a latency of about 160 ms [for a review see Olofsson et al., 2008; Pourtois et al., 2013]. A closer inspection of the N1 findings reveals that results are not entirely homogeneous. Although Keil et al. [2002, 2001] found N1 modulation for emotional compared to neutral IAPS images in one study, in a subsequent study they reported N1 modulation for pleasant images only. Just recently, Weinberg and Hajcak 2010 reported of an N1 modulation at a latency of 100 ms after picture onset for emotional compared to neutral IAPS images. Although the modulation mirrors the earlier finding by Keil et al. 2001, the N1 in the Weinberg and Hajcak study was measured much earlier compared to the studies by Keil et al. [2002, 2001] who reported a peak latency of about 180 and 135 ms. Interestingly, Weinberg and Hajcak found no differences in N1 amplitudes when they compared for picture content within a valence category. In contrast, the EPN in the time window between 200 and 280 ms showed such a pattern, that is, erotic images compared to other pleasant images without erotic content, or mutilation and threat for unpleasant images. As of today, it appears as if the EPN is a more robust neural indicator of emotional cue extraction what motivated us to focus on the EPN in the present study. These more robust findings with regard to the EPN might be due to the fact that the EPN is mostly measured as a difference wave with the possible effect of integrating modulations of a number of individual VEP components such as the N1, P2, N2, and even the slope of the P3. Against that argument of a component integration stands the morphology of the EPN when depicted for individual conditions. In general, the EPN is not exhibiting pronounced peaks and troughs as indicators of other VEP components [for examples see Schupp et al., 2004; Sabatinelli et al., 2013].

The second component of interest is a late positive potential (LPP) that usually starts at about 400 ms after picture onset and can last for several hundred milliseconds when the stimulus remains on the screen [cf., Schupp et al., 2000; Codispoti et al., 2007; Flaisch et al., 2008; Bradley, 2009; Sabatinelli et al., 2013]. The LPP is a positive deflection at centroparietal electrodes with greater positivity for emotional compared to neutral images. In general the LPP is seen to signify in‐depth processing of the complex IAPS images, that is, (sustained) allocation of attentional resources for a better evaluation of emotional scenes [cf., Schupp et al., 2000; Sabatinelli et al., 2013]. The majority of VEP studies were interested in the LPP. These studies showed that the LPP is sensitive to top‐down modulated spatial attention [Dunning and Hajcak, 2009], reappraisal techniques [Moser et al., 2009], and habituates to a certain extend [Codispoti et al., 2007]. Compared to the LPP, the EPN is less affected by such manipulations. In a habituation paradigm, in which emotional and neutral pictures were repeated up to 90 times, Codispoti et al. 2007 found an emotional habituation effect for the LPP but not for the EPN. Given not only the different latencies between the EPN and LPP but also their different behaviors with respect to a number of manipulations, it is quite obvious that these two components represent different processing stages of complex IAPS images as stated above. Just recently, Sabatinelli et al. [2013] performed a study to link EPN and LPP amplitudes with BOLD responses from two independent subject groups. For the EPN they reported significant correlations with BOLD activity in anterior cingulate (ACC) and the amygdala only, whereas the LPP showed a much broader correlation pattern to a number of cortical (such as occipital, intraparietal, and inferotemporal cortex) and corticolimbic regions‐of‐interest (ROI; such as insula, anterior cingulated, nucleus accumbens, and amygdala). The authors concluded that the LPP indeed seems to reflect sustained processing of emotional content as suggested in a number of other studies that were cited above. Given only weak correlations between the EPN and ROIs in early visual, occipital and inferotemoral cortex, the unequivocal link of the EPN as a signature of emotional cue extraction was not as clear cut as the functional significance of the LPP. However, one reason that might explain such weak correlations is the fact that the EPN has on average a very short duration of about 150 ms what is quite different to the long‐live LPP und, thus, a link to BOLD activation is more difficult to obtain.

This study was motivated to test whether the EPN precedes the SSVEP modulation and to what extend the onset of the LPP is temporarily linked to the time‐course of SSVEP differences. Furthermore, we correlated the amplitudes of the difference values (unpleasant minus neutral background image) of the EPN, LPP, and SSVEP to see whether they are dependent upon each other in terms of amplitude modulation effects. In our previous studies such a concurrent analysis was not possible due to the facts that (a) the squares flickered at a frequency of 7.5 Hz what resulted in a superimposition of the VEP and SSVEP that does not allow to filter the one response out of the other, and (b) in particular for the short presentation study, we used a fading‐in and fading‐out procedure to avoid an onset VEP. To achieve our goals, we increased the flicker rate to 15 Hz allowing us to filter the SSVEP out of the VEP and vice versa [Müller and Hillyard, 2000; Andersen et al., 2011]. To increase contrast between flickering squares and background images to receive an SSVEP with high amplitude, we presented the images as black and white versions what has been shown in previous studies to elicit the same components in the VEP compared to their colored versions [Bradley et al., 2007, 2003; Codispoti et al., 2012]. Second, to evoke a VEP to the onset of the background image, pictures were presented with a sudden onset, synchronized to the 15 Hz cycle of flickering squares (see “Methods” Section). We expected that the onset of SSVEP amplitude modulation as a function of valence of IAPS background images follows the peak of the EPN evoked by the background images. Furthermore, the onset of the LPP was expected to fall within the time window of ongoing SSVEP differences as a reciprocal measure of attentional resource allocation between the instruction to attend to the flickering squares and the in‐depth processing of emotional content of unpleasant background images.

METHODS

Participants

Twenty subjects (13 female) with a mean age of 23.85 years (SD = 5.02 years; range: 18–39 years) participated in the experiment. They had normal or corrected‐to‐normal visual acuity. All participants received class credits or money as compensation and gave written informed consent at the beginning of the experiment. The experiment was conducted in compliance with the Code of Ethics of the World Medical Association (Declaration of Helsinki) and the local ethic committee (University of Leipzig).

Stimuli and Procedure

All stimuli were presented on a 19‐inch computer monitor set to a refresh rate of 60 Hz. Distance between observers and monitor was kept constant at 80 cm. A fixation cross was presented in the center of the screen throughout each trial. Task‐relevant stimuli were 100 moving red squares (each dot 0.6° × 0.6° of visual angle), which were randomly allocated superimposed upon a neutral or unpleasant background picture or a scrambled version of these pictures (12.2° × 8° or 8° × 12.2° visual angle; Fig. 1). Pictures were selected from the IAPS [Lang et al., 1997a] as well as from the Emotional Picture Set [EmoPicS, Wessa et al., 2010]. Normative mean valence and arousal ratings for the 45 neutral pictures were 5.49 (SD = 0.77) and 3.73 (SD = 0.84), respectively. For the 45 unpleasant pictures, mean valence was 2.17 (SD = 0.62) and mean arousal was 6.24 (SD = 0.63). To increase the contrast between the flickering squares and the background images all pictures were converted into black‐and‐white images with Adobe Photoshop [CS5 12 Extended, 2010]. Such greater contrast was necessary, since we had to increase the flicker frequency in order to be able to filter the SSVEP out of the VEP to image onset and vice versa [Müller and Hillyard, 2000]. Mean luminance was 93.25 RGB value (SD = 16.66) for neutral and 95.33 RGB (SD = 16.06) for unpleasant images. As a baseline measure, a scrambled view of the following picture was displayed first (Fig. 1). Scrambling was done by Fourier transformation of amplitude and phase of each picture. Phase spectrum was displaced by random values but amplitude was not changed. Following that, images were reconstructed by an inverse Fourier transformation. As a result, we received scrambled images with the same low‐level features as the concrete version of it but without any picture content.

Figure 1.

Examples of the stimulus material. A trial started with a scrambled view of the picture. After a given time the picture changed to a normal view of either an unpleasant or neutral picture. Squares represent stimuli of the task and were in red to increase the contrast between squares and black and white images. The example is not taken out of the IAPS image set. It represents an example of an unpleasant image out of public domain photographs.

Red squares flickered at a frequency of 15 Hz (two frames on and off). Every 16.67 ms each individual square moved randomly upwards, downwards, to the left or right side (0.04° visual angle) to create a random moving square kinematogram. One trial lasted for 4,533 ms (i.e., 68 cycles of 15 Hz) and started with the simultaneous onset of a scrambled version of the IAPS image and the flickering squares. From time to time, and unpredictably for our subjects, 32% of the squares moved coherently in one of the cardinal directions (left, right, upward, or downward) for four cycles (i.e., 267 ms). Such coherent motion events were defined as targets and subjects were instructed to attend to the moving squares and to press a button as fast and accurately as possible upon detection of such events. In one trial between zero to four such coherent motion events were possible with a minimum separation between two events of 733 ms. To get the time course for behavioral responses as a function of time, coherent motion events were equally distributed in time bins of 67 ms across all trials per experimental condition. In that way, we had four target events per bin for each experimental condition.

Subjects were further instructed to fixate the central fixation cross and to avoid eye movements and blinks during stimulation. Furthermore, they were informed that the background pictures were task irrelevant and had to be ignored. As mentioned above, the scrambled version of the IAPS image served as a baseline measure. To evoke a VEP to the onset of the concrete picture, the scrambled version of the image changed to its concrete version at a certain time point from one cycle to the next synchronized to the 15 Hz flicker rate of the squares and remained visible in the background until the end of the trial. To avoid that subjects anticipated the time point of change, we varied the point of change in three time windows that were pseudo‐randomized across the whole experiment. These time points were early (13% of trials, with a change in between 200 and 1,000 ms after trial onset), middle (60% of trials, 1,267–2,400 ms), or late (27% of trials, 2,467–4,333 ms). Early and late time points served as catch trials and were not included in the analysis. For these catch trials, we selected 30 additional neutral and unpleasant pictures out of the IAPS picture set, respectively. After each trial a blank screen with a red “×” in the middle was presented between 1,000 and 1,500 ms before the next trial started. Across the whole experiment each picture was presented twice but without a repetition within the next three trials, resulting in 90 trials for unpleasant und neutral background images, respectively. In total, the experiment consisted of 300 trials with 90 trials per valence category that entered the analysis and 120 catch trials. These trials were divided into five blocks and after each block participants received verbal feedback upon their performance. Subjects started the next block by pressing a button.

Before the beginning of the experiment, all subjects completed at least three training blocks until they achieved a target detection rate of at least 60%. For training blocks IAPS images were chosen that were not part of the images used in the experimental trials. For the experimental trials, responding hand was changed after half of the experiment, counterbalanced across participants. After the experiment subjects rated valence and arousal of neutral and unpleasant pictures on the 9‐point scale Self‐Assessment Manikin [SAM, Bradley and Lang, 1994].

Data Recording and Analysis

Electrophysiological data

EEG was recorded at a sampling rate of 256 Hz from 64 Ag‐AgCl scalp electrodes mounted in an elastic cap using a BioSemi ActivTwo amplifier system (BioSemi, Amsterdam; The Netherlands) following an extended version of the international 10–20 system. At left and right earlobes external electrodes were mounted as offline reference. Additionally, four electrodes for monitoring lateral and vertical eye movements as well as blinks were affixed above and below the right eye (vertical electrooculogram) and at the outer canthi (horizontal electrooculogram). For EEG analysis epochs ranging from −1,500 ms before to 2,500 ms after picture change onset were extracted. Trials with eye blinks or eye movements exceeding 2° of visual angle were excluded from further analysis. Subsequently, a variant of the SCADs algorithm [statistical correction of artifacts in dense array studies, Junghöfer et al., 2000] was performed. The mean rejection rate was 8.6% across both conditions with no differences between trials with neutral or unpleasant background images. Remaining trials were algebraically re‐referenced to average reference. After that procedure, we calculated the mean across trials for each experimental condition (neutral vs. unpleasant), respectively. These averaged epochs were then linearly detrended (removal of any linear trends).

For extracting the time course of steady‐state signals we used a Gabor filter [Gabor, 1946]. The Gabor‐filter employed here is implemented as convolutions of the EEG time series with a complex kernel, which consists of an exponential oscillation localized in the time domain by a Gaussian window. It is given by

$$G_{{\sigma }_t,f_0}(t)=K·e^{\frac{-t^2}{2{{\sigma }_t}^2}+2\pi i{f}_{0}t}$$ (1)

where f ₀ is the tuning frequency (15 Hz in the present case) and σ_t determines the bandwidth of the filter (2 Hz here). The normalization constant K is chosen so that the filter has unit gain. The frequency domain representation of the filter kernel is also localized by a Gaussian whose standard deviation σ_f is given by

$${\sigma }_f=1/2\pi {\sigma }_t$$ (2)

A common way of specifying the bandwidth of a signal or a filter is by giving its width at half height, the so‐called full‐width‐at‐half‐maximum (FWHM) bandwidth. For a Gaussian distribution the FWHM‐bandwidth is given by:

$$\text{FWHM}=2\sigma \sqrt{2\ln (2)\approx }2.35\sigma$$ (3)

Inserting σ_f or σ_t in (3) yields the spectral bandwidth or the temporal resolution of the employed Gabor‐filter, respectively. The Gabor filter yields complex coefficients as a result. These complex coefficients contain both the phase and the amplitude of the signal. Here, we were focusing on the amplitude only. The base‐to‐peak amplitude A is calculated by

$$A=\sqrt{{\text{Re}}^2\mathrm{+}{\text{Im}}^2}$$ (4)

where Re and Im are the real and imaginary parts of the complex coefficient for the specific frequency or time‐point of interest.

In this study, the Gabor filter had a center frequency of 15 Hz and a spectral bandwidth of 2 Hz (FWHM) that resulted in a time resolution of ±110.3 ms. To indentify electrodes with greatest overall SSVEP amplitude that entered statistical analysis, we calculated the mean amplitude across the entire epoch and both experimental conditions across all subjects for each electrode. An iso‐contour voltage map was drawn out of that mean (Fig. 3A). Greatest amplitudes were found at electrodes Oz and Iz and the mean across these two electrodes entered statistical analysis for the SSVEP amplitude. For baseline correction we calculated the mean amplitude from 780 to 280 ms before picture change and subtracted that mean from each data point of the entire epoch. To determine the time window for significant SSVEP amplitude differences between the neutral and unpleasant background condition, we calculated running paired t‐tests for every data point, starting at time point zero (picture change). The criterion for a significant time window was more than 10 consecutive data points with a P‐value smaller than 0.05 [Andersen and Müller, 2010]. From the resulting time window (see Results) we calculated the mean amplitude that was subsequently tested by means of a paired t‐test.

Figure 3.

SSVEP, EPN, and LPP (left panels and iso‐contour voltage maps of respective signal (right panels). (A) Grand mean time course of SSVEP amplitudes averaged across electrodes Oz and Iz for neutral (dotted line) or unpleasant background images (solid line). At time point zero in all panels the scrambled image changed to a concrete image. Gray box in all panels indicate the time window of significant differences between the two experimental conditions. The iso‐contour voltage map depicts the topographical distribution of the SSVEP amplitude averaged across all subjects, both conditions and the entire stimulation period. Electrodes that entered the statistical analysis are represented with bigger black dots in all panels. (B) Unfiltered grand mean EPN at electrode P9, rest as in (A). Iso‐contour voltage map represents the scalp distribution of the difference (unpleasant minus neutral) out of the averaged amplitude in the time window between 200 and 350 ms across all subjects. (C) Same as in (B) but for LPP at electrode CPz. Note: Different scales.

Based on previous studies on the EPN and LPP cited above, and the topographical distributions averaged across all subjects of the respective component that are depicted in Figure 2B,C, using the spherical spline algorithm of Perrin and colleagues [Perrin et al., 1989], we selected a cluster of 12 temporal‐occipital electrodes (I1, I2, P7, P8, P9, P10, PO7, PO8, PO3, PO4, O1, and O2) for the EPN and a cluster of six electrodes (CP1, CP2, CPz, Pz, P1, P2) for the statistical analysis of the LPP. For each cluster and experimental condition we calculated the average across the respective electrodes and each data point was corrected by the average of a 100 ms prepicture onset baseline by subtracting that baseline value from each data point of the post picture onset epoch. Given that, we were mainly interested in the time points and duration of significant differential activity in the EPN and LPP, we calculated the same running paired t‐test statistics as described for the SSVEP. Following that, we additionally tested the averaged amplitudes across the identified time windows for the EPN and LPP by paired t‐tests.

Figure 2.

Time course of target detection rates across all subjects in trials with neutral (dotted line) or unpleasant background images (solid line). Gray box indicates time window with significant differences in target detection rates between the two conditions.

To test whether amplitude differences of the EPN or LPP are related to the magnitude of the distraction effect as measured by the SSVEP amplitudes, we build a differences score (unpleasant minus neutral background image) for all three variables. These difference values were subjected to Pearson correlations. We calculated correlations for the relation between EPN and LPP; EPN and SSVEP; and, finally, between LPP and SSVEP, respectively. All EEG analyses were performed off‐line with either the EEGlab tool box (6.03v) or in‐house written programs (Matlab‐Toolbox, Matlab 2006Rb). Statistical analyses were performed with SPSS 11.5 for Windows.

To get an estimation of the cortical sources of the effects in EPN, LPP, and SSVEP, we calculated statistical parametric maps (SPMs) of the cortical current–density distributions with Variable Resolution Electromagnetic Tomography [VARETA, Bosch‐Bayard et al., 2001]. Comparable to the widely used Low Resolution Electromagnetic Tomography [LORETA, Pascual‐Marqui et al., 1994], VARETA is based on a discrete spline distributed inverse model, that is, the obtained generator estimates are the spatially smoothest solutions compatible with the observed scalp topographies. In contrast to LORETA, which imposes a global spatial smoothness for the whole brain, VARETA uses different amounts of spatial smoothness for point as opposed to distributed sources. Thus, it reveals focal solutions in the first case as well as distributed solutions in the latter case. Furthermore, VARETA minimizes the possibility of ghost solutions, which are often present in linear inverse solutions [Trujillo‐Barreto et al., 2004]. In brief, VARETA estimates primary current densities in source space at predefined 3D grid locations (or voxels) that generate the measured EEG data. 3,244 grid points (7.00‐mm grid spacing) and the recording array of 64 electrodes will be placed in registration with the average probabilistic MRI atlas (average brain,) produced by the Montreal Neurological Institute [MNI; Evans et al., 1993]. Importantly, with this approach only voxels are included for regions in which the probability of gray matter is unequal zero, thus the method places anatomical constraints upon the allowable solutions. Here, SPMs were calculated on the basis of the difference values (unpleasant minus neutral background image) and tested them with Hotelling t ²‐tests against zero. Significant voxels were set at threshold of P < 0.001 (Bonferroni corrected for multiple comparisons). Centers of gravity are then given in Talairach coordinates [Talairach and Tournoux, 1987].

Behavioral Data and SAM Ratings

Only button presses that occurred in a time window between 200 and 1,000 ms after target onset were considered as correct responses. To keep behavioral data comparable to our original study [Hindi Attar et al., 2010b] and to have a critical amount of events in each time bin that entered the statistical analysis, we averaged the responses of four bins, resulting in a temporal resolution of 267 ms and 16 events per bin. Furthermore, that bin time was identical to target duration (four cycles of 15 Hz).

Mean target detection rates were tested by means of repeated‐measures analyses of variance (ANAOVAs) comprising the factors of valence (neutral versus unpleasant background image) and time (either three time bins before picture change (from −801 to 0 ms) or in a separate ANOVA eight time bins after picture change (from 0 to 2,136 ms). Post hoc comparisons were calculated with paired t‐tests. Valence and arousal SAM ratings were analyzed with paired t‐tests, respectively. Bonferroni corrections for multiple comparisons were applied where necessary.

RESULTS

SAM Ratings

As expected mean valence ratings for neutral pictures was significant higher (5.73; SD = 0.54) than for unpleasant pictures, 2.53 (SD = 0.63); t(19) = 16.15; P < 0.001. Arousal ratings for neutral pictures was significant lower (2.53; SD = 1.07) than for unpleasant pictures (5.54; SD = 1.53), t(19) = −11.17; P < 0.001.

Behavioral Data

Target detection rate

For the three time bins before picture change there was neither a significant main effect nor a significant interaction. Mean target detection rates in the baseline condition across the three windows were 69.10 ± 5.80% in trials with an upcoming neutral and 69.62 ± 6.26% in trials with an upcoming unpleasant image (Fig. 2). The ANOVA that tested the eight time windows from picture change onward revealed main effects for the factors of time window, F(7,13) = 6.82; P = 0.002; η² = 0.79, as well as for the factor of valence, F(1,19) = 46.38; P < 0.001; η² = 0.71. The interaction valence × time window was not significant, F(7,13) = 1.97; P = 0.138; η² = 0.52. Post hoc analysis for every time bin showed that from 534 to 1,335 ms after the change to a concrete picture target detection rates with unpleasant background pictures was significantly decreased compared to neutral background pictures (see Fig. 2; t _534–801ms(19) = −3.19; P = 0.005; t _{802–1,068ms}(19) = −4.11; P = 0.001; t _{1,069–1,335ms}(19) = −4.49; P < 0.001. On average in these three time windows target detection rates with unpleasant background pictures dropped to 68.78 ± 3.78% compared to neutral ones (77.58 ± 4.78%).

EEG Data

SSVEP amplitudes

Results of running paired t‐tests identified a time window from 382 to 726 ms after picture change with significant amplitude differences between neutral compared to unpleasant background images (Fig. 3A). Consequently, mean amplitudes within that time window were significantly different between the two conditions [t(19) = −3.86; P = 0.001].

EPN and earlier components P1 and N1

As described in “Methods” Section, we first performed a running t‐test to identify the time window for the EPN as well. These t‐tests resulted in a significant difference between the two conditions from 190 to 359 ms. As a result, a statistical test of the baseline corrected mean amplitude revealed significantly greater negative values for unpleasant compared to neutral pictures (t(19) = −6.28; P < 0.001) at the temporo‐occipital cluster that was chosen for analysis (Fig. 3B). As depicted in Figure 3B, the window between 190 and 359 ms starts right at a pronounced negative peak with a latency of 190 ms. We tested the baseline corrected mean peak amplitude (190 ± 10 ms) with the same 12 electrodes as for the EPN, because in the difference topography maximum activity covered the same area of electrodes. Our test revealed a significant difference between unpleasant and neutral images t(19) = −2.78; P = 0.012. In a next step, we also tested the mean P1 component at a latency of 120 ms (±10 ms) averaged across electrodes PO7/PO8 and found no significant differences [t(19) = 1.62; P = 0.12].

LPP

Running paired t‐tests revealed that unpleasant pictures elicited a greater positive going waveform compared to neutral pictures from 707 to 1,035 ms after picture onset at the relevant centro‐parietal electrodes (Fig. 3C). Accordingly, mean amplitude for that time window showed a significant difference [t(19) = 3.64; P = 0.002].

Correlations between EPN, LPP, and SSVEP

Pearson correlations between the difference scores of EPN and LPP were not significant (r = 0.31; P = 0.19). That was also true between EPN and SSVEP (r = 0.07; P = 0.77) as well as between LPP and SSVEP (r = −0.38; P = 0.09).

Source Localization

Figure 4 depicts the cortical sources of the differences values (emotional minus neutral averaged across the respective time windows with differential activity) for the SSVEP, EPN, and LPP, respectively, with Talairach coordinates for the centers of gravity listed in Table 1.

Figure 4.

SPMs of the cortical current–density distributions during the time windows of significant differences (see “Results” Section and gray boxes in Fig. 2) for the SSVEP (A), EPN (B), and LPP (C). The scale represents t ² values and the thresholds corresponds to P < 0.001, corrected for multiple comparisons.

Table 1.

Talairach coordinates of center of gravity for unpleasant‐neutral differences for the different components

Unpleasant‐neutral	Area	Talairach coordinates
		X	y	z
SSVEP	Lingual gyrus left	−16	−91	11
SSVEP	Lingual gyrus right	14	−91	−10
EPN	Angular gyrus left	−43	−69	33
EPN	Occipito‐temporal gyrus left	−48	−62	−12
LPP	Occipito‐Temporal gyrus left	−30	−88	−10
LPP	Superior Temporal gyrus left	−55	−31	−1

Similar to all our previous studies in which we presented flickering squares or dots foveally, in the present experiment sources of the SSVEP differential effect were located in early visual cortex including V1 (Fig. 4A). For the EPN and LPP difference values we found left hemisphere sources. For the EPN, sources were in left occipito‐temporal regions with a maximum in lateral occipital gyrus and in left occipito‐parietal areas with a maximum in middle occipital and angular gyrus (Fig. 4B). For the time window window of the LPP, sources were located in left occipito‐temporal areas with a maximum in lateral occipital‐temporal and superior temporal gyrus. As depicted in Figure 4, the respective sources of differential activity showed practically no overlap, with the exception of sources in early visual areas for the SSVEP and LPP (see Fig. 4A,C).

DISCUSSION

In this study, we examined the EPN and LPP evoked by the onset of a neutral or unpleasant irrelevant background IAPS image concurrently with the time course of SSVEP amplitudes elicited by flickering squares that formed the foreground task. We were able to replicate a number of previous results. As in our previous studies [Müller et al., 2008; Hindi Attar et al., 2010a; Müller et al., 2011], SSVEP amplitude was significantly decreased when we presented an unpleasant compared to a neutral IAPS image in the background. Identical to our previous studies, this difference in SSVEP amplitudes started with a latency of about 400 (382 ms exactly) and lasted several hundred milliseconds. Behavioral data showed significantly reduced hit rates for the emotional compared to the neutral background image condition beginning at about 500 ms and lasting for the following 800 ms. The EPN evoked by the IAPS images was significantly more negative for unpleasant compared to neutral images, and, also in line with previous studies, the LPP was significantly more positive for unpleasant compared to neutral pictures [cf., Schupp et al., 2003, 2004; Bradley et al., 2007; Weinberg and Hajcak, 2010; Sabatinelli et al., 2013]. Running t‐tests that were calculated to estimate the exact onset and duration of the EPN and LPP, respectively, resulted in windows of differential activation between 190 and 360 ms for the EPN and about 700 and 1,035 ms for the LPP. We found no significant differences for the P1 amplitude at a latency of 120 ms at occipito‐parietal electrodes. The first time window with significant differences between unpleasant and neutral images, started at 190 ms, right with a pronounced negative peak that can be seen as a (delayed) N1. When we tested that peak with the same electrode cluster as for the entire time window, we found a significant difference between unpleasant and neutral images. Whether or not that peak is related to a different processing mechanism compared to the entire time window cannot be answered in this study. As illustrated in the introduction section findings regarding the N1 still need some future exploration. That is true with regard to the functional significance [see for example the study by Weinberg and Hajcak, 2010, discussed in the introduction section] as well as for the latency.

Interestingly, we found no statistically significant correlations between EPN and LPP. Only the correlation between SSVEP and LPP difference values was under the 10% alpha range, indicating some weak reciprocal relationship between the two components (see below). Source analysis for emotional effects in EPN, LPP, and SSVEP revealed basically nonoverlapping distinct sources in visual cortex, respectively. Identical to a number of previous studies, we found the modulation of SSVEP amplitudes in early visual cortex from V1 to V3 [Hillyard et al., 1997; Müller et al., 2006, 1998a; Di Russo et al., 2007; Andersen et al., 2008; Andersen and Müller, 2010].

EPN and SSVEP

In the time range of the early negativities, N1 and EPN, the EPN seems to be a robust neural marker for emotional cue extraction in complex scenes [Schupp et al., 2003, 2004; Sabatinelli et al., 2013]. Similar to what we mentioned in the introduction section with regard to the latency of the N1 component, the onset of the EPN also varies between different studies. Although some studies reported of an onset latency in between 150 and 200 ms after picture onset [cf., Sabatinelli et al., 2013], others reported of an onset between 200 and 240 ms [cf., Schupp et al., 2007; Weinberg and Hajcak, 2010]. As with the differences in the latency of the N1, currently there is no study that systematically looked at these time differences. In this study, the EPN (or the negative peak) had a latency of 190 ms. Given the temporal resolution of ±110 ms for the SSVEP time course, the differentiation of SSVEP amplitudes between neutral and unpleasant background pictures started approximately after the EPN peak (Fig. 3). Thus, our results suggest that competition for processing resources in early visual cortex as measured by the SSVEP elicited by the squares that form the foreground task follows directly the emotional cue extraction. Interestingly, we neither found a significant correlation between the EPN and SSVEP effect nor any obviously overlapping cortical sources. Such a pattern points to the direction of rather independent processes that are—however—temporarily aligned. Although we found that the EPN precedes the SSVEP divergence, our data does not allow to unambiguously draw conclusions on the neural mechanism, that is, whether the impact on SSVEP amplitude is due to a top‐down modulation from higher cortical areas (perhaps the EPN generators) to early visual cortex, or due to re‐entrant feedback mechanisms as suggested by Keil et al. 2009.

LPP and SSVEP

In a recent study, Hajcak et al. [2013] presented emotional and neutral IAPS pictures that flickered at the same frequency as in this study. A trial started with a passive viewing time followed by a circle that either directed attention to a nonarousing or arousing part of the image. As expected, the LPP was significantly more positive in the passive viewing window for unpleasant compared to neutral pictures. After the occurrence of the circle, LPP amplitudes were only significantly different from the neutral picture condition when attention was directed to an arousing part of the image. Results of SSVEP amplitudes (that were elicited by the flickering IAPS images) resulted in an identical pattern, with greater amplitudes for unpleasant compared to neutral images during passive viewing and greater amplitudes when the circle directed attention to the arousing part of the image. Similar to the present experiment, the authors found no significant correlations between LPP and SSVEP amplitudes. Hajcak et al. [2013, page 5] speculated that LPP and SSVEP represent different but complementary stages in the processing stream “reflecting the dynamic interplay between bottom‐up and top‐down processes that influence sustained attention.”

Although their design was quite different from the one of this study, here we also found just a trend for a correlation between LPP and SSVEP. In our distraction paradigm, the onset of the LPP followed the onset of the differential steady state activation with a temporal overlap (note the temporal resolution of SSVEP amplitude time course) and reciprocal effects on amplitudes. Although the SSVEP amplitude decreased the amplitude of the LPP became more positive with unpleasant images. That pattern clearly indicates a shift of processing resources away from the foreground task (reduction in SSVEP amplitude) toward the (unpleasant) IAPS image for elaborative processing of the picture content as reflected in greater LPP amplitudes. The shift of resources away from the task towards the image might also be responsible for the relatively late onset of the LPP in the present study, that is, about 300 ms later compared to what has been found with presentation of IAPS images without a foreground task [cf., Hajcak et al., 2007; Weinberg and Hajcak, 2010; Hajcak et al., 2013; Weinberg and Hajcak, 2011]. In a recent feature‐based attention study with a similar design as here, we presented red and blue superimposed flickering dots and found shifting times to one color of about 220 ms as reflected in a significant increase in SSVEP amplitude compared to a baseline period before the cue [Andersen and Müller, 2010]. In the present study, it is conceivable to add some additional time to that shifting time, given that attention first had to be disengaged from the flickering squares [see Posner and Cohen, 1984] and then shifted towards the background image resulting in the late onset of the LPP. Interestingly, even if we take the temporal resolution of the SSVEP time course into account, the effect of the LPP lasted some 200 ms longer compared to the SSVEP differentiation. Thus, a reorientation of attentional resources back to the flickering squares was not invariably linked to a reduction in LPP, that is, content evaluation of the unpleasant picture. So, it seems as if these processes are not entirely linked to each other and act to a certain extend independently, what is reflected in the weak correlation between the two EEG signals. A faint hint for the ongoing content evaluation might be that target detection rates, as depicted in Figure 2, remained under the amount of hits for neutral images for the rest of the presentation time (although not statistically significant).

Similar to Sabatinelli et al. [in press], we found sources in left lateral occipital and inferotemporal cortex for the LPP. In a very recent study, Liu and colleagues recorded EEG and fMRI simultaneously and correlated the LPP amplitudes of single trials with BOLD activity [Liu et al., 2012]. They also found increased visual cortex activity for emotional compared to neutral IAPS images. Similar to the findings of Sabatinelli et al. [2013] they also reported of a broad activation network, including the amygdala that was correlated with the LPP. In extension to the first study, they also found that pleasant and unpleasant images seem to activate different neural networks, respectively, and, thus the LPP seems to represent a highly valence specific component in the EEG. Together, both fMRI studies found LPP related BOLD activity in the same visual areas as we did with our source localization. When we compare these sources with the ones we obtained for the SSVEP, only the source in lateral occipital cortex share about the same location for both signals, what might—speculatively—be linked as part of overlapping cortical activity related to the competitive interaction and, thus, reciprocal amplitudes. No sources in inferotemporal cortex were found for the SSVEP what is not surprising, given that squares most certainly do not require higher object processing in visual cortex.

CONCLUSION

In this study, we analyzed the time course of SSVEP amplitudes elicited by a flickering foreground task and the VEP elicited by the onset of an unpleasant or neutral IAPS image concurrently. We found that the reduction in SSVEP amplitude was preceded by the EPN with a partially temporal overlap. The LPP followed the differential amplitude effect of the SSVEP with a much clearer temporal overlap compared to the EPN and reciprocal behavior of amplitudes. Our results, thus, support the idea of reciprocal and/or competitive interactions between cortical systems that guide top‐down (endogenous) modulation and circuits that guide emotional attention in early visual cortex as suggested recently [Pourtois et al., 2013]. It remains to be determined what the exact neural circuits that guide these interactions are and to what extend temporal dynamics of such shifting processes—away from the foreground task towards an emotional stimulus—depend on the emotional stimuli. The interesting question for the future would be whether these temporal dynamics are temporally modulated by the speed to which emotional cues can be extracted and/or identified. If that would be the case, one would expect a shorter latency of differential SSVEP effects with faces as background stimuli, given that emotional compared to neutral faces show differential effects at the latency of the P1 already [Pourtois et al., 2004, 2005].

Numbers for EmoPicS and IAPS Pictures that were used in the Current Study

EmoPicS numbers of neutral pictures

91, 100, 114, 126, 135, 139.

IAPS numbers of neutral pictures

2037, 2102, 2190, 2191, 2221, 2235, 2240, 2272, 2320, 2372, 2393, 2396, 2435, 2441, 2442, 2480, 2485, 2512, 2560, 2570, 2749, 2840, 2850, 4100, 4542, 7140, 7500, 7546, 7550, 8010, 8090, 8130, 8162, 8232, 8250, 8330, 8371, 8620, and 9210.

IAPS numbers of unpleasant pictures

1200, 1300, 2375.1, 2661, 2683, 2691, 2703, 2710, 2730, 2800, 2811, 3030, 3053, 3060, 3064, 3101, 3110, 3120, 3130, 3170, 3220, 3225, 3230, 3266, 3301, 3350, 3500, 3530, 6022, 6213, 6313, 6360, 6510, 6550, 6560, 8230, 9040, 9042, 9181, 9250, 9254, 9300, 9410, 9433, and 9520.