Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Mar 1.
Published in final edited form as: Biol Psychol. 2012 Apr 26;92(3):513–519. doi: 10.1016/j.biopsycho.2012.04.005

Emotional perception: Correspondence of early and late event-related potentials with cortical and subcortical functional MRI

Dean Sabatinelli 1, Andreas Keil 2, David W Frank 1, Peter J Lang 2
PMCID: PMC3447123  NIHMSID: NIHMS373357  PMID: 22560889

Abstract

Using a picture perception task, here we investigate the relationship of early occipitotemporal and later centroparietal emotion-modulated event-related potentials (ERPs) in one sample to functional magnetic resonance imaging (fMRI) estimates of neural activity in another sample in a replicated experiment. Using this approach, we aimed to link effects found in time-resolved electrocortical measures to specific cerebral structures across individual emotional and nonemotional picture stimuli. The centroparietal late positive potential (LPP) showed covariation with emotion-modulated regions of hemodynamic activation across multiple dorsal and ventral visual cortical structures, while the early occipitotemporal potential was not reliably associated. Subcortical and corticolimbic structures involved in the perception of motivationally relevant stimuli also related to modulation of the LPP, and were modestly associated to the amplitude of the early occipitotemporal potential. These data suggest that early occipitotemporal potentials may reflect multiple sources of modulation including motivational relevance, and supports the perspective that the slow-wave LPP represents aggregate cortical and subcortical structures involved in emotional discrimination.

Keywords: Emotion, attention, perception, fMRI, ERP

Introduction

Emotional perception is hypothesized to be an efficient, iterative process, characterized by feedback of subcortical to cortical sites, evoking a cascade of orienting, attention, metabolic mobilization, and action preparation (Hegdé & Felleman, 2007; Freese & Amaral, 2005; Lamme & Roelfsema, 2000; Lang & Bradley, 2010; Sugase et al.,1999; Vuilleumier, 2005). Such feedback is also at the core of contemporary models of selective attention, which highlight the importance of re-entrant bias signals into visual areas, acting to prioritize representations of objects with behavioral relevance (Kastner & Pinsk, 2004). Viewing affective pictures varying in content has been used frequently as a laboratory approximation for selective attention under natural viewing conditions, where objects are selected based on their ability to engage fundamental emotion systems (Lang et al. 2000). Such engagement is thought to lead to re-entry in to visual cortex, facilitating perception in a fashion that is consistent with bias signals observed in selective attention research. The neural structures involved in this process are beginning to be characterized, yet real-time mechanisms are incompletely described. Event-related and oscillatory electroencephalography (EEG) have provided precise information regarding the timing of emotional stimulus differentiation (Kemp et al., 2002; Keil et al., 2003; 2009; Foti et al., 2009; Pourtois et al., 2004), however, because of deep location and closed dipole architecture relative to cortex, the direct contribution of subcortical nuclei and most deep cortical structures to scalp-recorded EEG is thought to be small. Intracranial EEG in humans (Oya et al., 2002, Krolak-Salmon et al., 2004, Pourtois et al., 2010) can gauge specific subcortical input during emotional perception, but these experiments are exceptional, the sampled regions are sparse, and subjects are often not representative of the normal population. A potentially productive means of obtaining temporally and spatially resolved data is to compare surface-recorded event related potential (ERP) data with functional magnetic resonance imaging (fMRI) data collected during the same experiment, and relate experimental effects observed for the blood oxygen level dependent (BOLD) fMRI signal with ERP components across a range of stimuli.

Previous work of this type (Sabatinelli et al., 2007a) suggests that the emotionally-modulated centroparietal late positive potential (LPP; Cuthbert et al., 2000; Codispoti et al., 2007; Hajcak et al., 2006; Schupp et al., 2000) of the picture onset ERP correlates most reliably with aggregated activity across lateral occipital, intraparietal, and inferior temporal visual cortex. Thus the slow-wave LPP apparently reflects widespread and concurrent activity across the visual system, and not the action of a single localized structure. That study sampled BOLD signal from posterior visual cortical areas, and thus did not assess the contribution of sub-and anterior cortical regions to the LPP.

Here we extend this work by linearly relating the averaged content-related modulation of early and late ERP components recorded in one group of participants with averaged fMRI BOLD signal modulation recorded in a second group, across individual picture exemplars. Thus, observations are averaged responses for individual pictures, not participants. This approach is limited in that it does not reflect within-participant variability, and it does not allow direct correlation of the variables themselves. The between-participants approach employed here will however be sensitive to co-variations of ERP and BOLD as emotional content is manipulated. In particular, this method will detect co-variations related to consistent responses for specific picture types – similar across participants and measurement modalities. If not overstated, such information is useful to guide future work using simultaneous co-registration of EEG and fMRI. To heighten the internal validity of the present method, it is important that all participants view the same series of grayscale images, including erotic couples, neutral people, and mutilation scenes. External validity of any conclusions is enhanced when using equivalent participant samples, and depends on the supposition any given group of approximately 20 undergraduate students will, on average, show a consistent pattern of reactivity across individual picture exemplars. The participant-related sources of error are minimized when the averages of one group are consistent with the average response of another group of 20 undergraduates. In this manner, the pattern of ERP and BOLD modulation across stimuli could be expected to covary, and thus suggest the existence of a consistent neural process underlying both measures.

The initial goal of the current study is to replicate the association of LPP modulation with extrastriate visual cortical sources, relevant the topic of this special issue (Keil et al., 2002; Sabatinelli et al., 2007). In addition, we will compare LPP modulation to BOLD signal from subcortical structures that are reliably modulated by emotional picture perception (Kensinger & Schacter 2007; Sabatinelli et al., 2005; 2011). Here we expect a significant relationship, however the strength of the potential association relative to visual areas is unclear. Finally, we will explore the correlation of visual- and sub- cortical BOLD signal with emotional modulation of the comparatively brief early posterior negativity (EPN; De Cesarei & Codispoti, 2006; Flaisch et al., 2008; Junghöfer et al., 2001; Schupp et al., 2003; 2007), which precedes the LPP, and shows a distinct topography over bilateral occipitotemporal regions. Predictions on the relationship of this comparatively short-lived (~150 ms) potential shift with BOLD signal are tenuous, however we might expect a correlation between EPN amplitude and BOLD signal from lateral and middle occipital visual cortex.

Methods

Participants and procedure

fMRI

Twenty-four introductory psychology students at the University of Florida participated in the experiment for course credit, or $20 compensation. Two subjects' data were lost due to excessive head motion, thus 22 subjects (11 male) remained in the final sample. All volunteers consented to participate after reading a description of the study, approved by the local institutional review board. Prior to entering the bore of the Siemens 3T Allegra MR scanner, subjects were fitted with earplugs and given a patient-alarm squeezeball. A vacuum pillow, padding and explicit verbal instruction were used to limit head motion. Participants viewed the picture stimuli via a coil mounted mirror, presented on a 7 inch LCD screen (25° visual angle) mounted behind the head (IFIS MR-compatible hardware, Intermagnetics, Latham, NY).

EEG

Twenty introductory psychology students (7 male) at the University of Florida participated in the experiment for course credit, or $20 compensation. All volunteers consented to participate after reading a description of the study, approved by the local human subjects review board. Subjects sat in a reclining chair and viewed a 22” monitor from 40” away, yielding a viewing angle of 25 degrees.

Stimuli and experimental paradigm

Participants were instructed to fixate on a central point during a 5.4 min series of pictures. After 3 acclimation trials in which checkerboard stimuli were presented, 24 picture stimuli (IAPS; Lang et al., 2005) were presented in a mixed order. The grayscale pictures depicted 3 categories, including erotic couples; 4611, 4680, 4659, 4690, 4669, 4658, 4676, 4694, neutral people; 2383, 2396, 2102, 2305, 2595, 2513, 2393, 2037, and mutilations; 3069, 3102, 3225, 3060, 3068, 3000, 3030, 3100. Picture categories were matched for luminance and 90% quality JPEG file size, and erotic and mutilation pictures were equivalent in normative ratings of emotional arousal (Lang et al., 2008). Each picture was presented for 3 s, followed by a 9 s fixation-only period. Picture order was pseudo-randomized, allowing no more than 2 successive presentations of a stimulus category.

Acquisition and analysis

fMRI

Once comfortably inside the scanner, participants laid still during an 8m structural scan, which comprised of 160 sagittal slices with 1mm isomorphic voxels in a 256mm FOV. Once the structural acquisition was complete, the functional prescription was applied. This included 50 coronal slices (2.5mm thick, 0.5mm gap) covering the entire brain. Images were collected with 3s temporal resolution and 16ml voxel size (TR 3s, TE 30ms, flip angle 70°, FOV 160mm, 64 × 64 matrix). Each functional time series was high-pass filtered at 0.02 Hz, linear de-trended, mean intensity adjusted, and smoothed with a 5mm spatial filter using BrainVoyager QX (www.brainvoyager.com). Trials containing head motion (<2%) were manually removed from data analyses. Each structural volume was standardized to Talairach space and each functional volume was co-registered to this space.

fMRI regions of interest

For each participant, an analysis of variance identified voxels associated with the time course of picture presentation after convolution with a standard two-gamma hemodynamic response function (Friston et al., 1998). These individual functional maps were thresholded with a False Discovery Rate (Genovese et al., 2002) of p < .01 and 200 ml cluster minimum. Regions of interest (ROIs) were sampled in each participant’s significant voxels located in 10 visual cortical and corticolimbic regions known to be active during emotional picture perception (Britton et al., 2006; Kensinger & Schacter 2007; Sabatinelli et al., 2005; 2007) including calcarine fissure, middle occipital gyrus, lateral occipital cortex, intraparietal sulcus, inferotemporal cortex, parahippocampal gyrus, anterior cingulate (ACC), insula, ventral striatum / nucleus accumbens (vSTR / NAcc), and amygdala. The intention is to include as many regions as is reasonable, with the requirement that voxels within each region be reliably active during stimulus presentation. BOLD signal change values were computed as the percent BOLD signal change from the image volume immediately preceding each picture to the image volume collected from 3–6 s after picture onset. An ROI signal change value was thus derived for each of the 10 ROIs, for each of the 24 picture stimuli.

EEG

Electrocortical activity was recorded from 256 electrodes (Electrical Geodesics, Inc., Eugene, OR; www.egi.com) throughout the picture series. The EEG data were bandpass filtered between 0.1 and 100 Hz and sampled at 250 Hz. The Cz recording reference was re-referenced to the average offline. Trial epochs (200 ms before and 2000 ms after picture onset) were analyzed with Brain Electrical Source Analysis 5.0 (BESA: Megis Software, www.besa.de), with which baseline (100 ms prior to picture onset) correction was performed, ocular and cardiac artifacts removed, and trials with excessive noise (< 2%) excluded. The LPP was sampled as a 13-channel centro-parietal average (channels 7, 8, 9, 16, 17, 43, 44, 80, 133, 186, 187, 199 & 257) from 450–900 ms after stimulus onset. The EPN was sampled as an 8-channel bilateral occipito-temporal average (channels 106, 107, 115, 116, 160, 161, 169 & 170) from 150–300 ms after stimulus onset.

To investigate the covariation of these two measures across individual picture stimuli, the group averaged percent BOLD signal change in each ROI will be correlated with the group averaged µvolt change from the EPN and LPP values.

Results

Picture content effects

Analyses of variance were conducted on fMRI and ERP measures using individual picture stimuli, as opposed to subjects, as random factors. The first picture of the series was excluded from the analyses, as BOLD and ERP reactivity was exaggerated relative to remaining trials, and could have biased the correlation data. Table 1 lists the means and ANOVA results of each ROI. With the exception of the calcarine fissure and parahippocampal gyrus ROIs, all regions showed reliable modulation by picture content. In middle occipital gyrus, inferotemporal cortex, amygdala, anterior cingulate, and insula, enhanced BOLD signal was evident for both erotic and mutilation, relative to neutral people pictures. In lateral occipital cortex, intraparietal sulcus, and vSTR / NAcc, the picture content effect was driven by a difference between erotic and neutral people picture periods, while BOLD signal evoked by mutilation and neutral pictures did not reliably differ.

fMRI ROI (% Δ) x y z Erotica People Mutilations F
(2,20)		 p
Middle occipital gyrus ±20 −89 6 0.838 0.529 0.583 8.65 < .01
Calcarine fissure 0 −86 −5 1.089 1.024 1.120 < 1 ns
Lateral occipital ctx ±42 −76 5 0.885 0.352 0.182 53.06 < .001
Intraparietal sulcus ±20 −73 48 0.888 0.378 0.361 13.61 < .001
Inferotemporal ctx ±28 −62 −12 1.232 0.873 1.100 7.80 < .01
Parahippocampal gyrus ±21 −37 −11 0.701 0.568 0.604 2.14 ns
Amygdala ±19 −5 −11 0.405 −0.060 0.195 17.10 < .001
Anterior cingulate 2 2 33 0.336 0.030 0.197 11.71 < .001
vSTR / NAcc ±7 5 −3 0.280 −0.040 0.030 21.08 < .001
Anterior insula ±37 19 7 0.251 0.062 0.180 4.35 < .05
ERP component (µv)
Late Positive Potential 2.667 −0.270 1.202 18.82 < .001
Early Posterior Negativity 0.813 2.339 1.306 7.98 < .01
Open in a new tab

As expected, the LPP and EPN components also showed reliable modulation by picture content, with greater scalp negativity over occipito-temporal sensors between 150 and 300 ms after picture onset during erotic and mutilation, relative to neutral people pictures, and greater scalp positivity over centro-parietal sensors from 450–900 ms after picture onset. Figure 1 depicts a difference topography of scalp potential across the chosen epoch (a) and waveform average of the selected channels representing the LPP and EPN components (b) across the first second of picture presentation.

Figure 1.

Figure 1

Open in a new tab

Topographical distribution of the EPN and LPP difference maps demonstrate the relative negativity over occipitotemporal sensors between 150–300 ms post picture onset, and relative positivity over centroparietal sensors between 450–900 ms post picture onset. Waveform averages of the 8 sensors included in the EPN and 13 sensors included in the LPP show the time course of voltage change for erotic (blue), neutral people (green) and mutilation (red) picture stimuli.

fMRI – ERP correlations

Figure 2a shows the relationship of LPP scores across 4 visual cortical ROIs, with each point representing reactivity during an individual picture stimulus. Blue, green, and red points represent erotic, neutral people, and mutilation pictures. Consistent with prior work, the LPP showed reliable (R values over .42 are significant at p < .05) positive correlations with BOLD signal in extrastriate occipital (.51), posterior parietal (.56), and inferotemporal (.61) visual cortex. Figure 2b shows the expected negative correlations of EPN scores with the same ROIs, however the relationships were weak, and not statistically significant. Comparisons of R values (absolute) between the LPP and EPN correlations with each visual cortical ROI were not reliable. In other words, though the LPP showed reliable associations with visual ROIs, while the EPN did not, the differences between the significant LPP and nonsignificant EPN correlations were not reliable.

Figure 2.

Figure 2

Open in a new tab

Correlations are shown between LPP and EPN amplitude on the X axes, and BOLD signal change from 4 visual cortical ROIs on the Y axes. R values inset each plot indicate the linear relationship, values over .42 are significant at p < .05. Point colors represent erotic (blue) neutral people (green) and mutilation (red) picture exemplars.

Figure 3a shows the relationship of LPP scores with BOLD signal in 2 subcortical and 2 anterior cortical ROIs, with each point representing reactivity during an individual picture stimulus. Here the LPP showed reliable positive correlations with amygdala (.68), vSTR / NAcc (.72), anterior cingulate (.75) and anterior insula (.51). Figure 3b shows the expected negative correlations of EPN scores with the same ROIs, with more modest correlations reaching significance with anterior cingulate (−.45) and amygdala (−.44). Weaker correlations with vSTR / NAcc (−.37) and anterior insula (−.24) did not reach significance. Again, comparisons of R values (absolute) between the LPP and EPN correlations with each subcortical and anterior cortical ROI were not reliable.

Figure 3.

Figure 3

Open in a new tab

Correlations are shown between LPP and EPN amplitude on the X axes, and BOLD signal change from 4 corticolimbic and subcortical ROIs on the Y axes. R values inset each plot indicate the linear relationship, values over .42 are significant at p < .05. Point colors represent erotic (blue) neutral people (green) and mutilation (red) picture exemplars.

A more stringent test of a potential relationship between ERP and BOLD measures across a set of experimental stimuli was performed in which the 8 nonemotional images were removed from the correlation analyses. Because all measures investigated here have been shown to be more sensitive to emotional arousal than emotional valence, any remaining correlation in measure modulation would be driven by the smaller differences across the equivalently arousing erotic and mutilation images. Table 2 lists the correlations of BOLD signal in the 8 ROIs with the LPP and EPN (R values over .51 are significant at p < .05). In visual cortex, all 4 LPP- BOLD correlations remain significant. Outside the visual system, modulation of the LPP remains significantly correlated with modulation of BOLD signal in anterior cingulated and vSTR-NAcc. No reliable correlations were found between EPN modulation and BOLD signal modulation in any ROI.

Table 2.

Correlation values (asterisks indicate significance of p < .05) of BOLD percent signal change in 8 regions of interest and two ERP components across highly arousing erotic and mutilation picture stimuli.

Measure Late Positive Potential Early Posterior Negativity
Middle occipital gyrus 0.55* 0.04
Lateral occipital ctx 0.56* −0.12
Intraparietal sulcus 0.55* 0.20
Inferotemporal ctx 0.53* −0.05
Anterior Insula 0.34 0.41
Anterior cingulate 0.55* 0.04
vSTR / NAcc 0.61* −0.06
Amygdala 0.33 0.14
Open in a new tab

Discussion

Replicating past work (Sabatinelli et al., 2007a), the emotion-modulated slow-wave LPP and BOLD signal in extrastriate occipital, posterior parietal, and inferotemporal visual cortex showed strong correlations across emotional and neutral picture stimuli. Although widespread, it is important to note that no relationship existed between LPP modulation and BOLD signal originating in calcarine fissure or in parahippocampal gyrus, thus demonstrating regional specificity to the visual structures that are sensitive to the emotional quality of the visual stimuli.

The current data also show that modulation of the LPP correlates strongly with BOLD signal in amygdala, vSTR / NAcc, and anterior cingulate, and moderately well with BOLD signal in the anterior insula. This result is consistent with perspectives and data linking the amygdale with visual cortical modulation during emotional perception (Morris et al.,1998; Pessoa & Adolphs, 2010, Sabatinelli et al., 2009; Vuilleumier, 2005). This covariation with subcortical activity further supports conceptions of the LPP as an index of the registration and sustained processing of motivationally relevant contextual cues (Cuthbert et al., 2000; Lang et al., 1997; Lang & Bradley, 2010; Löw et al., 2008). It seems likely that the relationship between the scalp LPP and subcortical activity is expressed through the cortical structures mediating the perceptual process, though this distinction may require future research with greater sampling accuracy to resolve.

The lack of a significant relationship between modulation of the EPN and visual cortical BOLD is suprising, considering that the picture stimuli used here evoked reliable modulation of the EPN and visual ROIs. The lack of direct relationship across pictures therefore suggests that different picture exemplars are driving the emotion effect for each measure. Examination of the correlation plot between the LPP and EPN (Figure 4) suggests that, while reactivity evoked by neutral people pictures cluster together well, modulation to erotic and mutilation pictures alone are unrelated. While the homogeneous content of each picture category constrains the variance by which a relationship might be expressed, it may be that the EPN is modulated by factors in addition to emotional arousal. Supporting this possibility, early occipitotemporal ERPs have been shown to be sensitive to perceptual characteristics of picture stimuli, perhaps reflecting the mapping of distinctive object features (faces, body parts) present in a visual scene (Bradley et al., 2007), and is particularly sensitive to erotic images (De Cesarei & Codispoti, 2006; Flaisch et al., 2008; Schupp et al., 2003; 2007). Furthermore, relative to the LPP, the EPN is markedly reduced with diminishing stimulus size (De Cesarei & Codispoti, 2006) and resistant to associative transfer via classical conditioning (Franken et al., 2010).

Figure 4.

Figure 4

Open in a new tab

The significant correlation of LPP and EPN amplitude is shown across erotic (blue) neutral people (green) and mutilation (red) picture stimuli.

Emotional modulation of the EPN did correlate modestly with signal modulations registered in the amygdala and anterior cingulate (Figure 3). Similar to the relationship with the LPP, responses to the neutral stimuli are most consistent with the BOLD data, while greater variance is evident across erotic and mutilation pictures. The current study employed a potent stimulus set, intended to evoke reliable emotional reactivity in an attempt to maximize the likelihood of identifying significant regions of BOLD signal modulation, as well as provide sufficient effect sizes in the ERP data across a small number of trials. This experimental practicality also restricted the range of semantic contents depicted in the picture stimuli, and thus limited the range of measurable reactivity. Future experiments in which data is collected across a wider variety of picture stimulus contents may help to identify additional underlying sources of modulation of the EPN, as well as to further differentiate BOLD signal modulation across visual and subcortical structures involved in emotional discrimination.

The comparison of electrocortical and hemodynamic measures of neural activity is laden with many challenges, from technical to interpretive. Though the relationship between BOLD signal and neuronal firing is not completely understood, compelling evidence (Logothetis et al., 2001, 2004; Ogawa et al., 2001) has demonstrated that, though offset in time, there is a strong association between local field potentials recorded intracranially, and BOLD signal in primate visual cortex. The delayed and extended nature of the indirect hemodynamic BOLD contrast likely acts as a temporal filter through which stimulus-coincident neural activation soon apparent on the scalp is integrated and amplified through the gradual and overabundant nature of the metabolic hemodynamic process that is the source of BOLD contrast. The comparison of ERP modulation to later BOLD signal modulation has been recently demonstrated for several components (P50; Bak et al., 2010; P1/N1: Novitskiy et al., 2011; P3; Hesselmann et al., 2011). Thus despite the varying time scales, the emotional modulation of the EPN and LPP with BOLD signal may reflect equivalent neural activity.

Furthermore, here we simplified the process considerably by recording each measure in separate sessions, from separate subjects, and relied on stimulus-driven responses for our comparative analyses. While a within-subject design (e.g., Sabatinelli et al., 2007) is more powerful, a between subject analysis that correlates responses to individual pictures offers evidence of consistency across the ERP and BOLD measures. Both early and late ERP components, and all BOLD ROIs are known to be sensitive to emotional stimulus intensity, and thus we can predict relationships to exist across all comparisons as a result of this common modulation. However, for the correlations to be strong, the responses to particular exemplars will need to be similarly modulated across and within stimulus category. Moreover, the current results demonstrate that some relationships are stronger than others; the later ERP component and all secondary visual system ROIs are modulated with considerable consistency, while the earlier ERP component, despite reliable modulation by emotional arousal, shows no relationship to the arousal-modulated BOLD signal in those same structures. The fact that all 4 visual cortical ROIs (and 2 corticolimbic) continued to show a reliable relationship with LPP modulation after the removal of neutral stimuli from the analyses (Table 2) indicates that the small stimulus content variations remaining across the highly arousing erotic and mutilation stimuli influenced the magnitude of each measure consistently.

It should also be noted that an experiment conducted in a MR scanner presents a different set of physical demands to participants relative to the EEG laboratory, including scanner noise, a supine position, and a greater restriction of movement. Past work has indicated that, despite these additional demands, the pattern of ERP (Sabatinelli et al., 2007) and peripheral physiological modulation (Sabatinelli et al., 2000) by emotional picture perception is highly consistent with emotional modulation collected in laboratory settings (Cuthbert et al., 2000; Schupp et al., 2000).

As a further limitation, the use of channel grouping and epoch average scores of the LPP and EPN does not take full advantage of the dense-array recording technique. However, previous work in which scalp potentials were localized into regionally independent sources did not improve the correspondence with regional BOLD signal, and in fact the scalp LPP showed the only reliable relationship with visual cortical activity (Sabatinelli, et al., 2007). Therefore for the purpose of examining the relationship between the emotional modulation of the LPP and subcortical structures, the rudimentary process used here was sufficient. Future work may further exploit the potential for more sophisticated comparisons of these disparate datasets.

The data presented here replicate and extend the results of our initial multimodal study (Sabatinelli, et al., 2007), supporting the perspective that emotional modulation of the late positive potential represents the aggregated activity of regions of ventral, dorsal, and occipital visual cortex, and suggests that this modulation is supported by subcortical and corticolimbic feedback. This data is consistent with a perspective that considers emotional perception to be characterized as an efficient, iterative process, subject to feedback from subcortical to cortical sites, and associated with a cascade of orienting, attention, metabolic mobilization, and action preparation (Lang & Bradley, 2010; Pessoa & Adolphs, 2010; Vuilleumier, 2005). The measurable neural signals of this process, evident on the scalp surface soon after stimulus onset, show considerable correspondence with neural activity in dorsal and ventral secondary visual cortex, anterior corticolimbic, and subcortical structures, which may act in concert to facilitate emotional perception and attentive processing.

This special issue on emotional attention includes recent work in the field that relates to three basic research problems; 1) what, if any, are the unique brain mechanisms of emotional attention, 2) how can research in this area improve the lives of those that suffer from disorders of emotion, and 3) how do methodological advances in data acquisition and analyses foster advances in our theoretical understanding of emotion and attention? To address the first question, studies in which a combination of instructed and motivated attention manipulations are investigated yield the potential for mechanistic discrimination of the two types of attention. The structures and mechanisms of action that distinguish the two classes of attention may be assumed to underlie any potential neural dissociation. Several studies have done so (Keil et al., 2005, Pourtois & Vuilleumier, 2006; Schupp et al., 2003), and suggest that the effects of emotional stimulus intensity and instructed attention are additive, although the distinct structural and / or functional nature of these types of attention enhancement, if any, is unresolved.

The second question involves the translation of basic science for the treatment of clinical patients. As the fundamental mechanisms of emotional perception and selective attention are defined, we expand our ability to identify dysfunctions associated with specific disorders, as a solid understanding of the typical is essential in the identification of the atypical. Considerable progress has been achieved through the use of functional imaging across multiple diagnoses (Etkin & Wager, 2007; Shin & Liberzon, 2010) as well as genetic categorization (Caspi et al., 2010; Martinowich & Lu, 2009). Psychophysiological and electrocortical measures (McTeague et al., 2010; 2011; 2012) have also advanced the understanding of the neural bases of the emotional dysfunction by identifying overarching dimensions of severity that underlie multiple anxiety disorders. The use of multiple coverging measures is critical to this area of research.

The third special issue topic concerns the value of multimodal and interdisciplinary research in the study of emotional attention. A great deal of research in this area is conducted in nonclinical samples, using noninvasive, repeatable measures that have inherent limitations. These limitations encourage methodological refinement, careful experimental design, and constant analytic development. Expanding the capabilities of individual methods, such as increasing the spatial resolution of dense EEG (Keil et al., 2011) and increasing the temporal resolution of fMRI (Sabatinelli et al., 2009) can enable specific research questions relevant to emotional attention to be addressed. Moreover, the use of multiple recording techniques and a focus on dynamic functional accounts has become common. Approaching a behavioral capability from different signal directions improves out ability to triangulate the hidden mechanism. In this paper we sought to explore the link between two distinct brain meaures of emotional attention that tend to show consistent modulation patterns across emotional and nonemotional stimulus processing. Our data clearly support that core areas identified in a host of studies on emotion perception, reliably co-vary across participants with modulation of the LPP, a well-established electrophysiological index of emotional engagement. Thus the present research further supports the theoretical notion that viewing emotional scenes differentially engages a network of structures, including cortical and deep/subcortical, which then act to optimize information processing and ultimately, action.

Highlights.

  • *

    Emotional modulation of the LPP correlates strongly with sub & visual cortical signal

  • *

    Emotional modulation of the EPN correlates with sub, but not visual cortical signal

  • *

    The slow-wave LPP reflects an aggregate of emotional discrimination processes

Acknowledgements

We thank Robert Sivinski for assistance in data collection.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Bak N, Glenthoj BY, Rostrup E, Larsson HB, Oranje B. Source localization of sensory gating: A combined EEG and fMRI study in healthy volunteers. Neuroimage. 2011;54:2711–2718. doi: 10.1016/j.neuroimage.2010.11.039. [DOI] [PubMed] [Google Scholar]
  2. Bradley MM, Hamby S, Löw A, Lang PJ. Brain potentials in perception: Picture complexity and emotional arousal. Psychophysiology. 2007;44:364–373. doi: 10.1111/j.1469-8986.2007.00520.x. [DOI] [PubMed] [Google Scholar]
  3. Britton JC, Taylor SF, Sudheimer KD, Liberzon I. Facial expressions and complex IAPS pictures: common and differential networks. Neuroimage. 2006;31:906–919. doi: 10.1016/j.neuroimage.2005.12.050. [DOI] [PubMed] [Google Scholar]
  4. Codispoti M, Ferrari V, Bradley MM. Repetition and event-related potentials: distinguishing early and late processes in affective picture perception. Journal of Cognitive Neuroscience. 2007;19:577–586. doi: 10.1162/jocn.2007.19.4.577. [DOI] [PubMed] [Google Scholar]
  5. Cuthbert BN, Schupp HT, Bradley MM, Birbaumer N, Lang PJ. Brain potentials in affective picture processing: covariation with autonomic arousal and affective report. Biological Psychology. 2000;52:95–111. doi: 10.1016/s0301-0511(99)00044-7. [DOI] [PubMed] [Google Scholar]
  6. De Cesarei A, Codispoti M. When does size not matter? Effects of stimulus size on affective modulation. Psychophysiology. 2006;43:207–215. doi: 10.1111/j.1469-8986.2006.00392.x. [DOI] [PubMed] [Google Scholar]
  7. Flaisch T, Junghöfer M, Bradley MM, Schupp HT, Lang PJ. Rapid picture processing: Affective primes and targets. Psychophysiology. 2008;45:1–10. doi: 10.1111/j.1469-8986.2007.00600.x. [DOI] [PubMed] [Google Scholar]
  8. Foti D, Hajcak G, Dien J. Differentiating neural responses to emotional pictures: evidence from temporal-spatial PCA. Psychophysiology. 2009;46:521–530. doi: 10.1111/j.1469-8986.2009.00796.x. [DOI] [PubMed] [Google Scholar]
  9. Franken IHA, Van Strien JW, Bocanegra BR, Huijding J. The P3 Event-Related Potential as an Index of Motivational Relevance A Conditioning Experiment. Journal of Psychophysiology. 2011;25:32–39. [Google Scholar]
  10. Freese JL, Amaral DG. The organization of projections from the amygdala to visual cortical areas TE and V1 in the macaque monkey. Journal of Comparative Neurology. 2005;486:295–317. doi: 10.1002/cne.20520. [DOI] [PubMed] [Google Scholar]
  11. Friston KJ, Josephs O, Rees G, Turner R. Non-linear event-related responses in fMRI. Magnetic Resonance in Medicine. 1998a;39:41–52. doi: 10.1002/mrm.1910390109. [DOI] [PubMed] [Google Scholar]
  12. Genovese CR, Lazar NA, Nichols T. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage. 2002;15:870–878. doi: 10.1006/nimg.2001.1037. [DOI] [PubMed] [Google Scholar]
  13. Hajcak G, Nieuwenhuis S. Reappraisal modulates the electrocortical response to unpleasant pictures. Cognitive, Affective, & Behavioral Neuroscience. 2006;6:291–297. doi: 10.3758/cabn.6.4.291. [DOI] [PubMed] [Google Scholar]
  14. Hegdé J, Felleman DJ. Reappraising the functional implications of the primate visual anatomical hierarchy. The Neuroscientist. 2007;13:416–421. doi: 10.1177/1073858407305201. [DOI] [PubMed] [Google Scholar]
  15. Hesselmann G, Flandin G, Dehaene S. Probing the cortical network underlying the psychological refractory period: A combined EEG-fMRI study. Neuroimage. 2011;56:1608–1621. doi: 10.1016/j.neuroimage.2011.03.017. [DOI] [PubMed] [Google Scholar]
  16. Junghöfer M, Bradley MM, Elbert TR, Lang PJ. Fleeting images: a new look at early emotion discrimination. Psychophysiology. 2001;38:175–178. [PubMed] [Google Scholar]
  17. Keil A, Sabatinelli D, Ding M, Lang PJ, Ihssen N, Heim S. Re-entrant projections modulate visual cortex in affective perception: evidence from Granger causality analysis. Human Brain Mapping. 2009;30:532–540. doi: 10.1002/hbm.20521. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Keil A, Costa V, Smith JC, Sabatinelli D, McGinnis EM, Bradley MM, Lang PJ. Tagging cortical networks in emotion: A topographical analysis. Human Brain Mapping. 2011 doi: 10.1002/hbm.21413. [Epub ahead of print] [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Kemp AH, Gray MA, Eide P, Silberstein RB, Nathan PJ. Steady-state visually evoked potential topography during processing of emotional valence in healthy subjects. Neuroimage. 2002;17:1684–1692. doi: 10.1006/nimg.2002.1298. [DOI] [PubMed] [Google Scholar]
  20. Kensinger EA, Schacter DL. Processing emotional pictures and words: Effects of valence and arousal. Cognitive Affective & Behavioral Neuroscience. 2006;6:110–126. doi: 10.3758/cabn.6.2.110. [DOI] [PubMed] [Google Scholar]
  21. Krolak-Salmon P, Henaff MA, Vighetto A, Bertrand O, Mauguiere F. Early amygdala reaction to fear spreading in occipital, temporal, and frontal cortex: a depth electrode ERP study in human. Neuron. 2004;42:665–676. doi: 10.1016/s0896-6273(04)00264-8. [DOI] [PubMed] [Google Scholar]
  22. Lamme VA, Roelfsema PR. The distinct modes of vision offered by feedforward and recurrent processing. Trends in Neurosciences. 2000;23:571–579. doi: 10.1016/s0166-2236(00)01657-x. [DOI] [PubMed] [Google Scholar]
  23. Lang PJ. International affective picture system (IAPS): Affective ratings of pictures and instruction manual. 2008 [Google Scholar]
  24. Lang PJ, Bradley MM. Emotion and the motivational brain. Biological Psychology. 2010;84:437–450. doi: 10.1016/j.biopsycho.2009.10.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Lang PJ, Bradley MM, Cuthbert BN. Motivated attention: affect activation and action. In: Lang PJ, Simons RF, Balaban MT, editors. Attention and orienting: sensory and motivational processes. Mahwah, N.J.: Lawrence Erlbaum Associates; 1997. pp. 97–135. [Google Scholar]
  26. Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A. Neurophysiological investigation of the basis of the fMRI signal. Nature. 2001;412:150–157. doi: 10.1038/35084005. [DOI] [PubMed] [Google Scholar]
  27. Logothetis NK, Wandell BA. Interpreting the BOLD signal. Annual Review of Physiology. 2004;66:735–769. doi: 10.1146/annurev.physiol.66.082602.092845. [DOI] [PubMed] [Google Scholar]
  28. Löw A, Lang PJ, Smith JC, Bradley MM. Both predator and prey: emotional arousal in threat and reward. Psychological Science. 2008;19:865–873. doi: 10.1111/j.1467-9280.2008.02170.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. McTeague LM, Lang PJ, Laplante M-C, Cuthbert BN, Shumen JR, Bradley MM. Aversive imagery in posttraumatic stress disorder: trauma recurrence, comorbidity, and physiological reactivity. Biological Psychiatry. 2010;67:346–356. doi: 10.1016/j.biopsych.2009.08.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. McTeague LM, Shumen JR, Wieser MJ, Lang PJ, Keil A. Social vision: sustained perceptual enhancement of affective facial cues in social anxiety. Neuroimage. 2011;54:1615–1624. doi: 10.1016/j.neuroimage.2010.08.080. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. McTeague LM, Lang PJ, Wangelin BC, Laplante M-C, Bradley MM. Defensive Mobilization in Specific Phobia: Fear Specificity, Negative Affectivity, and Diagnostic Prominence. Biological Psychiatry. 2012 doi: 10.1016/j.biopsych.2012.01.012. [Epub ahead of print] [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Morris JS, Friston KJ, Buchel C, Frith CD, Young AW, Calder AJ, Dolan RJ. A neuromodulatory role for the human amygdala in processing emotional facial expressions. Brain. 1998;121:47–57. doi: 10.1093/brain/121.1.47. [DOI] [PubMed] [Google Scholar]
  33. Novitskiy N, Ramautar JR, Vanderperren K, De Vos M, Mennes M, Mijovic B, Vanrumste B, Stiers P, Van den Bergh B, Lagae L, Sunaert S, Van Huffel S, Wagemans J. The BOLD correlates of the visual P1 and N1 in single-trial analysis of simultaneous EEG-fMRI recordings during a spatial detection task. Neuroimage. 2011;54:824–835. doi: 10.1016/j.neuroimage.2010.09.041. [DOI] [PubMed] [Google Scholar]
  34. Ogawa S, Lee TM, Stepnoski R, Chen W, Zhu XH, Ugurbil K. An approach to probe some neural systems interaction by functional MRI at neural time scale down to milliseconds. Proceedings of the National Academy of Sciences of the United States of America. 2000;97:11026–11031. doi: 10.1073/pnas.97.20.11026. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Oya H, Kawasaki H, Howard MA, 3rd, Adolphs R. Electrophysiological responses in the human amygdala discriminate emotion categories of complex visual stimuli. Journal of Neuroscience. 2002;22:9502–9512. doi: 10.1523/JNEUROSCI.22-21-09502.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Pessoa L, Adolphs R. Emotion processing and the amygdala: from a 'low road' to 'many roads' of evaluating biological significance. Nature Reviews Neuroscience. 2010;11:773–783. doi: 10.1038/nrn2920. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Pourtois G, Grandjean D, Sander D, Vuilleumier P. Electrophysiological correlates of rapid spatial orienting towards fearful faces. Cerebral Cortex. 2004;14:619–633. doi: 10.1093/cercor/bhh023. [DOI] [PubMed] [Google Scholar]
  38. Pourtois G, Spinelli L, Seeck M, Vuilleumier P. Temporal precedence of emotion over attention modulations in the lateral amygdala: Intracranial ERP evidence from a patient with temporal lobe epilepsy. Cognitive, Affective, & Behavioral Neuroscience. 2010;10:83–93. doi: 10.3758/CABN.10.1.83. [DOI] [PubMed] [Google Scholar]
  39. Pourtois G, Vuilleumier P. Dynamics of emotional effects on spatial attention in the human visual cortex. Progress in Brain Research. 2006;156:67–91. doi: 10.1016/S0079-6123(06)56004-2. [DOI] [PubMed] [Google Scholar]
  40. Sabatinelli D, Bradley MM, Fitzsimmons JR, Lang PJ. Parallel amygdala and inferotemporal activation reflect emotional intensity and fear relevance. Neuroimage. 2005;24:1265–1270. doi: 10.1016/j.neuroimage.2004.12.015. [DOI] [PubMed] [Google Scholar]
  41. Sabatinelli D, Bradley MM, Lang PJ, Costa VD, Versace F. Pleasure rather than salience activates human nucleus accumbens and medial prefrontal cortex. Journal of Neurophysiology. 2007a;98:1374–1379. doi: 10.1152/jn.00230.2007. [DOI] [PubMed] [Google Scholar]
  42. Sabatinelli D, Keil A, Fitzsimmons JR, King W, Lang PJ. Psychophysiology in a simulated MR scanner. Psychophysiology. 2000;37:S84. [Google Scholar]
  43. Sabatinelli D, Fortune EE, Li Q, Siddiqui A, Krafft C, Oliver WT, Beck S, Jeffries J. Emotional perception: meta-analyses of face and natural scene processing. Neuroimage. 2011;54:2524–2533. doi: 10.1016/j.neuroimage.2010.10.011. [DOI] [PubMed] [Google Scholar]
  44. Sabatinelli D, Lang PJ, Bradley MM, Costa VD, Keil A. The timing of emotional discrimination in human amygdala and ventral visual cortex. Journal of Neuroscience. 2009;29:14864–14868. doi: 10.1523/JNEUROSCI.3278-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Sabatinelli D, Lang PJ, Keil A, Bradley MM. Emotional perception: Correlation of functional MRI and event-related potentials. Cerebral Cortex. 2007b;17:1085–1091. doi: 10.1093/cercor/bhl017. [DOI] [PubMed] [Google Scholar]
  46. Schupp HT, Cuthbert BN, Bradley MM, Cacioppo JT, Ito T, Lang PJ. Affective picture processing: the late positive potential is modulated by motivational relevance. Psychophysiology. 2000;37:257–261. [PubMed] [Google Scholar]
  47. Schupp HT, Junghöfer M, Weike AI, Hamm AO. Emotional facilitation of sensory processing in the visual cortex. Psychological Science. 2003;14:7–13. doi: 10.1111/1467-9280.01411. [DOI] [PubMed] [Google Scholar]
  48. Schupp HT, Stockburger J, Codispoti M, Junghöfer M, Weike AI, Hamm AO. Selective visual attention to emotion. Journal of Neuroscience. 2007;27:1082–1089. doi: 10.1523/JNEUROSCI.3223-06.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Sugase Y, Yamane S, Ueno S, Kawano K. Global and fine information coded by single neurons in the temporal visual cortex. Nature. 1999;400:869–873. doi: 10.1038/23703. [DOI] [PubMed] [Google Scholar]
  50. Vuilleumier P. How brains beware: neural mechanisms of emotional attention. Trends in Cognitive Sciences. 2005;9:585–594. doi: 10.1016/j.tics.2005.10.011. [DOI] [PubMed] [Google Scholar]

RESOURCES