Skip to main content
PMC home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2012 Oct 17;32(42):14563–14572. doi: 10.1523/JNEUROSCI.3109-12.2012

Neural Substrate of the Late Positive Potential in Emotional Processing

Yuelu Liu 1, Haiqing Huang 1, Menton McGinnis-Deweese 2, Andreas Keil 2, Mingzhou Ding 1,✉
PMCID: PMC3516184  NIHMSID: NIHMS416103  PMID: 23077042

Abstract

The late positive potential (LPP) is a reliable electrophysiological index of emotional perception in humans. Despite years of research, the brain structures that contribute to the generation and modulation of LPP are not well understood. Recording EEG and fMRI simultaneously, and applying a recently proposed single-trial ERP analysis method, we addressed the problem by correlating the single-trial LPP amplitude evoked by affective pictures with the blood oxygen level-dependent (BOLD) activity. Three results were found. First, relative to neutral pictures, pleasant and unpleasant pictures elicited enhanced LPP, as well as heightened BOLD activity in both visual cortices and emotion-processing structures such as amygdala and prefrontal cortex, consistent with previous findings. Second, the LPP amplitude across three picture categories was significantly correlated with BOLD activity in visual cortices, temporal cortices, amygdala, orbitofrontal cortex, and insula. Third, within each picture category, LPP–BOLD coupling revealed category-specific differences. For pleasant pictures, the LPP amplitude was coupled with BOLD in occipitotemporal junction, medial prefrontal cortex, amygdala, and precuneus, whereas for unpleasant pictures significant LPP–BOLD correlation was observed in ventrolateral prefrontal cortex, insula, and posterior cingulate cortex. These results suggest that LPP is generated and modulated by an extensive brain network composed of both cortical and subcortical structures associated with visual and emotional processing and the degree of contribution by each of these structures to the LPP modulation is valence specific.

Introduction

The event-related potential (ERP) method is used extensively in affective neuroscience. A key feature observed in ERPs evoked by emotionally engaging stimuli is the late positive potential (LPP), which is characterized by an amplitude enhancement for pleasant and unpleasant stimuli, relative to neutral stimuli, and has a centroparietal maximum topography. For affective picture viewing, LPP starts ∼300–400 ms after picture onset and is often sustained throughout the duration of picture presentation (Cuthbert et al., 2000). LPP amplitude has been shown to vary systematically with the experienced intensity of the affective picture content (Schupp et al., 2000; Keil et al., 2002) and exhibit abnormal patterns in mood disorders and other psychiatric conditions (Leutgeb et al., 2011; Weinberg and Hajcak, 2011; Jaworska et al., 2012). In parallel, functional magnetic resonance imaging (fMRI) has found that viewing of affective pictures is associated with increased blood oxygen level-dependent (BOLD) activity in widespread brain regions, including occipital, parietal, inferotemporal cortices, and amygdala (Breiter et al., 1996; Lang et al., 1998a; Bradley et al., 2003; Norris et al., 2004; Sabatinelli et al., 2005, 2009), suggesting that emotionally salient content enhances visual stimulus processing by attracting attentional resources (Lang et al., 1998b; Lang and Bradley, 2010). Together, if enhanced LPP and BOLD reflect a common underlying mechanism, one might expect a coupling between LPP amplitude and BOLD activity in the above reported regions.

A prior study recording EEG and fMRI from the same subjects but in separate sessions has found that LPP amplitude was positively correlated with BOLD responses in lateral occipital, parietal, and inferotemporal cortices (Sabatinelli et al., 2007a). This study did not examine LPP–BOLD coupling in other higher-order emotional processing areas such as prefrontal cortex and deep subcortical structures known to be involved in emotional perception (Sabatinelli et al., 2009). A more recent study using a between-subjects design observed coupling between LPP amplitude and BOLD activity in both deep and anterior structures (Sabatinelli et al., 2012), but it is still unclear whether these structures are engaged with the LPP in a category-specific way based on trial-by-trial information within each picture category. The advent of the simultaneous EEG–fMRI recording technique, together with reliable estimation of single-trial ERPs, opens new avenues to address this problem.

We recorded simultaneous EEG–fMRI while subjects passively viewed emotionally arousing and neutral pictures. The single-trial LPP amplitudes were estimated using a recently proposed method (Xu et al., 2009) and then correlated with the single-trial evoked BOLD responses across the entire brain to identify brain structures whose activity is linearly related to the trial-by-trial variation of the scalp-recorded LPP. In addition, in light of a host of prior studies reporting differential engagement of cortical and subcortical structures in appetitive versus aversive processing (Sabatinelli et al., 2007a), we investigated whether trial-by-trial LPP amplitude fluctuations are mediated by different neural generators during different affective states by examining the coupling between LPP amplitude and BOLD within each picture category (pleasant, neutral, unpleasant).

Materials and Methods

Participants.

Fifteen healthy volunteers participated in the experiment in exchange for either course credits or a financial incentive of $30. One participant withdrew from the experiment. In addition, data from three participants were discarded due to artifacts generated by excessive movement inside the scanner. The remaining 11 participants (7 females; mean age, 20; SD, 2.65) had normal or corrected-to-normal vision. The experimental protocol was approved by the Institutional Review Board of the University of Florida. Informed consent was obtained from all participants before the experiment.

Stimuli and procedure.

The stimuli consisted of 20 pleasant, 20 neutral, and 20 unpleasant pictures selected from the International Affective Picture System (IAPS) (Lang et al., 2008) based on their normative valence and arousal levels. The IAPS picture numbers used in this study are as follows: pleasant: 4311, 4599, 4610, 4624, 4626, 4641, 4658, 4680, 4694, 4695, 2057, 2332, 2345, 8186, 8250, 2655, 4597, 4668, 4693, 8030; neutral: 2398, 2032, 2036, 2037, 2102, 2191, 2305, 2374, 2377, 2411, 2499, 2635, 2347, 5600, 5700, 5781, 5814, 5900, 8034, 2387; unpleasant: 1114, 1120, 1205, 1220, 1271, 1300, 1302, 1931, 3030, 3051, 3150, 6230, 6550, 9008, 9181, 9253, 9420, 9571, 3000, 3069. The selected pictures cover a wide range of contents and normative ratings. The pleasant pictures in general included sport scenes, romance, and erotic couples, whereas the unpleasant pictures incorporated threat, attack scenes, and bodily mutilations. The neutral pictures included landscapes and neutral human beings. The mean pleasure (valence) rating for pleasant, neutral, and unpleasant pictures was 7.0, 6.3, and 2.8, respectively. The pleasant and unpleasant pictures had similar mean arousal levels (pleasant, 5.8; unpleasant, 5.9), both being higher than neutral pictures (4.2). Pictures were chosen to be similar overall in composition, matched in jpeg size across categories, and comparable in rated complexity, to minimize confounds.

The experimental paradigm was implemented in an event-related fMRI design. Each IAPS picture was centrally displayed on a monitor for 3 s followed by a variable (2800 or 4300 ms) interstimulus interval. All participants completed five experimental sessions in which the pictures were repeated in different random orders. The order of picture presentation was also randomized across different participants. A cross was displayed at the center of the screen during the entire experiment to aid fixation. Stimuli were presented on an MR-compatible monitor using E-Prime software (Psychology Software Tools). The monitor was placed outside the scanner bore over the head of the subject. Participants viewed the task presentation in the scanner via a reflective mirror. Before the start of the first experimental session, participants were instructed to maintain eye fixation whenever the fixation cross is present and viewed the pictures without moving their eyes. After the experiment, as a validation, participants were asked to provide their ratings of 12 representative pictures (4 pictures within each category) they had not seen during the experiment along the scales of valence and arousal using a paper and pencil version of the self-assessment manikin (Bradley and Lang, 1994). The entire experiment lasted ∼40 min.

Simultaneous EEG–fMRI acquisition.

MRI data were collected on a 3 T Philips Achieva scanner (Philips Medical Systems). Two hundred twelve volumes of functional images were acquired using a gradient-echo echo-planar imaging sequence during each session [echo time (TE), 30 ms; repetition time (TR), 1.98 s; flip angle, 80°; slice number, 36; field of view, 224 mm; voxel size, 3.5 × 3.5 × 3.5 mm; matrix size, 64 × 64]. The slices were acquired in ascending order and oriented parallel to the plane connecting the anterior and posterior commissure. Slice acquisition was performed within an interval of 1850 ms during each TR, leaving an interval of 130 ms toward the end of the TR where no image acquisition was performed. This image acquisition approach allowed us to visually monitor the EEG recording within each volume during the no-scan period in which EEG was not contaminated by gradient switching. A T1-weighted high-resolution structural image was also obtained.

EEG data were recorded during the experiment using a 32 channel MR-compatible EEG system (Brain Products). Thirty-one sintered Ag/AgCl electrodes were placed according to the 10–20 system, and one additional electrode was placed on subject's upper back to monitor electrocardiograms (ECGs). The recorded ECG will be used to detect heartbeat events to be used for the removal of the cardioballistic artifact. The EEG channels were referenced to site FCz during recording. The impedance from all scalp channels was kept below 10 kΩ during experiment as suggested by the manufacturer. EEG signal was recorded with a built-in 0.1∼250 Hz bandpass filter and digitized to 16 bit at a sampling rate of 5 kHz. The digitized EEG signal was then transferred to the recording computer via a fiber-optic cable. The EEG recording system was synchronized with the scanner's internal clock throughout the recording session to ensure the successful removal of the gradient artifact in subsequent analyses.

EEG data preprocessing.

Brain Vision Analyzer 2.0 (Brain Products) was used for data preprocessing. Gradient artifacts in the EEG data were removed using a modified version of the original algorithms proposed by Allen et al. (2000). Briefly, an artifact template was created by segmenting and averaging the data according to the onset of each volume within a sliding window consisting of 41 consecutive volumes, and subtracted from the raw EEG data. To remove the cardioballistic artifact, an average artifact subtraction method (Allen et al., 1998) was used, in which R peaks were detected in the low-pass-filtered ECG signal and used to construct a delayed average artifact template over 21 consecutive heartbeat events in a sliding-window approach, which was subtracted from the original EEG signal. The resulting EEG data were then low-pass filtered with the cutoff set at 50 Hz, downsampled to 250 Hz, and rereferenced to the average reference. These data were then exported to the EEGLAB (Delorme and Makeig, 2004). Second-order blind identification (SOBI) (Belouchrani et al., 1993) was performed to further correct for eye blinking, residual cardioballistic, and movement-related artifacts. Recent work has shown that SOBI is effective in removing the residual cardioballistic artifact (Vanderperren et al., 2010), as well as in separating EEG data into physiologically interpretable components (Tang et al., 2005; Klemm et al., 2009). The artifacts-corrected data were then epoched from −300 to 2000 ms with 0 ms being the onset of affective pictures. The prestimulus baseline was defined as −300 to 0 ms. The EEG epochs were averaged within each condition separately to produce the average ERP (AERP). The AERP coming from each subject was further averaged across subjects to produce the grand average ERP.

Single-trial estimation of LPP.

Channel Pz was chosen to guide our subsequent EEG-informed fMRI analysis as it showed strong LPP difference between both emotional conditions and the neutral condition (Fig. 1B,C). The ERP of each trial at Pz was estimated using the analysis of single-trial ERP and ongoing activity (ASEO) method (Xu et al., 2009). ASEO has the following basic steps. First, according to the VSPOA (variable signal plus ongoing activity) generative model (Chen et al., 2006), the recorded EEG data for the rth trial (r = 1, 2, …, R) are expressed as follows:

graphic file with name zns04212-2671-m01.jpg

where sn(t) (n = 1, 2, …, N) is the nth ERP component and zr(t) is an AR (autoregressive) process modeling the ongoing activity. Within each individual trial, the nth ERP component is characterized by an amplitude scaling factor βrn and a latency shift τrn to account for trial-to-trial variability. Second, using a proper initial condition, the ASEO algorithm estimates the waveforms of the ERP components and their associated amplitude scaling factors and latency shifts in an iterative fashion. Third, from scaled and latency-adjusted ERP component estimates, the single-trial ERP was reconstructed on a trial-by-trial basis. Fourth, the LPP amplitude on each trial was obtained by averaging the single-trial ERP amplitude within a time interval around the peak of LPP (Figs. 2, 3). To date, ASEO has been applied to study both monkey local field potential data and human EEG data (Wang et al., 2008; Wang and Ding, 2011; Liu et al., 2012). See Xu et al. (2009) for a more detailed description of the ASEO algorithm.

Figure 1.

Figure 1.

Open in a new tab

ERP analysis. A, Grand average (n = 11 subjects) ERP showing the LPP at Pz with time 0 set to the onset of pictures. B, The scalp topography showing the ERP difference between pleasant and neutral conditions. Here, ERP was averaged within the time interval from 300 to 600 ms. C, The scalp topography showing the ERP difference between unpleasant and neutral conditions. Here, ERP was averaged within the same interval.

Figure 2.

Figure 2.

Open in a new tab

Single-trial ERP analysis. A, Epoched EEG data at Pz from a representative subject after artifact removal. B, Single-trial ERPs estimated using ASEO from data shown in A. C, Raster plot of the EEG data in A (smoothed with a moving average across 5 trials for visualization purpose). D, Raster plot of single-trial ERP data in B smoothed the same way as in C. E, Comparison between AERP using data in A and averaged ASEO single-trial ERPs (ASEO AERP) using data in B.

Figure 3.

Figure 3.

Open in a new tab

Single-trial LPP dynamics. A, Trial-by-trial LPP amplitude for the pleasant and neutral conditions. B, Trial-by-trial LPP amplitude for the unpleasant and neutral conditions. The horizontal axes in A and B represent the sequential index of picture presentation within each picture category. C, Schematic illustrating the use of single-trial LPP amplitude as a parametric modulation in GLM. The height of each boxcar function is scaled by the mean-removed LPP amplitude.

MRI data analysis.

The fMRI data were processed using SPM5 (http://www.fil.ion.ucl.ac.uk/spm/). The first five volumes in an experimental session were discarded to allow the scanner to stabilize. Slice timing was corrected using sinc interpolation to account for differences in acquisition time. The images were then corrected for head movement by spatially realigning the images to the sixth image of each session. Images were further normalized and registered to a standard template within SPM [the Montreal Neurological Institute (MNI) space]. The functional volume images were resampled to a spatial resolution of 3 × 3 × 3 mm. The transformed images were then smoothed by a Gaussian filter with a full-width at half-maximum of 8 mm. The low-frequency temporal drifts were removed from the functional images by applying a high-pass filter with a cutoff frequency of 1/128 Hz, and the global signal was removed by dividing every voxel in a slice by the estimated global signal value.

Two separate general linear models (GLMs) with parametric modulation were constructed to model the relationship between LPP amplitude and BOLD. In the first model, we were mainly interested in examining the overall LPP–BOLD coupling across all three picture categories (i.e., pleasant, neutral, and unpleasant). Therefore, in this model, we combined all three picture categories and modeled the resulting single experimental condition with two task-related regressors. The first regressor described the combined condition and consisted of a sequence of boxcar functions with unit height synchronized with the onset of pictures. The width of each boxcar function was set to the duration of picture presentation (3 s). For the second regressor, the height of the boxcar functions in the first regressor varied according to the ASEO-estimated single-trial LPP amplitude, with the mean level of the single-trial LPP amplitudes removed (see Fig. 3C). This regressor was intended to account for both the between-category and the within-category variability in LPP amplitude and its correlation with BOLD. The two regressors were further convolved with a canonical hemodynamic response function (HRF) before being incorporated into the design matrix. Six regressors describing the subjects' head movement obtained from image preprocessing were further introduced to account for any movement-related artifacts during scan. We referred to our first model as the “full model.” A single contrast was performed based on this model resulting in one statistical map for each subject.

In the second model, we included six task-related regressors, with three corresponding to the three picture categories (pleasant, neutral, and unpleasant), and the other three modeling the relationship between LPP amplitudes and BOLD within each picture category. Since this model captures trial-by-trial coupling between LPP amplitude and BOLD within each picture category, we refer to it as the “within-category model.” The three regressors modeling the three picture categories consisted of sequences of boxcar functions with unit height placed according to the picture onset within the corresponding categories. The width of the boxcar functions remained the same as those in the “full model” (3 s). For regressors modeling trial-by-trial LPP–BOLD coupling, we further scaled the height of the boxcar functions with the corresponding mean removed single-trial LPP amplitudes within each picture category. Similar to the full model described above, the six task-related regressors were convolved with a canonical HRF. Additional six regressors describing subjects' head movement were also introduced as covariates in the model. The following contrasts were performed based on this model: pleasant versus neutral, unpleasant versus neutral, LPP–BOLD coupling within each category (pleasant, neutral, and unpleasant).

Second-level analyses were performed using random-effects models based on the statistical maps obtained from the within-subjects analyses to examine reproducible effects across all subjects. For conventional BOLD contrasts, the group level T maps were thresholded at p < 0.05 [false discovery rate (FDR) corrected]. For LPP–BOLD coupling, because it is derived from the trial-to-trial variability signal on top of the large picture-evoked response, and this residual variability is generally smaller than the large picture-evoked response and may contain other ongoing brain processes that are not related to the experimental task, the correlation was generally smaller and required a more relaxed statistical threshold. In line with recent studies using EEG-informed fMRI analysis (Debener et al., 2005; Eichele et al., 2005; Bénar et al., 2007; Scheeringa et al., 2011), for LPP–BOLD coupling effects, the group-level T maps were thresholded at p < 0.003 (uncorrected). A cluster-level threshold of k = 5 voxels was further imposed.

Results

ERP analysis

Postexperiment ratings of 12 representative pictures indicate that subjects correctly distinguished the three categories of pictures (valence: pleasant, 6.5; neutral, 5.3; unpleasant, 2.6; arousal: pleasant, 4.7; neutral, 2.9; unpleasant, 4.0). Figure 1A shows enhanced positivity for both pleasant and unpleasant pictures, relative to neutral pictures, in the grand average ERP at Pz, starting from ∼300 ms after picture onset. Since the time interval during which LPP reached a maximum was relatively broad, the LPP amplitude was measured by taking the mean within 300–600 ms. A one-way ANOVA on LPP amplitudes with repeated measures identified a significant picture category-related difference in LPP amplitudes (F(2,20) = 23.11; p < 0.05). As further indicated by the results of post hoc tests with Bonferroni adjusted significance level, the mean (M) LPP amplitudes for both the pleasant (M, 3.153; SD, 1.733) and unpleasant (M, 3.090; SD, 2.048) pictures were significantly larger than that for the neutral pictures (M, 1.523; SD, 1.684; pleasant vs neutral: t(10) = 6.26, p < 0.001; unpleasant vs neutral: t(10) = 5.65, p < 0.001). However, no significant difference was found in LPP amplitudes between the pleasant and unpleasant categories (t(10) = 0.227; p = 0.825). The ERP difference topography further confirmed that the positivity is strongest among parietal channels for both pleasant and unpleasant conditions (Fig. 1B,C), agreeing with prior ERP studies of emotion and motivation (Lang and Bradley, 2010). The enhanced positivity was sustained throughout the duration of picture presentation for both pleasant and unpleasant pictures.

Artifact-removed raw EEG data and ASEO-estimated single-trial ERPs at Pz are shown in Figure 2, A and B, for a representative subject. Displayed as raster plots in Figure 2, C and D, the ASEO-estimated single-trial ERPs improved signal-to-noise ratio, and preserved the trial-by-trial dynamics of the LPP amplitude, which is important because the single-trial LPP amplitudes were used to correlate with BOLD response in subsequent analyses. The validity of the single-trial ERPs can be further supported by averaging the data in Figure 2, A and B (Wang et al., 2008; Xu et al., 2009; Wang and Ding, 2011). The similarity between ASEO AERP and the original AERP indicated that the algorithm accurately estimated the single-trial ERPs from the raw data (Fig. 2E). Figure 3, A and B, displays single-trial LPP amplitudes as functions of trial index. From these two figures, one can see that, on average, the single-trial LPP amplitudes for both pleasant and unpleasant pictures are higher than those for the neutral condition, yielding further support for the grand average ERP result in Figure 1. The estimated single-trial LPP amplitudes were used to scale the boxcar functions to examine the relationship between LPP amplitude and BOLD (Fig. 3C).

fMRI analysis

The traditional fMRI group level activation maps contrasting (1) pleasant against neutral and (2) unpleasant against neutral picture categories are shown in Figure 4, A and B. Both pleasant and unpleasant pictures activated the emotion-processing network, encompassing the visual cortices and deep structures. Specifically, relative to neutral pictures, pleasant pictures mainly activated areas in bilateral occipito-temporal junctions, bilateral posterior parietal cortices, medial prefrontal cortex, and left orbital frontal cortex. Other activated areas included fusiform gyrus, lingual gyrus, middle frontal gyrus, supramarginal gyrus, parahippocampal gyrus, and temporal pole (Fig. 4A). Unpleasant pictures mainly activated areas such as the bilateral occipito-temporal junctions, bilateral posterior parietal cortices, bilateral ventral lateral prefrontal cortices, left orbital frontal cortex, bilateral amygdalae/hippocampi, insula, and supplementary motor area. Other activated areas included fusiform gyrus, lingual gyrus, supramarginal gyrus, temporal pole, and postcentral cortex (Fig. 4B). In general, the activation results agree with a previous report using a similar experimental protocol (Sabatinelli et al., 2007a) and serve to demonstrate that the quality of the fMRI data is preserved despite the presence of the EEG recording system in the scanner.

Figure 4.

Figure 4.

Open in a new tab

Activation maps based on BOLD contrast. A, Pleasant versus neutral (P vs N) condition. B, Unpleasant versus neutral (U vs N) condition. Activations are thresholded at p = 0.05 FDR corrected. PPC, Posterior parietal cortex; OFC, orbital frontal cortex; MPFC, medial prefrontal cortex; OTJ, occipitotemporal junction; AMYG, amygdala; HIPP, hippocampus; TP, temporal pole; SMA, supplementary motor area; VLPFC, ventral lateral prefrontal cortex; INS, insula.

Trial-by-trial coupling of LPP and BOLD

To assess the coupling between the LPP amplitude and BOLD, the coefficients for regressors associated with LPP amplitude variations were examined. Using the “full” model, which combined the neutral, pleasant, and unpleasant picture categories as a single regressor to describe the effect of both between- and within-category LPP amplitude variations on BOLD, the single-trial LPP amplitude was positively correlated with evoked BOLD responses in bilateral occipito-temporal junctions, insula, amygdala/hippocampus, temporal poles, and left orbital frontal cortex (Fig. 5A).

Figure 5.

Figure 5.

Open in a new tab

LPP–BOLD coupling maps in which highlighted regions indicate significant correlation between trial-by-trial fMRI response and the corresponding single-trial LPP amplitude. A, Pleasant, neutral, and unpleasant combined. B, Pleasant. C, Unpleasant. All maps are thresholded at p = 0.003. A cluster threshold k = 5 is further applied. OTJ, Occipitotemporal junction; INS, insula; AMYG, amygdala; HIPP, hippocampus; TP, temporal pole; PCu, precuneus; PCC, posterior cingulate cortex; MPFC, medial prefrontal cortex; VLPFC, ventrolateral prefrontal cortex; NAcc, nucleus accumbens.

Using the “within-category” model, which allowed us to examine the coupling between LPP and BOLD within each picture category, it was found that, for the neutral condition, no significant coupling between single-trial LPP amplitude and BOLD existed among subjects. For the pleasant condition, the single-trial LPP amplitude was positively correlated with BOLD responses in bilateral occipito-temporal junctions, amygdala, temporal poles, precuneus, right nucleus accumbens (NAcc), medial prefrontal cortex (MPFC), and cerebellum (Fig. 5B). For the unpleasant condition, the single-trial LPP amplitude was positively correlated with BOLD responses in bilateral ventral lateral prefrontal cortices, bilateral insula, temporal poles, precuneus, left middle temporal cortex, and left postcentral cortex (Fig. 5C). Table 1 listed the MNI coordinates of these regions. It is worth noting that we did not find any structures in which BOLD was negatively correlated with LPP amplitude under the same significance level.

Table 1.

Regions showing coupling between LPP amplitude and BOLD

Anatomical regions Side MNI coordinates (x, y, z) Z score
Three picture categories combined
    Occipital cortex Left −27, −96, 3 3.33
Right 27, −99, 15 3.62
    Superior temporal cortex Left −45, −3, −6 3.51
Right 45, 0, −9 3.19
    Middle temporal cortex Left −45, −63, −3 3.00
    Inferior temporal cortex Right 54, −63, −12 3.18
    Insula Left −45, 9, −9 4.30
Right 42, −6, −6 3.83
    Orbitofrontal cortex Left −30, 12, −15 3.33
    Amygdala/hippocampus Left −24, −3, −24 3.07
Right 21, −6, −19 3.18
    Temporal pole Left −33, −3, −42 3.10
Right 27, −3, −42 3.62
Pleasant pictures
    Occipital cortex Left −33, −90, −9 3.83
Right 33, −87, −6 4.81
    Middle temporal cortex Left −53, −72, 12 3.63
Right 60, −60, 9 4.49
    Inferior temporal cortex Right 54, 63, −6 3.41
    Amygdala Left −21, 0, −18 4.28
Right 21, 0, −18 3.67
    Temporal pole Left −27, 6, −21 3.94
Right 42, 21, −27 4.27
    Precuneus 3, −48, 57 3.75
    Medial prefrontal cortex Right 9, 63, 15 3.47
    Cerebellum Left −45, −63, −24 3.63
    Nucleus accumbens Right 6, 12, −9 3.36
Unpleasant pictures
    Ventral lateral prefrontal cortex Left −51, 33, 0 4.19
Right 57, 33, 6 3.20
    Middle temporal cortex Right 51, −6, −15 3.60
    Temporal pole Left −57, 0, −6 3.20
Right 54, 9, −9 3.63
    Insula Left −39, 9, −3 3.05
Right 45, 12, −6 3.39
    Precuneus Left −12, −51, 21 3.34
    Postcentral cortex Left −27, −36, 57 3.18
Open in a new tab

Discussion

Emotional stimuli evoke a LPP that is interpreted to signify enhanced attention and visual processing (Bradley, 2009). This signature ERP response is known to be altered in mood disorders and other related psychiatric illnesses (Foti et al., 2010; Leutgeb et al., 2011; Weinberg and Hajcak, 2011; Jaworska et al., 2012; Weymar et al., 2012). Despite the importance of LPP, its neural substrate is not clear. ERP source localization is only partly successful (Keil et al., 2002; Sabatinelli et al., 2007a). This problem is addressed here by recording simultaneous EEG and fMRI while subjects viewed IAPS affective pictures. Extracting LPP on a trial-by-trial basis, the overall LPP amplitude variability across three picture categories (pleasant, neutral, and unpleasant) was found to be correlated with BOLD responses in an extensive cortical and subcortical network, including visual cortices and deep emotion-processing structures. In addition, consistent with the notion that appetitive and aversive information may engage different neural substrates, the brain areas in which BOLD activity was correlated with LPP amplitude during pleasant picture viewing were not the same as those during unpleasant picture viewing.

Methodological considerations

Prior investigation of the association between LPP and evoked BOLD responses relied on recording EEG and fMRI over separate sessions and correlating averaged responses across subjects (Sabatinelli et al., 2007a, 2012). One potential drawback of such an approach is that it is difficult to keep the psychological and biological conditions exactly the same in different recording sessions, and moreover, the correlation between average LPP and BOLD does not reflect their trial-by-trial covariations and coupling toward individual pictures within each subject. A new technology, simultaneous EEG–fMRI, has become available over the past few years. As has been recently demonstrated (Nagai et al., 2004; Debener et al., 2005; Eichele et al., 2005; Scheeringa et al., 2011), simultaneous EEG–fMRI is capable of overcoming these limitations and has the potential to allow the interrogation of trial-by-trial associations between the two recording modalities. The present study benefits from another methodological development. Applying a recently proposed single-trial analysis algorithm called ASEO, we were able to estimate the LPP amplitude on a trial-by-trial basis. The improved signal-to-noise ratio helps to more fully reveal the brain areas whose BOLD responses are correlated with LPP fluctuations. The ASEO algorithm has been tested in both monkey local field potential data (Wang et al., 2008; Xu et al., 2009) and human scalp EEG data (Fogelson et al., 2009; Wang and Ding, 2011; Liu et al., 2012), and proven useful to address questions arising in a number of contexts, ranging from the proper preprocessing of event-related data for functional connectivity analysis to the temporal dynamics of emotional conditioning.

LPP–BOLD coupling and its theoretical significance

The brain regions in which BOLD activities covary with single-trial LPP amplitude across three picture categories reflected a joint involvement of the visual system and a network of structures known to be associated with emotional processing. Past source-space modeling of LPP has only been able to identify generators in the visual system, including occipito-temporal, parietal, and inferior temporal cortices (Keil et al., 2002; Sabatinelli et al., 2007a), despite the fact that the amplitude of LPP is closely related to the rated intensity of emotion (Schupp et al., 2000; Keil et al., 2002). In line with a recent study (Sabatinelli et al., 2012), the present study extends the prior findings by showing that deep structures such as the insula and the amygdala, along with visual structures, contribute to the generation of LPP and its amplitude modulation. These results provide further evidence supporting the view that emotional pictures naturally attract attentional resources as a result of the engagement of the fundamental motivational system (Cacioppo et al., 1993, 1994; Lang et al., 1997, 1998b; Palomba et al., 1997; Schupp et al., 2000; Pastor et al., 2008; Lang and Bradley, 2010).

The contribution to scalp-recorded potentials by the emotional processing areas may be modulatory and mediated by the visual cortex. It has been hypothesized that, when observers view emotionally engaging scenes, cortical and deep subcortical structures modulate visual cortex in a reentrant fashion (Keil et al., 2009; Pessoa and Adolphs, 2010). These structures include the amygdala, insula, and prefrontal cortex (Rotshtein et al., 2001; Adolphs, 2002; Phan et al., 2002; LeDoux, 2003; Zald, 2003; Luo et al., 2007). As evidenced by recent human intracranial studies, amygdala and orbitofrontal cortex show fast responses to the emotional content of stimuli, which would enable them to provide reentrant feedback to the visual cortices (Oya et al., 2002; Krolak-Salmon et al., 2004), to potentially affect the gain of visual neurons. In addition, the activation of emotion-related BOLD modulations in the amygdala is found to precede that in the fusiform gyrus, the medial occipital gyrus, and the calcarine fissure, consistent with the putative role of the amygdala in initiating the reentrant interaction (Sabatinelli et al., 2009). Together, the positive correlation found in the current study between single-trial LPP amplitude and BOLD activity in the amygdala, insula, and areas in prefrontal cortex is supportive of the reentrant hypothesis of emotional perception.

Category-specific network processing

Restricting to pictures within each valence category revealed category-dependent differences in regions showing LPP and BOLD coupling. For pleasant pictures, LPP amplitude variability was found to be linearly related to BOLD activity in bilateral amygdalae, whereas for unpleasant pictures, this correlation was absent. This finding was further corroborated by a contrast of LPP–BOLD coupling maps between pleasant and unpleasant conditions showing that the amygdala was preferentially engaged in LPP modulation in the pleasant condition. For the unpleasant pictures, the amygdala was found to be activated by a traditional fMRI contrast between unpleasant and neutral conditions, suggesting a rather constant level of amygdala activation on a trial-by-trial basis. Specifically, while the amygdala activity is clearly enhanced by unpleasant pictures as a whole, it does not covary with trial-by-trial LPP changes for different pictures within this category. This may reflect a limited response variability of the amygdala for unpleasant scenes or the presence of additional sources of variance that govern the modulation of LPP. A wider selection of unpleasant scenes may help to identify the potential sources of covariation between electrophysiological measures and amygdala BOLD activity during aversive/defensive engagement.

Whether amygdala responds to pleasant stimuli is debated, although several studies reported amygdala activation during processing of pleasant stimuli, especially when these stimuli have highly arousing erotica content (Lane et al., 1997; O'Doherty et al., 2001; Hamann et al., 2002; Zald, 2003). In the present study, the fact that the amygdala was not activated in a traditional fMRI analysis by contrasting the pleasant with the neutral category may indicate that the overall mean amygdala BOLD response to pleasant pictures was small. Yet, the positive correlation between LPP amplitude and BOLD activities in bilateral amygdalae suggests that the BOLD fluctuations in the amygdala are parametrically related to intensity variations of pleasant emotional content, as measured by the LPP. This finding further demonstrates that combining electrophysiological recordings and functional imaging can yield information not possible with either modality alone.

The LPP–BOLD coupling was found in MPFC and NAcc only for pleasant pictures. Contrasting LPP–BOLD coupling maps between pleasant and unpleasant conditions showed that the MPFC is preferentially coupled with LPP in the pleasant condition. In addition, the MPFC was activated by contrasting pleasant with neutral pictures, but the same region was not activated when comparing unpleasant and neutral pictures. NAcc and MPFC are densely interconnected (Ferry et al., 2000; Roberts et al., 2007) and often show correlated activities in human reward studies (Knutson et al., 2003; Rogers et al., 2004), leading to the view that both NAcc and MPFC are part of the human reward system mediating appetitive behaviors. Several studies have reported involvement of both structures in the perception of pleasant emotional stimuli including attractive faces, romance, and erotica (Aharon et al., 2001; Karama et al., 2002; O'Doherty et al., 2003; David et al., 2005; Ferretti et al., 2005; Sabatinelli et al., 2007b) as well as in vivid imagery of pleasant scenes (Costa et al., 2010). Hence, the observed positive correlation between LPP amplitude and key structures in the reward system may reflect the contribution to the cortical potential by the appetitive system.

For unpleasant pictures, the LPP was correlated with BOLD in insula and adjacent temporal and ventrolateral prefrontal cortices (VLPFC). The same regions were found to be active when contrasting unpleasant with neutral pictures. It has been shown repeatedly that the human insula is involved in tasks that challenge the representation of bodily states as well as processing of emotions (Craig, 2009; Gu et al., 2010; Fan et al., 2011), especially for aversive emotions such as disgust and threat (Phillips et al., 1997, 1998; Adolphs, 2002; Straube and Miltner, 2011). Reliable covariation of the insula and periinsula with the LPP during aversive perception demonstrates that these structures contribute to the modulation of cortical potential during aversive events. It also suggests that the aversive/defensive circuitry involved in processing unpleasant pictures is not engaged in an all-or-none fashion, but varies parametrically as a function of aversive motivation, indexed by the LPP amplitude. It is worth noting that insula was not activated when LPP–BOLD coupling maps were contrasted between pleasant and unpleasant conditions. In light of the finding that insula is activated during viewing of highly arousing pleasant stimuli (e.g., erotica) (Karama et al., 2002), this may suggest that, for pleasant pictures, insula is engaged in LPP modulation but the degree of modulation did not reach the level of statistical significance.

Finally, for both pleasant and unpleasant pictures, BOLD activity in regions within midline parietal cortex is linearly correlated with the LPP. For pleasant pictures, the region that was most correlated with LPP amplitude was within the precuneus, whereas for unpleasant pictures, such correlation occurred in more ventral regions, particularly the posterior cingulate cortex and precuneus. The involvement of these parietal regions is in line with electrophysiological data and with the theoretical notion of “motivated attention” in which features of emotionally arousing scenes attract perceptual processing resources (Keil et al., 2002, 2012).

Footnotes

This work was supported by NIH Grant MH097320.

References

  1. Adolphs R. Neural systems for recognizing emotion. Curr Opin Neurobiol. 2002;12:169–177. doi: 10.1016/s0959-4388(02)00301-x. [DOI] [PubMed] [Google Scholar]
  2. Aharon I, Etcoff N, Ariely D, Chabris CF, O'Connor E, Breiter HC. Beautiful faces have variable reward value: fMRI and behavioral evidence. Neuron. 2001;32:537–551. doi: 10.1016/s0896-6273(01)00491-3. [DOI] [PubMed] [Google Scholar]
  3. Allen PJ, Polizzi G, Krakow K, Fish DR, Lemieux L. Identification of EEG events in the MR scanner: the problem of pulse artifact and a method for its subtraction. Neuroimage. 1998;8:229–239. doi: 10.1006/nimg.1998.0361. [DOI] [PubMed] [Google Scholar]
  4. Allen PJ, Josephs O, Turner R. A method for removing imaging artifact from continuous EEG recorded during functional MRI. Neuroimage. 2000;12:230–239. doi: 10.1006/nimg.2000.0599. [DOI] [PubMed] [Google Scholar]
  5. Belouchrani A, Abed-Meraim K, Cardoso JF, Moulines E. Second-order blind separation of temporally correlated sources. Proc Int Conf on Digital Sig Proc; Cyprus. 1993. pp. 346–351. [Google Scholar]
  6. Bénar CG, Schön D, Grimault S, Nazarian B, Burle B, Roth M, Badier JM, Marquis P, Liegeois-Chauvel C, Anton JL. Single-trial analysis of oddball event-related potentials in simultaneous EEG-fMRI. Hum Brain Mapp. 2007;28:602–613. doi: 10.1002/hbm.20289. [DOI] [PubMed] [Google Scholar]
  7. Bradley MM. Natural selective attention: orienting and emotion. Psychophysiology. 2009;46:1–11. doi: 10.1111/j.1469-8986.2008.00702.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Bradley MM, Lang PJ. Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry. 1994;25:49–59. doi: 10.1016/0005-7916(94)90063-9. [DOI] [PubMed] [Google Scholar]
  9. Bradley MM, Sabatinelli D, Lang PJ, Fitzsimmons JR, King W, Desai P. Activation of the visual cortex in motivated attention. Behav Neurosci. 2003;117:369–380. doi: 10.1037/0735-7044.117.2.369. [DOI] [PubMed] [Google Scholar]
  10. Breiter HC, Etcoff NL, Whalen PJ, Kennedy WA, Rauch SL, Buckner RL, Strauss MM, Hyman SE, Rosen BR. Response and habituation of the human amygdala during visual processing of facial expression. Neuron. 1996;17:875–887. doi: 10.1016/s0896-6273(00)80219-6. [DOI] [PubMed] [Google Scholar]
  11. Cacioppo JT, Crites SL, Berntson GG, Coles MGH. If attitudes affect how stimuli are processed, should they not affect the event-related brain potential? Psychol Sci. 1993;4:108–112. [Google Scholar]
  12. Cacioppo JT, Crites SL, Jr, Gardner WL, Bernston GG. Bioelectrical echoes from evaluative categorizations: I. A late positive brain potential that varies as a function of trait negativity and extremity. J Pers Soc Psychol. 1994;67:115–125. doi: 10.1037//0022-3514.67.1.115. [DOI] [PubMed] [Google Scholar]
  13. Chen Y, Bressler SL, Knuth KH, Truccolo WA, Ding M. Stochastic modeling of neurobiological time series: power, coherence, Granger causality, and separation of evoked responses from ongoing activity. Chaos. 2006;16 doi: 10.1063/1.2208455. 026113. [DOI] [PubMed] [Google Scholar]
  14. Costa VD, Lang PJ, Sabatinelli D, Versace F, Bradley MM. Emotional imagery: assessing pleasure and arousal in the brain's reward circuitry. Hum Brain Mapp. 2010;31:1446–1457. doi: 10.1002/hbm.20948. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Craig AD. How do you feel—now? The anterior insula and human awareness. Nat Rev Neurosci. 2009;10:59–70. doi: 10.1038/nrn2555. [DOI] [PubMed] [Google Scholar]
  16. Cuthbert BN, Schupp HT, Bradley MM, Birbaumer N, Lang PJ. Brain potentials in affective picture processing: covariation with autonomic arousal and affective report. Biol Psychol. 2000;52:95–111. doi: 10.1016/s0301-0511(99)00044-7. [DOI] [PubMed] [Google Scholar]
  17. David SP, Munafò MR, Johansen-Berg H, Smith SM, Rogers RD, Matthews PM, Walton RT. Ventral striatum/nucleus accumbens activation to smoking-related pictorial cues in smokers and nonsmokers: a functional magnetic resonance imaging study. Biol Psychiatry. 2005;58:488–494. doi: 10.1016/j.biopsych.2005.04.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Debener S, Ullsperger M, Siegel M, Fiehler K, von Cramon DY, Engel AK. Trial-by-trial coupling of concurrent electroencephalogram and functional magnetic resonance imaging identifies the dynamics of performance monitoring. J Neurosci. 2005;25:11730–11737. doi: 10.1523/JNEUROSCI.3286-05.2005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods. 2004;134:9–21. doi: 10.1016/j.jneumeth.2003.10.009. [DOI] [PubMed] [Google Scholar]
  20. Eichele T, Specht K, Moosmann M, Jongsma ML, Quiroga RQ, Nordby H, Hugdahl K. Assessing the spatiotemporal evolution of neuronal activation with single-trial event-related potentials and functional MRI. Proc Natl Acad Sci U S A. 2005;102:17798–17803. doi: 10.1073/pnas.0505508102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Fan J, Gu X, Liu X, Guise KG, Park Y, Martin L, de Marchena A, Tang CY, Minzenberg MJ, Hof PR. Involvement of the anterior cingulate and frontoinsular cortices in rapid processing of salient facial emotional information. Neuroimage. 2011;54:2539–2546. doi: 10.1016/j.neuroimage.2010.10.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Ferretti A, Caulo M, Del Gratta C, Di Matteo R, Merla A, Montorsi F, Pizzella V, Pompa P, Rigatti P, Rossini PM, Salonia A, Tartaro A, Romani GL. Dynamics of male sexual arousal: distinct components of brain activation revealed by fMRI. Neuroimage. 2005;26:1086–1096. doi: 10.1016/j.neuroimage.2005.03.025. [DOI] [PubMed] [Google Scholar]
  23. Ferry AT, Ongür D, An X, Price JL. Prefrontal cortical projections to the striatum in macaque monkeys: evidence for an organization related to prefrontal networks. J Comp Neurol. 2000;425:447–470. doi: 10.1002/1096-9861(20000925)425:3<447::aid-cne9>3.0.co;2-v. [DOI] [PubMed] [Google Scholar]
  24. Fogelson N, Wang X, Lewis JB, Kishiyama MM, Ding M, Knight RT. Multimodal effects of local context on target detection: evidence from P3b. J Cogn Neurosci. 2009;21:1680–1692. doi: 10.1162/jocn.2009.21071. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Foti D, Olvet DM, Klein DN, Hajcak G. Reduced electrocortical response to threatening faces in major depressive disorder. Depress Anxiety. 2010;27:813–820. doi: 10.1002/da.20712. [DOI] [PubMed] [Google Scholar]
  26. Gu X, Liu X, Guise KG, Naidich TP, Hof PR, Fan J. Functional dissociation of the frontoinsular and anterior cingulate cortices in empathy for pain. J Neurosci. 2010;30:3739–3744. doi: 10.1523/JNEUROSCI.4844-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hamann SB, Ely TD, Hoffman JM, Kilts CD. Ecstasy and agony: activation of the human amygdala in positive and negative emotion. Psychol Sci. 2002;13:135–141. doi: 10.1111/1467-9280.00425. [DOI] [PubMed] [Google Scholar]
  28. Jaworska N, Thompson A, Shah D, Fisher D, Ilivitsky V, Knott V. Acute tryptophan depletion effects on the vertex and late positive potentials to emotional faces in individuals with a family history of depression. Neuropsychobiology. 2012;65:28–40. doi: 10.1159/000328992. [DOI] [PubMed] [Google Scholar]
  29. Karama S, Lecours AR, Leroux JM, Bourgouin P, Beaudoin G, Joubert S, Beauregard M. Areas of brain activation in males and females during viewing of erotic film excerpts. Hum Brain Mapp. 2002;16:1–13. doi: 10.1002/hbm.10014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Keil A, Bradley MM, Hauk O, Rockstroh B, Elbert T, Lang PJ. Large-scale neural correlates of affective picture processing. Psychophysiology. 2002;39:641–649. doi: 10.1017.S0048577202394162. [DOI] [PubMed] [Google Scholar]
  31. Keil A, Sabatinelli D, Ding M, Lang PJ, Ihssen N, Heim S. Re-entrant projections modulate visual cortex in affective perception: directional evidence from Granger causality analysis. Hum Brain Mapp. 2009;30:532–540. doi: 10.1002/hbm.20521. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Keil A, Costa V, Smith JC, Sabatinelli D, McGinnis EM, Bradley MM, Lang PJ. Tagging cortical networks in emotion: a topographical analysis. Hum Brain Mapp. 2012 doi: 10.1002/hbm.21413. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Klemm M, Haueisen J, Ivanova G. Independent component analysis: comparison of algorithms for the investigation of surface electrical brain activity. Med Biol Eng Comput. 2009;47:413–423. doi: 10.1007/s11517-009-0452-1. [DOI] [PubMed] [Google Scholar]
  34. Knutson B, Fong GW, Bennett SM, Adams CM, Hommer D. A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI. Neuroimage. 2003;18:263–272. doi: 10.1016/s1053-8119(02)00057-5. [DOI] [PubMed] [Google Scholar]
  35. Krolak-Salmon P, Hénaff MA, Vighetto A, Bertrand O, Mauguière F. Early amygdala reaction to fear spreading in occipital, temporal, and frontal cortex: a depth electrode ERP study in human. Neuron. 2004;42:665–676. doi: 10.1016/s0896-6273(04)00264-8. [DOI] [PubMed] [Google Scholar]
  36. Lane RD, Reiman EM, Bradley MM, Lang PJ, Ahern GL, Davidson RJ, Schwartz GE. Neuroanatomical correlates of pleasant and unpleasant emotion. Neuropsychologia. 1997;35:1437–1444. doi: 10.1016/s0028-3932(97)00070-5. [DOI] [PubMed] [Google Scholar]
  37. Lang PJ, Bradley MM. Emotion and the motivational brain. Biol Psychol. 2010;84:437–450. doi: 10.1016/j.biopsycho.2009.10.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Lang PJ, Simons RF, Balaban MT. Attention and orienting: sensory and motivational processes. Hillsdale, NJ: Lawrence Erlbaum Associates; 1997. [Google Scholar]
  39. Lang PJ, Bradley MM, Fitzsimmons JR, Cuthbert BN, Scott JD, Moulder B, Nangia V. Emotional arousal and activation of the visual cortex: an fMRI analysis. Psychophysiology. 1998a;35:199–210. [PubMed] [Google Scholar]
  40. Lang PJ, Bradley MM, Cuthbert BN. Emotion, motivation, and anxiety: brain mechanisms and psychophysiology. Biol Psychiatry. 1998b;44:1248–1263. doi: 10.1016/s0006-3223(98)00275-3. [DOI] [PubMed] [Google Scholar]
  41. Lang PJ, Bradley MM, Cuthbert BN. International affective picture system (IAPS): affective ratings of pictures and instruction manual. Gainesville, FL: University of Florida; 2008. Technical Report A-8. [Google Scholar]
  42. LeDoux J. The emotional brain, fear, and the amygdala. Cell Mol Neurobiol. 2003;23:727–738. doi: 10.1023/A:1025048802629. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Leutgeb V, Schäfer A, Schienle A. Late cortical positivity and cardiac responsitivity in female dental phobics when exposed to phobia-relevant pictures. Int J Psychophysiol. 2011;79:410–416. doi: 10.1016/j.ijpsycho.2011.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Liu Y, Keil A, Ding M. Effects of emotional conditioning on early visual processing: temporal dynamics revealed by ERP single-trial analysis. Hum Brain Mapp. 2012;33:909–919. doi: 10.1002/hbm.21259. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Luo Q, Holroyd T, Jones M, Hendler T, Blair J. Neural dynamics for facial threat processing as revealed by gamma band synchronization using MEG. Neuroimage. 2007;34:839–847. doi: 10.1016/j.neuroimage.2006.09.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Nagai Y, Critchley HD, Featherstone E, Fenwick PB, Trimble MR, Dolan RJ. Brain activity relating to the contingent negative variation: an fMRI investigation. Neuroimage. 2004;21:1232–1241. doi: 10.1016/j.neuroimage.2003.10.036. [DOI] [PubMed] [Google Scholar]
  47. Norris CJ, Chen EE, Zhu DC, Small SL, Cacioppo JT. The interaction of social and emotional processes in the brain. J Cogn Neurosci. 2004;16:1818–1829. doi: 10.1162/0898929042947847. [DOI] [PubMed] [Google Scholar]
  48. O'Doherty J, Rolls ET, Francis S, Bowtell R, McGlone F. Representation of pleasant and aversive taste in the human brain. J Neurophysiol. 2001;85:1315–1321. doi: 10.1152/jn.2001.85.3.1315. [DOI] [PubMed] [Google Scholar]
  49. O'Doherty J, Winston J, Critchley H, Perrett D, Burt DM, Dolan RJ. Beauty in a smile: the role of medial orbitofrontal cortex in facial attractiveness. Neuropsychologia. 2003;41:147–155. doi: 10.1016/s0028-3932(02)00145-8. [DOI] [PubMed] [Google Scholar]
  50. Oya H, Kawasaki H, Howard MA, 3rd, Adolphs R. Electrophysiological responses in the human amygdala discriminate emotion categories of complex visual stimuli. J Neurosci. 2002;22:9502–9512. doi: 10.1523/JNEUROSCI.22-21-09502.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Palomba D, Angrilli A, Mini A. Visual evoked potentials, heart rate responses and memory to emotional pictorial stimuli. Int J Psychophysiol. 1997;27:55–67. doi: 10.1016/s0167-8760(97)00751-4. [DOI] [PubMed] [Google Scholar]
  52. Pastor MC, Bradley MM, Löw A, Versace F, Moltó J, Lang PJ. Affective picture perception: emotion, context, and the late positive potential. Brain Res. 2008;1189:145–151. doi: 10.1016/j.brainres.2007.10.072. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Pessoa L, Adolphs R. Emotion processing and the amygdala: from a “low road ” to “many roads ” of evaluating biological significance. Nat Rev Neurosci. 2010;11:773–783. doi: 10.1038/nrn2920. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Phan KL, Wager T, Taylor SF, Liberzon I. Functional neuroanatomy of emotion: a meta-analysis of emotion activation studies in PET and fMRI. Neuroimage. 2002;16:331–348. doi: 10.1006/nimg.2002.1087. [DOI] [PubMed] [Google Scholar]
  55. Phillips ML, Young AW, Senior C, Brammer M, Andrew C, Calder AJ, Bullmore ET, Perrett DI, Rowland D, Williams SC, Gray JA, David AS. A specific neural substrate for perceiving facial expressions of disgust. Nature. 1997;389:495–498. doi: 10.1038/39051. [DOI] [PubMed] [Google Scholar]
  56. Phillips ML, Young AW, Scott SK, Calder AJ, Andrew C, Giampietro V, Williams SC, Bullmore ET, Brammer M, Gray JA. Neural responses to facial and vocal expressions of fear and disgust. Proc Biol Sci. 1998;265:1809–1817. doi: 10.1098/rspb.1998.0506. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Roberts AC, Tomic DL, Parkinson CH, Roeling TA, Cutter DJ, Robbins TW, Everitt BJ. Forebrain connectivity of the prefrontal cortex in the marmoset monkey (Callithrix jacchus): an anterograde and retrograde tract-tracing study. J Comp Neurol. 2007;502:86–112. doi: 10.1002/cne.21300. [DOI] [PubMed] [Google Scholar]
  58. Rogers RD, Ramnani N, Mackay C, Wilson JL, Jezzard P, Carter CS, Smith SM. Distinct portions of anterior cingulate cortex and medial prefrontal cortex are activated by reward processing in separable phases of decision-making cognition. Biol Psychiatry. 2004;55:594–602. doi: 10.1016/j.biopsych.2003.11.012. [DOI] [PubMed] [Google Scholar]
  59. Rotshtein P, Malach R, Hadar U, Graif M, Hendler T. Feeling or features: different sensitivity to emotion in high-order visual cortex and amygdala. Neuron. 2001;32:747–757. doi: 10.1016/s0896-6273(01)00513-x. [DOI] [PubMed] [Google Scholar]
  60. Sabatinelli D, Bradley MM, Fitzsimmons JR, Lang PJ. Parallel amygdala and inferotemporal activation reflect emotional intensity and fear relevance. Neuroimage. 2005;24:1265–1270. doi: 10.1016/j.neuroimage.2004.12.015. [DOI] [PubMed] [Google Scholar]
  61. Sabatinelli D, Lang PJ, Keil A, Bradley MM. Emotional perception: correlation of functional MRI and event-related potentials. Cereb Cortex. 2007a;17:1085–1091. doi: 10.1093/cercor/bhl017. [DOI] [PubMed] [Google Scholar]
  62. Sabatinelli D, Bradley MM, Lang PJ, Costa VD, Versace F. Pleasure rather than salience activates human nucleus accumbens and medial prefrontal cortex. J Neurophysiol. 2007b;98:1374–1379. doi: 10.1152/jn.00230.2007. [DOI] [PubMed] [Google Scholar]
  63. Sabatinelli D, Lang PJ, Bradley MM, Costa VD, Keil A. The timing of emotional discrimination in human amygdala, inferotemporal, and occipital cortex. J Neurosci. 2009;29:14864–14868. doi: 10.1523/JNEUROSCI.3278-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Sabatinelli D, Keil A, Frank DW, Lang PJ. Emotion perception: correspondence of early and late event-related potentials with cortical and subcortical functional MRI. Biol Psychol. 2012 doi: 10.1016/j.biopsycho.2012.04.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Scheeringa R, Mazaheri A, Bojak I, Norris DG, Kleinschmidt A. Modulation of visually evoked cortical fMRI responses by phase of ongoing occipital alpha oscillations. J Neurosci. 2011;31:3813–3820. doi: 10.1523/JNEUROSCI.4697-10.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Schupp HT, Cuthbert BN, Bradley MM, Cacioppo JT, Ito T, Lang PJ. Affective picture processing: the late positive potential is modulated by motivational relevance. Psychophysiology. 2000;37:257–261. [PubMed] [Google Scholar]
  67. Straube T, Miltner WH. Attention to aversive emotion and specific activation of the right insula and right somatosensory cortex. Neuroimage. 2011;54:2534–2538. doi: 10.1016/j.neuroimage.2010.10.010. [DOI] [PubMed] [Google Scholar]
  68. Tang AC, Sutherland MT, McKinney CJ. Validation of SOBI components from high-density EEG. Neuroimage. 2005;25:539–553. doi: 10.1016/j.neuroimage.2004.11.027. [DOI] [PubMed] [Google Scholar]
  69. Vanderperren K, De Vos M, Ramautar JR, Novitskiy N, Mennes M, Assecondi S, Vanrumste B, Stiers P, Van den Bergh BR, Wagemans J, Lagae L, Sunaert S, Van Huffel S. Removal of BCG artifacts from EEG recordings inside the MR scanner: A comparison of methodological and validation-related aspects. Neuroimage. 2010;50:920–934. doi: 10.1016/j.neuroimage.2010.01.010. [DOI] [PubMed] [Google Scholar]
  70. Wang X, Ding M. Relation between P300 and event-related theta-band synchronization: a single-trial analysis. Clin Neurophysiol. 2011;122:916–924. doi: 10.1016/j.clinph.2010.09.011. [DOI] [PubMed] [Google Scholar]
  71. Wang X, Chen Y, Ding M. Estimating Granger causality after stimulus onset: a cautionary note. Neuroimage. 2008;41:767–776. doi: 10.1016/j.neuroimage.2008.03.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Weinberg A, Hajcak G. Electrocortical evidence for vigilance-avoidance in generalized anxiety disorder. Psychophysiology. 2011;48:842–851. doi: 10.1111/j.1469-8986.2010.01149.x. [DOI] [PubMed] [Google Scholar]
  73. Weymar M, Schwabe L, Löw A, Hamm AO. Stress sensitizes the brain: increased processing of unpleasant pictures after exposure to acute stress. J Cogn Neurosci. 2012;24:1511–1518. doi: 10.1162/jocn_a_00174. [DOI] [PubMed] [Google Scholar]
  74. Xu L, Stoica P, Li J, Bressler SL, Shao X, Ding M. ASEO: a method for the simultaneous estimation of single-trial event-related potentials and ongoing brain activities. IEEE Trans Biomed Eng. 2009;56:111–121. doi: 10.1109/TBME.2008.2008166. [DOI] [PubMed] [Google Scholar]
  75. Zald DH. The human amygdala and the emotional evaluation of sensory stimuli. Brain Res Rev. 2003;41:88–123. doi: 10.1016/s0165-0173(02)00248-5. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES