Convergence of fMRI and ERP measures of emotional face processing in combat-exposed U.S. military veterans

Abstract

The late positive potential (LPP) and functional magnetic resonance imaging (fMRI) blood-oxygen-level dependent (BOLD) activity can provide complementary measures of the processing of affective and social stimuli. Separate lines of research using these measures have often employed the same stimuli, paradigms and samples; however, there remains relatively little understanding of the way in which individual differences in one of these measures relates to the other, and all prior research has been conducted in psychiatrically healthy samples and using emotional scenes (not faces). Here, 32 combat-exposed U.S. military veterans with varying levels of post-traumatic stress symptomatology viewed affective social stimuli (angry, fearful and happy faces) and geometric shapes during separate electroencephalographic and fMRI BOLD recordings. Temporospatial principal component analysis was used to quantify the face-elicited LPP in a data-driven manner, prior to conducting whole-brain correlations between resulting positivities and fMRI BOLD elicited by faces. Participants with larger positivities to fearful faces (> shapes) showed increased activation in the amygdala; larger positivities to angry and happy faces (> shapes) were associated with increased BOLD activation in the posterior fusiform gyrus and inferior temporal gyrus, respectively. Across all face types, larger positivities were associated with increased activation in the fusiform “face” area. Correlations using mean area amplitude LPPs showed an association with increased activation in the anterior insula for angry faces (> shapes). LPP-BOLD associations were not moderated by PTSD. Findings provide the first evidence of correspondence between face-elicited LPP and BOLD activation across a range of (normal to disordered) psychiatric health.

Introduction

Decades of work using functional magnetic resonance imaging (fMRI) has revealed a network of brain regions that show increased activation in response to salient, affective stimuli such as emotional scenes and faces, including in amygdala, visual cortex (fusiform gyrus and lateral occipital cortex), supramarginal gyrus, brainstem, medial frontal cortex, orbitofrontal cortex, inferior frontal gyrus, inferior temporal cortex and extrastriate occipital cortex (García-García et al., 2016; Sabatinelli et al., 2011). As an alternative means of assessing attention to salient stimuli, the late positive potential (LPP) - an event-related potential (ERP) component that begins around 300–500 ms after stimulus onset and is larger for emotional compared to neutral stimuli (Cuthbert, Schupp, Bradley, Birbaumer, & Lang, 2000; Hajcak, MacNamara, & Olvet, 2010; Schupp et al., 2004) – provides a relatively cost-effective, well-tolerated and dynamic measure. Both the LPP and fMRI have been used widely in the basic and clinical affective neuroscience literatures to assess the processing of emotional scenes and faces (e.g., De Taeye et al., 2015; Etkin & Wager, 2007; Foti, Olvet, Klein, & Hajcak, 2010; Fusar-Poli et al., 2009; Gentili et al., 2016; MacNamara et al., 2016; MacNamara, Post, Kennedy, Rabinak, & Phan, 2013; MacNamara, Schmidt, Zelinsky, & Hajcak, 2012). Though fMRI and LPP studies of emotion processing have often used the same stimuli and paradigms, there has – until recently (Liu, Huang, McGinnis-Deweese, Keil, & Ding, 2012; Sabatinelli, Keil, Frank, & Lang, 2013) – been little understanding of how these measures relate to each other. Further, no work to date has yet has examined correspondence between individual differences in the LPP and fMRI BOLD across a range of psychiatric health and disease (studies have been conducted exclusively in healthy individuals) and correlations between fMRI blood-oxygen-level dependent (BOLD) and the LPP have not been assessed in regards to affective faces (prior work has been conducted using emotional scenes).

The earliest studies examining correspondence between the LPP and fMRI BOLD used source modeling to infer the neural generators underlying variation in the LPP. This work suggested that the LPP might originate from the occipital and posterior parietal cortex (Keil et al., 2002). Subsequent to this, studies that recorded both fMRI BOLD response and the LPP in separate sessions showed correspondence between these measures in in lateral occipital, inferior temporal, and parietal visual areas (Sabatinelli, Lang, Keil, & Bradley, 2007). Replicating these results, Sabatinelli and colleagues (2013) found that the LPP and fMRI BOLD correlated in the extrastriate occipital cortex, posterior parietal cortex, inferior temporal visual cortex, as well as in the insula and the anterior cingulate cortex (Sabatinelli et al., 2013). Moreover, this latter work broke new ground by suggesting modulation of the LPP by activation in subcortical regions - the amygdala and the ventral striatum/nucleus accumbens. Although subcortical neural regions are unlikely to make a significant, direct contribution to scalp-elicited ERP amplitudes (i.e., because of their distance from the scalp), these results supported the notion that re-entrant projections from the amygdala to the visual cortex might moderate the LPP and that variation in amygdala-occipital activation may help explain variation in LPP magnitude (Keil et al., 2009; Sabatinelli et al., 2007). Subsequent work using single trial analysis of simultaneously recorded electroencephalography (EEG) and BOLD response also showed that emotional scenes with the largest LPPs were associated with BOLD activation in amygdala, as well as in the occipital, temporal and orbitofrontal cortices and insula (Liu et al., 2012).

Despite this significant progress in understanding fMRI BOLD correlates of within-subject variation in the LPP, there is limited knowledge of how individual differences (i.e., between-subject variation) in the LPP relate to fMRI BOLD. Given evidence that the LPP is altered in a number of psychiatric disorders (e.g., Foti et al., 2010; MacNamara & Hajcak, 2010; MacNamara, Kotov, & Hajcak, 2015; MacNamara et al., 2013; Weinberg & Hajcak, 2011), and given the potential for ERPs to translate to the clinic (i.e., increased cost-effectiveness and ease of administration compared to fMRI), a better understanding of correspondence between fMRI and ERP measures across the psychiatric continuum is desirable. In addition, despite the widespread use of facial stimuli in fMRI (and ERP) research, no prior work has examined correlations between the LPP and fMRI BOLD elicited by faces.

Therefore, the current study assessed correlations between the LPP and BOLD activation in a group of U.S. military veterans returning from Iraq and Afghanistan. Although simultaneous EEG-fMRI has its advantages, we used separately recorded EEG and fMRI sessions, allowing for better signal-to-noise ratio and optimization of paradigms to each modality (Wibral, Bledowski, & Turi, 2010). Many of the participants in our sample suffered from elevated symptoms of anxiety, depression and post-traumatic stress, and a subset met criteria for a diagnosis of PTSD. These sample characteristics allowed us to examine LPP-BOLD correlations across a wide range of psychiatric health and disease. Moreover, we also examined moderation of LPP-BOLD correlations across the sample using PTSD diagnosis and PTSD symptom scores. Because the LPP is comprised of a series of positivities that overlap in space and time (Foti, Hajcak, & Dien, 2009) and because decomposition of ERPs into their underlying components may facilitate better correspondence with fMRI BOLD activation (Carlson, Foti, Mujica-Parodi, Harmon-Jones, & Hajcak, 2011), we performed a temporospatial principal components analysis (PCA) to quantify the LPP prior to performing correlations with fMRI BOLD. For comparison with prior work, we also measured the LPP using the more standard procedure of averaging amplitudes across a given time window and pool of electrodes.

Prior work has suggested that the LPP may correspond to increased activation in visual regions that are sensitive to the emotional nature of stimuli, including the occipital, posterior parietal, and inferior temporal visual cortices, and that these activations may be driven by re-entrant projections from the amygdala (Keil et al., 2009; Liu et al., 2012; Sabatinelli et al., 2013, 2007). Therefore, we expected to observe significant positive correlations between the LPP and fMRI BOLD in these visual and subcortical regions, as well as in the amygdala and insula. Because this is the first study to examine correspondence between the LPP and BOLD activation to faces, we also tested the hypothesis that we would observe correlations between the LPP and the fusiform “face” area. We did not expect to observe modulation of LPP-BOLD correlations by PTSD symptoms or diagnosis; that is, although PTSD may be associated with overall increases or decreases in the LPP or fMRI BOLD in particular brain regions (Etkin & Wager, 2007; MacNamara et al., 2013), we hypothesized that the relationship between the LPP and BOLD should remain unchanged across individuals. Instead of extracting BOLD response from pre-defined ROIs and correlating it post-hoc with ERPs, we took a more data-driven approach by entering the LPP as a whole-brain predictor of BOLD. In order to focus our investigation on those regions we thought most likely to correlate with the LPP, while meeting the challenge of correcting for multiple comparisons in fMRI data, we thresholded results of these second-level analyses using a mask encompassing all regions previously found to correlate with the LPP (Liu et al., 2012).

Method

Participants

Participants were 32 U.S. military veterans who had participated in combat in Iraq or Afghanistan. The participation of these individuals in a larger protocol made available both EEG and fMRI measures in the same individuals using the same task. The average age of participants was 32 years (SD = 8.70). The majority of participants were white (n = 30, 93.8%); one participant (3%) was African American and one participant (3%) was Asian. Average education level was 14.09 years (SD = 1.97) and all participants were male. Several participants had current psychiatric diagnoses including 18 with PTSD, 4 with comorbid MDD and 2 with comorbid alcohol abuse, according to the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders, DSM-IV (SCID-NP; First, Spitzer, Gibbon, & Williams, 2002). Past disorders included alcohol abuse (n = 11), alcohol dependence (n = 4), panic disorder (n = 1) and MDD (n = 3). PTSD symptomatology and severity was measured using the Clinician Administered PTSD Scale (CAPS; Blake et al., 1995); on average, participants had a CAPS score of 41.94 (SD = 36.18), with scores ranging from 0 to 95. None of the participants had a history of a major medical or neurological illness or a history of traumatic brain injury; other exclusionary criteria included a history of bipolar disorder, psychotic disorder, mental retardation, or developmental disorders. All participants were right-handed and were free from psychotropic medications for at least 8 weeks at the time of study entry, with negative urine toxicology screens at the time of scanning. Study procedures were in compliance with the Helsinki Declaration of 1975 (as revised in 1983), and were approved by the University of Michigan and VA Ann Arbor Healthcare System institutional review boards.

Task

Participants completed a version of the Emotional Face-Matching Task (Hariri et al., 2002) previously validated for use with both the LPP (Kujawa, MacNamara, Fitzgerald, Monk, & Phan, 2015; MacNamara et al., 2016, 2013) and fMRI BOLD (Fitzgerald, Angstadt, Jelsone, Nathan, & Phan, 2006; Gorka, Fitzgerald, et al., 2015; Hariri, Mattay, Tessitore, Fera, & Weinberger, 2003). Participants completed the task twice, once during in the fMRI scanner and once during an EEG recording, with the fMRI recording always completed first. angry, fearful and happy faces were selected from the Gur emotional faces set (Gur et al., 2002); half of the faces depicted male actors and the other half depicted female actors. All images were presented on a black background using Presentation software (Neurobehavioral Systems, Inc., Albany, CA). On each trial, participants viewed three images – one at the top of the screen and two at the bottom of the screen; two of the faces were emotional and one always bore a neutral expression. Participants were instructed to select the face at the bottom of the screen that bore the same emotional expression as the ‘target’ face centered in the top portion of the screen. Some trials were shape-matching trials, on which participants were instructed to choose the geometric shape at the bottom of the screen that matched (i.e., had the same form as) the target shape at the top of the screen. In line with previous studies using this task (Labuschagne et al., 2010; Phan et al., 2008) geometric shapes were used as control stimuli instead of neutral faces, because neutral faces may be more influenced by individual differences (e.g., anxiety levels; Somerville, Kim, Johnstone, Alexander, & Whalen, 2004). Of note, although images were not explicitly matched on perceptual characteristics such as brightness, the LPP has been shown to be relatively “immune” to these effects (Bradley, Hamby, Löw, & Lang, 2007; De Cesarei & Codispoti, 2011; Miskovic et al., 2015). Therefore, although differences in brightness (or other low-level perceptual features) may have modulated early ERPs, it is unlikely that affective modulation of the LPP was attributable solely to these low-level perceptual differences.

As in our previous publications assessing the LPP using this task, subjects were allowed to shift their gaze during trials (Kujawa et al., 2015; MacNamara et al., 2016, 2013). However, eye movements (or correction for eye movements) is unlikely to have affected results because a) the LPP is a midline component (saccades tend to create artifacts in channels at the sides of the head - -e.g., F7/F8); b) LPP reliability has been shown to be excellent when using other paradigms in which participants were allowed to shift their gaze freely (and when using the same ocular correction procedure as in the current manuscript and as few as 12 trials; Moran, Jendrusina, & Moser, 2013) and c) the ocular correction procedure we employed is known to negate the need for participants to restrict eye movement activity (Gratton, Coles, & Donchin, 1983).

fMRI and EEG versions of the tasks were optimized for each measure. In the fMRI version of the task, participants performed 2 runs, with each run consisting of 3 angry, 3 fearful and 3 happy blocks of trials, interspersed with shape-matching blocks as control stimuli. Each block lasted 20 s and consisted of 4 back-to-back 5-s trials; there were a total of 6 blocks per condition across both runs. The EEG version of the task consisted of two blocks, each with 12 angry, 12 fearful, 12 happy and 12 shape-matching trials; trials were presented randomly within each block, for a total of 96 trials across both blocks. Stimuli were presented for 3 s and the inter-trial interval varied between 1 s and 3 s, during which time a white fixation cross was centrally presented on a black background.

EEG Recording and Data Reduction

Continuous EEG was recorded using an elastic cap and the ActiveTwo BioSemi system (BioSemi, Amsterdam, Netherlands). Thirty-four electrode sites (standard 32 channel setup, as well as FCz and Iz) were used, based on the 10/20 system. The electrooculogram (EOG) generated from eyeblinks and eye movements was recorded from four facial electrodes: vertical eye movements and blinks were measured with two electrodes placed approximately 1 cm above and below the right eye; horizontal eye movements were measured using two electrodes placed approximately 1 cm beyond the outer edge of each eye. The EEG signal was pre-amplified at the electrode to improve the signal-to-noise ratio. The data were digitized at 24-bit resolution with a Least Significant Bit (LSB) value of 31.25 nV and a sampling rate of 1024 Hz, using a low-pass fifth order sinc filter with a −3dB cutoff point at 208 Hz. The voltage from each active electrode was referenced online with respect to a common mode sense active electrode producing a monopolar (non-differential) channel.

Off-line analyses of EEG data were performed using Brain Vision Analyzer 2 software (Brain Products, Gilching, Germany). Data from correct trials were segmented for each trial beginning 200 ms prior to picture onset and continuing for 3.2 s (3 s beyond picture onset); baseline correction for each trial was performed using the 200 ms prior to picture onset. Offline, data were re-referenced to the average of all electrodes and band-pass filtered with high-pass and low-pass filters of 0.01 and 30 Hz, respectively. Eye blink and ocular corrections used the method developed by Miller, Gratton and Yee (1988). Artifact analysis was used to identify a voltage step of more than 50.0 μV between sample points, a voltage difference of 300.0 μV within a trial, and a maximum voltage difference of less than 0.50 μV within 100 ms intervals. Trials were also inspected visually for any remaining artifacts, and data from individual channels containing artifacts were rejected on a trial-to-trial basis.

Following published recommendations for the use of PCA with ERPs (Joseph Dien, 2010b), a temporal PCA was first performed on the data, using all 3,277 time points per trial condition average (1024 samples per second multiplied by a total trial-plus-baseline length of 3.2 s) as variables and considering as observations all 32 participants, 34 channels and four conditions. The temporal PCA was followed by a spatial PCA that used recording sites (electrodes) as variables and all participants, conditions, and temporal factor scores as observations. In order to directly assess timing and spatial voltage distributions, the factors were translated back into voltages. The PCA was conducted using the ERP PCA (EP) Toolbox (version 2.40; Joseph Dien, 2010a) for MatLab (The MathWorks Inc., Natick, MA) and the covariance matrix and Kaiser normalization (as per Dien, Beal & Berg’s, 2005 suggestions).

For comparison with prior work, we also scored the LPP using the more standard procedure of averaging area amplitudes across a pre-defined time window and group of electrodes. Based on visual inspection of waveforms and topographic maps, and in line with prior research (e.g., MacNamara et al., 2013), the LPP was scored by averaging amplitudes at pooling, CP1, CP2, Cz and Pz from 500–3,000 ms after picture onset (i.e., lasting the full duration picture presentation).

Functional MRI Data Acquisition and Analysis

Functional MRI based on BOLD contrast was performed on a 3T GE Signa System (General Electric; Milwaukee, WI) using a standard radiofrequency coil at the University of Michigan Functional MRI Laboratory. A T2*-sensitive gradient-echo echo reverse spiral sequence was used (Glover & Law, 2001) - 2s TR; 30 ms TE; 90° flip; 64×64 matrix; 22 cm FOV; 43 axial, 3 mm slices; 360 volumes. Data from all participants included in analyses met our criteria for image quality with minimal motion correction (all movements ≤ 2 mm in any direction across functional runs)¹.

The first 4 volumes from each run were discarded to allow for the magnetization to reach equilibrium. Statistical Parametric Mapping (SPM 8) software (Wellcome Trust Centre for Neuroimaging, London, www.fil.ion.ucl.ac.uk/spm) was used to perform conventional preprocessing steps. In brief, slice-time correction was performed to account for temporal differences between slice collection order, images were spatially realigned to the first image of the first run, functional images were normalized to a Montreal Neurological Institute (MNI) template using the EPI template, resampled to 2 mm³ voxels and smoothed with an 8 mm isotropic Gaussian kernel.

The time series data were subjected to a general linear model, convolved with the canonical hemodynamic response function and filtered with a 128 s high-pass filter. Conditions of interest were angry, fearful, happy and shapes trials, which were modeled separately, with effects estimated for each voxel for each participant. Individual motion parameters were entered in the model as covariates of no interest. Following processing at the first-level, angry > shapes, fearful > shapes, happy > shapes and all faces > shapes contrasts, created separately for each participant, were taken to the second level for random effects analysis.

Correlations between the LPP and fMRI BOLD

At the second level, we examined brain activity that covaried with individual differences in the LPP. Voltages corresponding to each of four PCA factors (at peak channels²; see Results, below) representing overlapping positivities underlying the LPP and capturing individual differences in ERP responding were entered as covariates of interest in separate one-sample t-tests examining whole-brain activity for each of angry > shapes, fearful > shapes, happy > shapes and all faces > shapes contrasts. In separate analyses, LPP mean area amplitudes were used as covariates of interest to predict BOLD activation. The results of these analyses yielded statistical maps of regional brain activity (one per face type and PCA factor; one per face type for mean area amplitudes) that occurred during face processing and correlated with individual differences in the LPP.

We thresholded results of these second-level analyses using a mask (volume = 280.46 cm³) encompassing all regions previously found to correlate with the LPP. Using coordinates from Liu and colleagues (2012), regions (all bilateral) were defined by anatomical landmarks using MARINA software (Walter et al., 2003) based on masks from the atlas of Tzourio-Mazoyer and colleagues (Tzourio-Mazoyer et al., 2002), and included: amygdala, insula; middle, inferior and superior temporal gyrus; middle and superior occipital gyrus; fusiform gyrus and hippocampus. For this covariation analysis, in which we used the LPP to predict BOLD, we tested the a priori hypotheses that the LPP would correlate positively with activity in occipital, posterior parietal, insula and inferior temporal visual cortex as well as the amygdala (Liu et al., 2012; Sabatinelli et al., 2007). Because we used faces, we also hypothesized that the LPP might correlate with activation in the fusiform “face” area.

Clusters of activation that covaried with the LPP were initially identified using an uncorrected voxel threshold of p < .001 (Eklund, Nichols, & Knutsson, 2016), and then subjected to correction for multiple comparisons within the ROI mask as determined via simulation using the 3dClustSim utility (http://afni.nimh.nih.gov/pub/dist/doc/program_help/3dClustSim.html; Dec. 16, 2015 updated release, Eklund et al., 2016; 10,000 iterations). Given smoothness estimates of the data within the mask, an FWE correction at α < 0.05 was achieved using a voxel threshold of p < .001 with a minimum cluster size of 25 voxels (fearful > shapes), 27 voxels (angry > shapes), 30 voxels (happy > shapes) or 29 voxels (all faces > shapes). To clarify the range in values as well as the direction of significant results from each analysis, we extracted BOLD signal responses (parameter estimates, β-weights in arbitrary units, a.u., of activation), averaged across all voxels within a 5 mm radius sphere surrounding each peak maxima voxel. To determine whether associations between PCA factors and BOLD activation varied according to PTSD diagnosis or symptoms (as measured by the CAPS), we performed a series of regression analyses in SPSS 23.0 (IBM, Chicago, Illinois).

Results

Quantification of the Late Positive Potential using Principal Components Analysis

Grand average waveforms (prior to PCA) are presented in Figure 1. In line with prior work (e.g., Cuthbert et al., 2000; Foti et al., 2009; MacNamara, Foti, & Hajcak, 2009), the LPP was evident by ~500 ms after stimulus onset at centro-partial recording sites.

Figure 1.

The LPP (prior to PCA): a) Spatial distribution of the faces > shapes voltage difference between 500–3,000 ms post-picture onset; b) Grand-average waveforms (at pooling CPz, CP1, CP2 and Pz) depicting the LPP for angry, fearful and happy faces as well as shapes (shown here using a low-pass 12 Hz filter).

The temporal PCA yielded 37 temporal factors based on the resulting Scree plot (Cattell, 1966; Cattell & Jaspers, 1967). These were submitted to Promax rotation (the preferred rotation for this step according to simulation results by Dien, Khoe & Mangun, 2007). Following this, a spatial PCA was performed on each temporal factor and Infomax was used to rotate to independence in the spatial domain (as per Dien et al.’s, 2007 simulations). Four spatial factors were extracted for each temporal factor, yielding a total of 148 temporospatial factor combinations. Of these, 11 factors accounted for more than 1% of the variance each and were retained for further examination.

Consistent with prior research, the PCA revealed a number of overlapping occipital and central positivities³ corresponding to the LPP (Foti et al., 2009; MacNamara et al., 2009). Because PCA is a data-driven approach that is “blind” to the spatial and temporal characteristics of ERPs, recommendations are to make a final selection of components based on knowledge of the literature/prior research and components of interest for a given study (J. Dien, Beal, & Berg, 2005). Therefore, based on their temporal and spatial similarity with previous PCA work on early and late positivities (Foti et al., 2009; MacNamara et al., 2009), six factors were selected for further statistical analysis. A repeated measures ANOVA with the factor condition (angry faces, fearful faces, happy faces, shapes) was conducted on factor amplitudes from the six temporospatial factors. Using Bonferroni correction for multiple comparisons (p = .05/6 = .008), we identified four factors that were sensitive to the effect of condition.

Figure 2 presents the spatial distribution of the all faces minus shapes voltage difference (i.e., topographic maps, in μV) for each of these four factors, as well as waveforms at peak channels for each factor. Because the LPP begins around 400 ms after picture onset; we refer to PCA components that peaked near this time as “early”, those that peaked near the middle of picture presentation “mid” latency and those that occurred near the end of picture presentation as “late”. Specifically, we identified: two early occipital positivities that peaked at 467 ms (LPP-early-occipital_1 in Figure 2a and LPP-early-occipital_2 in Figure 2b); a mid-latency frontal positivity that peaked at 1,041 ms (LPP-mid-frontal in Figure 2c) and a late central positivity that peaked at 2,692 ms (LPP-late-central in Figure 2d). The significant main effect of condition on these factors is evident in Figure 2, with faces eliciting larger (more positive) factor waveforms than shapes. Voltage differences corresponding to angry minus shapes, fearful minus shapes, happy minus shapes and all faces minus shapes were created from peak channels for each of the four PCA factors, and entered as covariates of interest in separate one-sample t-tests examining whole-brain activity for each of angry > shapes, fearful > shapes, happy > shapes and all faces > shapes BOLD contrasts.

Figure 2.

PCA factors significantly modulated by condition: a) Location of the faces > shapes voltage difference for PCA factor LPP-early-occipital_1 (left) and waveforms (at electrode Iz) for angry, fearful and happy faces as well as shapes (right); b) Location of the faces > shapes voltage difference for PCA factor LPP-early-occipital_2 (left) and waveforms (at electrode Iz) for angry, fearful and happy faces as well as shapes (right); c) Location of the faces > shapes voltage difference for PCA factor LPP-mid-frontal (left) and waveforms (at electrode Fz) for angry, fearful and happy faces as well as shapes (right); d) Location of the faces > shapes voltage difference for PCA factor LPP-late-central (left) and waveforms (at electrode Cz) for angry, fearful and happy faces as well as shapes (right).

Correlations between the LPP and fMRI BOLD Response

Table 1 presents areas of activation that surpassed our threshold for correlation with PCA factor amplitudes, shown separately for each factor and face type, as well as for all faces > shapes. Within our a priori regions, there was a positive correlation between fearful > shapes activation in the left amygdala and LPP-early-occipital_1 amplitude. Follow-up correlations using ROI-extracted BOLD signal (β weights) confirmed that participants with the largest fearful > shapes amplitudes showed the greatest BOLD activation in the amygdala (Figure 3a). For angry > shapes, a positive correlation was observed between activation in a priori region, the occipital cortex – specifically, the posterior fusiform gyrus – and PCA-derived factor, LPP-early-occipital_1. Follow-up correlations confirmed that participants with the largest angry > shapes amplitudes showed the greatest BOLD activation in this region (Figure 3b). For happy > shapes, LPP-early-occipital_1 amplitudes correlated positively with BOLD activation in the inferior temporal cortex (Figure 3c). There was also a positive correlation between all faces > shapes activation in the left (anterior) fusiform gyrus and LPP-early-occipital_1 amplitudes. Follow-up correlations confirmed that participants with the largest all faces > shapes amplitudes showed the greatest BOLD activation in this region (Figure 3d). No other correlations (with any factor) within our a priori regions surpassed our cutoff for statistical significance⁴.

Table 1.

Correlations between BOLD activation and LPP PCA components to faces.

	Brain Region	Volume mm3	Z-score	MNI Coordinates
				x	y	z
Fearful > shapes
LPP-early-occipital_1
	Cerebellum	840	4.57	−12	−36	−18
	Amygdala	312	3.85	−30	2	−22
Angry > shapes
LPP-early-occipital_1
	Fusiform Gyrus	352	3.84	48	−60	−18
LPP-early-occipital_2
	Middle Temporal Gyrus	416	3.59	52	−62	24
			3.42	52	−52	20
Happy > shapes
LPP-early-occipital_1
	Inferior Temporal Gyrus	248	3.6	38	−4	−42
All faces > shapes
LPP-early-occipital_1
	Fusiform Gyrus	304	4.24	−30	−32	−22
LPP-early-occipital_2
	Middle Temporal Gyrus	472	3.92	52	−64	26

All clusters significant at a voxel-wise threshold of p < 0.05, corrected, based on 3dClustSim (Dec. 16, 2015 updated release; Eklund et al., 2016). A priori ROIs shown in bold; MNI, Montreal Neurological Institute.

Figure 3.

Location of correlations between the LPP and BOLD activation for: a) fearful > shapes amygdala activation [−30, 2, −22] and PCA factor LPP-early-occipital_1; b) angry > shapes posterior fusiform gyrus activation [48, −60, −18] and LPP-early-occipital_1; c) happy > shapes inferior temporal gyrus activation [38, −4, −42] and LPP-early-occipital_1; d) all faces > shapes and fusiform “face” area activation [−30, −32, −22] and LPP-early-occipital_1 and e) angry > shapes insula activation [32, 4, 6] and mean area amplitude LPP. Threshold for all images is set at p < .001.

Table 2 presents areas of activation that surpassed our threshold for correlation with the LPP, as measured using mean area amplitudes from 500–3000 ms post-stimulus onset. Within our a priori regions, there was a positive correlation between angry > shapes activation in the anterior insula and the LPP. Follow-up correlations using ROI-extracted BOLD signal (β weights) confirmed that participants with the largest angry > shapes amplitudes showed the greatest BOLD activation in the insula (Figure 3e).

Table 2.

Correlations between BOLD activation and mean area amplitude LPP to faces.

	Brain Region	Volume mm3	Z-score	MNI Coordinates
				x	y	z
Angry > shapes
	Insula	3704	4.06	32	4	6
			3.69	48	30	10
			3.58	38	14	16
	Superior Temporal Gyrus	496	3.95	−62	−40	4

All clusters significant at a voxel-wise threshold of p < 0.05, corrected, based on 3dClustSim (Dec. 16, 2015 updated release; Eklund et al., 2016). A priori ROIs shown in bold. MNI, Montreal Neurological Institute.

To determine whether associations between the LPP and BOLD activation varied according to PTSD diagnosis, we created the interaction term, PTSD diagnosis X fearful > shapes LPP (early-occipital_1) to predict activity in the left amygdala. Similarly, we also created the interaction term, PTSD diagnosis X angry > shapes LPP (early-occipital_1) to predict activity in the posterior fusiform gyrus, PTSD diagnosis X happy > shapes LPP (early-occipital_1) to predict activity in the inferior temporal gyrus, PTSD diagnosis X all faces > shapes LPP (early-occipital_1) to predict activity in the fusiform “face” area and PTSD diagnosis X angry > shapes LPP (mean area amplitude) to predict activity in the insula. To assess associations with PTSD symptomatology, interaction terms were created for CAPS total scores X Fear > shapes LPP (early-occipital_1), CAPS total scores X angry > shapes LPP (early-occipital_1), CAPS total scores X happy > shapes LPP (early-occipital_1), CAPS total scores X all faces > shapes LPP (early-occipital_1) and CAPS total scores X angry > shapes LPP (mean area amplitude). None of these terms significantly predicted BOLD activation (all ps > .09), indicating that associations with the LPP were not moderated by PTSD diagnosis or symptoms.

Discussion

The current study set out to identify where brain activation covaried with individual differences in the LPP elicited by emotional faces in a group of U.S. military veterans, several of whom suffered from elevated symptoms of PTSD. Results showed that participants with larger early (467 ms) occipital positivities to fearful faces showed increased activation in the amygdala to these same stimuli. In addition, participants with larger early positivities to angry faces showed increased activation in the posterior fusiform gyrus, and those with larger positivities to happy faces showed increased activation in the inferior temporal gyrus. Larger positivities to faces in general (i.e., all faces > shapes) were associated with increased activation in the fusiform “face” area. Parallel analyses using mean area amplitudes instead of PCA factors revealed that larger LPPs to angry faces were associated with increased BOLD activation in the anterior insula. The results extend prior work, which has examined BOLD correlates of trial-level or picture-level fluctuations in the LPP elicited by emotional scenes (Liu et al., 2012; Sabatinelli et al., 2013). Here, results show that individual differences in the LPP when processing facial stimuli correspond to activation in discrete brain areas implicated in emotion and visual processing (e.g., Bradley et al., 2003). Results also indicate that differences in the way the LPP is quantified (e.g., PCA versus mean area amplitudes) can affect correlations with fMRI BOLD. Additionally, a diagnosis of PTSD and/or PTSD symptomatology does not seem to modulate associations between the LPP and fMRI BOLD in these regions (despite evidence for an effect of PTSD on BOLD activation and the LPP to affective faces; e.g., MacNamara et al., 2013; Rabellino, Densmore, Frewen, Théberge, & Lanius, 2016).

Although subcortical regions such as the amygdala, which is located deep within the brain, are unlikely to contribute significantly to amplitudes measured at the scalp, the LPP may be modulated by re-entrant projections from the amygdala to the visual cortex (e.g., Bradley et al., 2003). To date, two studies, both using emotional scenes, have supported that notion that within-subject, trial-to-trial fluctuations and across-subject, picture-specific variations in the LPP might be driven in part by amygdala activation (Liu et al., 2012; Sabatinelli et al., 2013). Here, the between-subjects correlation (using an early, occipital PCA factor) suggested that variation in amygdala activation may drive individual differences in the magnitude of the LPP elicited by fearful faces. Like the amygdala, the insula is thought to be involved in emotion generation, and in particular, the subjective experience of negative affect (e.g., Paulus & Stein, 2006). The correlation observed here between the LPP elicited by angry faces (as measured using mean area amplitudes) and insula activation is in line with prior work (Liu et al., 2012; Sabatinelli et al., 2013), and supports the notion of the LPP as a measure of the conscious and elaborative processing of stimulus meaning (Schupp, Flaisch, Stockburger, & Junghöfer, 2006).

In considering these results, it is evident that we observed different correlations depending on the way in which the LPP was scored (PCA versus mean area amplitude). One reason may be that PCA parses the LPP into underlying components, which may be differentially sensitive to particular stages of emotion-processing, whereas mean area quantification of the LPP is more likely to collapse across these stages, given that it is not data-driven and does not isolate variance that “hangs together” temporally and spatially. For example, prior work has shown that PCA components underlying the LPP are differentially sensitive to intrinsic or extrinsic experimental effects, with extrinsic effects modulating later PCA components in particular (MacNamara et al., 2009). Here, an early, occipital PCA component peaking around 500 ms post-stimulus onset was found to be correlated with BOLD activation in the amygdala, perhaps reflecting the amygdala’s role in the rapid identification and early processing of threatening stimuli (e.g., McFadyen, Mermillod, Mattingley, Halász, & Garrido, 2017). Using mean area amplitudes (which extended until stimulus offset), the LPP was correlated with activation in the insula - a brain region implicated in the conscious awareness and anticipation of threat (Critchley, Mathias, & Dolan, 2002; Phelps et al., 2001). Therefore, different results observed using PCA and LPP mean area amplitudes may highlight different stages or aspects of stimulus processing, which in turn may be associated with activation in different brain regions. Nonetheless, given the many differences between face and shape stimuli, associations between BOLD and electrocortical amplitudes might also reflect social, attentional or arousal effects more generally (e.g., instead of or in addition to emotion) – a possibility that is amplified by the relatively early latency of the PCA component (i.e., in the time range of the P300).

Amygdala activation (e.g., Killgore et al., 2013; Shah, Klumpp, Angstadt, Nathan, & Phan, 2009; Siegle, Thompson, Carter, Steinhauer, & Thase, 2007; Simmons et al., 2011), insula activation (Paulus & Stein, 2006) and the LPP (e.g., Foti et al., 2010; Kujawa et al., 2015; MacNamara & Hajcak, 2010; MacNamara, Kotov, et al., 2015; MacNamara et al., 2013) elicited by unpleasant stimuli may be aberrant in various anxiety, depressive and trauma-related psychopathologies. Here, diagnosis or symptoms of PTSD did not moderate associations between the LPP elicited by faces and fMRI BOLD response in any of the BOLD regions identified. Therefore, coupling between these regions and the LPP does not seem to be stronger at one end of the psychiatric continuum (i.e., from health to disease); that is, there is a set of common neural regions that appear to underlie potentiation of the LPP among both psychiatrically healthy and disordered individuals. These results may inform mechanistic understanding of trauma-related and other internalizing psychopathologies, and are in line with the notion that psychiatric health is dimensional, with the same biological systems at play across a range of symptomatology (Cuthbert, 2013). Knowledge of how individual differences in BOLD activation relate to the LPP across the psychiatric spectrum may facilitate the use of EEG as a viable index of BOLD (e.g., Keynan et al., 2016), paving the way for clinicians to make biologically-informed diagnostic or treatment decisions their offices (Stange et al., in press). Nonetheless, we caution that because we only examined moderation of LPP-BOLD associations in a limited number of neural regions (and because we observed a null effect of diagnosis and symptoms), we cannot say for certain that other brain regions do not contribute disproportionately to individual differences in the LPP at one end of the psychiatric spectrum. In addition, future work may wish to assess moderation of ERP-BOLD associations by other diagnoses and symptoms.

Prior work has suggested that the LPP reflects enhanced activation in higher-order visual areas (i.e., the extrastriate and lateral occipital cortex, inferior temporal visual cortex). Here, we observed a positive correlation between positivities elicited by angry faces and activation in the posterior fusiform gyrus, as well as a positive correlation between amplitudes elicited by happy faces and activation in a more anterior region of the inferior temporal gyrus (corresponding approximately to Brodmann area 20). The posterior fusiform gyrus, also located in the inferior temporal (and occipital) cortex, is distinct from more anterior regions of the fusiform gyrus associated with face processing (Haxby, Hoffman, & Gobbini, 2000; Kanwisher, McDermott, & Chun, 1997). According to meta-analytic work (Sabatinelli et al., 2011), emotional scenes activate more lateral occipital regions involved in the processing of pictorially complex stimuli (Lerner, Hendler, Ben-Bashat, Harel, & Malach, 2001), whereas emotional faces appear to activate more inferior regions of the occipital and temporal cortex, such as the posterior fusiform gyrus (Sabatinelli et al., 2011). Therefore, while prior work using emotional scenes has localized LPP-BOLD correlations to middle and lateral occipital cortex (Liu et al., 2012; Sabatinelli et al., 2013), the results observed here (i.e., localization to more inferior regions) might be explained by different patterns of activation elicited for emotional faces.

When collapsing across all face types (i.e., faces > shapes), LPP-early-occipital_1 correlated with activation in the anterior fusiform “face” area. Therefore, individuals with larger early positivities to faces showed enhanced activation in this key brain region implicated in the recognition and processing of facial stimuli. Perhaps because prior work used emotional scenes instead of faces (Liu et al., 2012; Sabatinelli et al., 2013), the LPP was not associated with BOLD in this region. On the whole, substantial overlap between the current and prior results was observed (e.g., amygdala, insula, visual cortex, inferior temporal gyrus); however, the LPP may additionally correlate with activation in different brain regions depending on the type of stimuli used.

Another difference with prior work is that trial-level coupling between the LPP and BOLD activation in the amygdala was driven by pleasant pictures, whereas the current study revealed LPP-BOLD correlations in the amygdala for fearful faces. In addition, Liu and colleagues (Liu et al., 2012) found that pleasant pictures did not activate the amygdala in a traditional BOLD-only contrast. Here, however, happy faces did activate the amygdala (at p < .05, FWE corrected; results not shown). Therefore, although the overall response in the amygdala was substantial, it did not relate linearly to the LPP in the current study. One possibility is that there was limited individual variability (i.e., restricted range) in amygdala response to happy faces, which would be in line with ceiling effects observed for some other measures when participants view happy faces – i.e., high accuracy rates with little variation for happy faces using this and other tasks (Calvo & Nummenmaa, 2015; Labuschagne et al., 2010). In addition, differences between the present results and prior work may reflect a combination of the different types of analyses used and the wider range of psychiatric symptomatology in the present sample.

The face matching task used here has been widely used in both the fMRI (e.g., Burkhouse et al., 2016; Hariri, Tessitore, Mattay, Fera, & Weinberger, 2002; Holz et al., 2017; Prater, Hosanagar, Klumpp, Angstadt, & Phan, 2013) and LPP (Kujawa et al., 2015; MacNamara et al., 2016, 2013) literatures, with shapes commonly employed as a control condition (as in the current study). As such, results reported here represent some of the first “crosstalk” between these otherwise relatively independent literatures. Nonetheless, using other tasks, it is also common within the fMRI literature to use happy faces as a control condition (e.g., Hall et al., 2014; Johnstone, van Reekum, Oakes, & Davidson, 2006). When examining correlations using fearful > happy and angry > happy contrasts in the current study, we failed to observe any significant results. This raises the question as to whether our findings with shapes as a control condition might simply reflect perceptual differences between faces and shapes (i.e., as opposed to increased motivational salience or emotionality or faces). Convergent evidence indicates that unlike earlier ERP components, the LPP is “immune” to low-level perceptual differences between stimuli (Bradley et al., 2007; De Cesarei & Codispoti, 2011; Miskovic et al., 2015). Additionally, the LPP is moderated primarily by arousal, rather than valence (Weinberg & Hajcak, 2010), which could in part explain our null findings for angry > happy and fearful > happy. However, the multitude of differences between faces and shapes means that we cannot draw strong conclusions about what the associations between BOLD and the LPP mean.

By finding evidence of LPP-BOLD correlations in higher order visual areas and the amygdala, and in the insula (when using mean area amplitudes) the current study extends trial-level and picture-level work conducted using emotional scenes, and validates conceptualizations of the LPP – elicited here by emotional faces - as an index of enhanced sensory processing, reflective of increased motivated attention. Importantly, the correlations observed here suggest that some of the same neural regions previously implicated in within-subject fluctuations in the LPP – i.e., the posterior fusiform gyrus and the amygdala – may also correspond to individual differences in the LPP, and that these associations are not moderated by the presence of significant psychiatric symptomatology.

Limitations

The current study has several limitations. 1) Given that all of our results involved faces > shape contrasts, it is possible that findings are related to perceptual complexity, salience, social cognition and/or attention. To avoid these potential confounds, future work may wish to use neutral faces instead of geometric shapes as a control condition. 2) Another limitation concerns order effects – for all participants, the fMRI session was recorded before the EEG session, raising the issue of memory effects, which could have amplified the LPP (e.g., Dolcos & Cabeza, 2002; Schaefer, Pottage, & Rickart, 2011), but nonetheless should have been attenuated somewhat by our use of difference scores to isolate face-specific effects. 3) To constrain our analysis and to threshold data, we used a mask derived from prior work that had identified brain regions that correlated with the LPP (Liu et al., 2012). However, this study used emotional scenes (not faces), involved trial-level analysis and an employed an unselected sample. 4) We employed an all-male sample, which, 5) while relatively large for a combined EEG-fMRI study, may still have been underpowered to detect effects in some brain regions, or as related to psychiatric symptomatology.

Conclusions

In sum, the results fit within a body of prior work aimed at identifying correspondence between individual differences in regional brain activity and electrocortical response (e.g., monetary reward; Carlson et al., 2011; Gorka, Phan, & Shankman, 2015), and serve as an initial step towards using ERP and fMRI measures more effectively in conjunction to improve understanding of the full continuum of individual differences in emotional face processing.