The neural sources of N170: Understanding timing of activation in face-selective areas

Abstract

The N170 ERP component has been widely identified as a face-sensitive neural marker. Despite extensive investigations conducted to examine the neural sources of N170, there are two issues in prior literature: (a) few studies used individualized anatomy as head model for the cortical source analysis of the N170, and (b) the relationship between the N170 and face-selective regions from fMRI studies is unclear. Here, we addressed these questions by presenting pictures of faces and houses to the same group of healthy adults and recording structural MRI, fMRI, and high-density ERPs in separate sessions. Source analysis based on the participant’s anatomy showed that the middle and posterior fusiform gyri were the primary neural sources for the face-sensitive aspects of the N170. Source analysis based on regions of interest from the fMRI revealed that the fMRI-defined fusiform face area was the major contributor to the N170. The current study suggests that the fusiform gyrus is a major neural contributor to the N170 ERP component and provides further insights about the spatiotemporal characteristics of face processing.

1. INTRODUCTION

Face perception plays an important role in our daily social interactions. fMRI studies have revealed a core system consisting of three temporal-occipital face-selective regions: the lateral inferior occipital cortex (the occipital face area—OFA), the middle and posterior fusiform gyrus (the fusiform face area—FFA), and the posterior superior temporal sulcus (pSTS face area—pSTS-FA; for review, see Bernstein & Yovel, 2015; Duchaine & Yovel, 2015; Freiwald, Duchaine, & Yovel, 2016; Haxby, Hoffman, & Gobbini, 2000; Kanwisher & Yovel, 2006; Müller, Höhner, & Eickhoff, 2018). The activity of these areas is integrated with the activity of an extended neural system that processes additional information of faces (e.g., identity, speech-related movements, eye gaze, facial expression; Gschwind, Pourtois, Schwartz, Van De Ville, & Vuilleumier, 2011; Haxby et al., 2000; Moeller, Freiwald, & Tsao, 2008; Pyles, Verstynen, Schneider, & Tarr, 2013). A recent model for face processing suggests the existence of two separate but interacting pathways: a ventral (OFA, FFA, anterior temporal lobe face area) and a dorsal (pSTS-FA, anterior STS-FA, inferior frontal gyrus face area) stream of face-selective areas. The former pathway contributes to the representation of invariant features of a face, whereas the latter is involved in processing dynamic aspects of faces (Duchaine & Yovel, 2015).

Studies of face perception using fMRI may identify the brain regions involved in face perception but cannot capture the temporal dynamic properties of those regions, many of which operate in a few hundreds of milliseconds. ERPs have been used to study the time course of face processing. The N170 is a negative posterior lateral ERP component that peaks at approximately 170 ms after stimulus onset and has been consistently found to be associated with face perception (Bentin, Allison, Puce, Perez, & McCarthy, 1996; Eimer, 2000a; Rossion, 2014; Rossion & Jacques, 2008; Rousselet, Husk, Bennett, & Sekuler, 2008). Temporal information provided by EEG recordings may be coupled with source localization techniques to identify the cortical sources of ERP components. We recently reviewed 25 studies that used source localization of the N170 ERP component, summarized in online supporting information, Table S1 (see reference note 1, tables 1, 4 in Richards, Gao, Conte, Guy, & Xie, 2018). The Q10 N170’s neural generator has been localized to the three brain regions of the core system for face perception corresponding to the fMRI functionally defined face-selective areas (e.g., FFA, OFA, pSTS-FA). Among the 71 areas that were reported as putative locations for the cortical source of the N170 in these 25 studies, 44 were labeled as coming from the middle fusiform gyrus (mFFG) or had coordinates landing in the mFFG. However, several of the coordinates listed as coming from the mFFG were likely in adjacent ventral lateral temporal-occipital areas such as the posterior FFG (pFFG) or the inferior portion of the lateral occipital gyrus (iLOG). Additionally, most of these studies used normalization methods (e.g., using regions of interest, ROIs, defined by group analyses) rather than individualized analyses, thus failing to consider individual anatomical differences (see Bobes et al., 2018, for an example). These studies suggest that the neural generator of the N170 component is located in the mFFG, which generally overlaps the fMRI-defined fusiform face area.

TABLE 1.

Anatomical regions of interest (ROIs) for the current study

ROI
anterior fusiform gyrus	aFFG
lateral inferior occipital gyrus	lIOG
lingual gyrus	LG
middle fusiform gyrus	mFFG
parahippocampal gyrus	PHG
posterior inferior temporal gyrus	pITG
superior temporal sulcus	STS
ventral temporal-occipital visual areas	FG1, FG2, FG3, FG4
fusiform gyrus	FG1, FG2, FG3, FG4
posterior fusiform gyrus	pFFG; from FG1, FG2
inferior occipital gryi	hOc1, hOc2, hOc3v, hOc4v

Note. Also see supplemental information table 6, supplemental information figures 4 and 6 (Richards et al., 2018) and …… (Rosenke et al., 2018).

TABLE 4.

Eta-squared and significance level values of N170 sources of a series of anatomical regions over both right and left hemisphere

ROI	Stimulus type	p	Stimulus orientation	p
Left aFFG	0.302	.0002	0.025	.0641
Right aFFG	0.169	.0011	0.032	.0116
Left FG3	0.236	.0002	0.031	.0256
Right FG3	0.258	.0007	0.036	.0060
Left FG4	0.143	.0009	0.022	.0628
Right FG4	0.213	.0034	0.026	.0073
Left mFFG	0.200	.0003	0.044	.0017
Right mFFG	0.266	.0007	0.028	.0015
Left pITG	0.156	.0019	0.034	.0129
Right pITG	0.258	.0008	0.017	.0028
Left FG1	0.201	.0019	0.039	.0048
Right FG1	0.210	.0033	0.021	.0079
Left FG2	0.148	.0068	0.036	.0075
Right FG2	0.222	.0030	0.023	.0025
Left FGp	0.173	.0034	0.038	.0057
Right FGp	0.219	.0031	0.022	.0039
Left lIOG	0.205	.0012	0.030	.0052
Right lIOG	0.180	.0057	0.031	<.0001
Left hOc4v	0.188	.0039	0.041	.0021
Right hOc4v	0.203	.0037	0.027	.0035
Left hOc3v	0.184	.0145	0.038	.0081
Right hOc3v	0.152	.0070	0.025	.0103
Left hOc2	0.150	.0239	0.044	.0280
Right hOc2	0.160	.0013	0.029	.0238
Left hOc1	0.157	.0352	0.035*	.0299
Right hOc1	0.158	.0017	0.027	.0277
Left STS	0.227	.0002	0.032	.0566
Right STS	0.207	.0002	0.027	.0305
Left PHG	0.324*	.0001	0.026	.0277
Right PHG	0.205	.0007	0.033	.0076
Left LG	0.253	.0027	0.029	.0195
Right LG	0.147	.0043	0.022	.0149

Note. Significant effects are marked in bold.

^*Maximum value of variance.

The face-sensitive brain areas identified by fMRI and source analysis of ERP may be compared by examining the specific locations found in fMRI and N170 sources studies. To do this, we searched for studies using fMRI that found significant activation to faces in the mFFG (i.e., the functionally defined fusiform face area) and compared them with locations from the studies on source localizations of the N170 (Richards et al., 2018).¹ We found a list of Q11 studies using a face localizer task to identify the coordinates of the fusiform face area (Berman et al., 2010), a recent meta-analysis of fMRI studies on face-sensitive brain locations (Müller et al., 2018), and 35 additional studies of the fusiform face area (table 1 in Berman et al., 2010; tables 1 and 2 in Richards et al., 2018). These resulted in 123 studies and 259 coordinates of the fMRI-defined fusiform face area. There was consistency among the three sets of references with the Talairach coordinates of the fMRI-defined fusiform face area generally being in the mFFG (Richards et al., 2018). Figure 1 shows the fMRI-based coordinates plotted on a 3D representation of an average MRI template from adult participants, along with the coordinates found for the N170 mFFG sources. Overall, the N170 and fMRI-defined fusiform face area locations were consistent. However, the N170 source locations were scattered more widely than the fMRI locations, and some of the N170 locations labeled as coming from the mFFG were coordinates of neighboring areas.

TABLE 2.

Eta-squared values of N170 amplitude over each posterior lateral electrode cluster as a measure of the variance

Electrode cluster	Stimulus type	Stimulus orientation
P7	0.281	0.291
P8	0.348	0.146
P9	0.366	0.222
P10	0.471	0.217
PO7	0.355	0.380
PO8	0.457	0.217
PO9	0.468	0.356
PO10	0.530*	0.418*
TP7	0.038	0.092
TP8	0.146	0.016
TP9	0.212	0.066
TP10	0.346	0.031

^*Maximum value of variance.

FIGURE 1.

Left: 3D rendering of the overlap between historical FFA locations (red circles) and N170 source locations (green circles). A list of the coordinates may be found in Richards et al., 2018 (tables 1, 2, 3, 5). Right: Anatomical ROIs shown on the 20–24 years average template. These regions include both the most relevant FFA literature locations (from Guy, Zieber, & Richards, 2016) and the ventral temporal-occipital visual areas for the fusiform gyrus (Rosenke et al., 2018). For all figures for fMRI, ERP, and cortical sources, the right side of the figure is the right side of the head

The relation between the face-sensitive brain areas identified by fMRI and those identified by source analysis of the N170 has also been studied with multimodal studies that record fMRI and the N170 ERP component in the same study. Four studies conducted a correlation between N170 amplitudes for faces and the fMRI BOLD signal and found significant correlations between the ERP signal and fMRI-defined fusiform face area and pSTS face area (Horovitz, Rossion, Skudlarski, & Gore, 2004; Iidaka, Matsumoto, Haneda, Okada, & Sadato, 2006; Nguyen & Cunnington, 2014; Sadeh, Podlipsky, Zhdanov, & Yovel, 2010). For example, one study presented participants with faces and chairs with simultaneous EEG-fMRI recording (Sadeh et al., 2010). The BOLD responses in fusiform and pSTS face areas were significantly correlated with the N170 amplitudes. Other studies have included fMRI and cortical source analysis of ERP on the same participants. Corrigan et al. (2009) found an overlap between fMRI activation areas and ERP source localization areas including the fusiform gyrus, superior temporal gyrus, and bilateral precuneus. Another approach used in a recent study defined the contribution of fMRI-defined regions of interest (ROIs) to the generation of ERPs by conducting a fMRI-constrained source analysis of the N170 component (Bobes et al., 2018). They found that fMRI-defined ROIs were cortical sources of the ERP signal for time windows fewer than 380 ms, but not of a late ERP component. These findings suggest that combined fMRI and ERP may be complementary in understanding the temporal sequence of activity within the face processing neural network.

The current literature provides valuable insights into the neural origin of the N170, but there are discrepancies or shortcomings of the prior findings. The relation between face-sensitive locations found with fMRI and the cortical sources of the N170 remains unclear. We conducted an ERP experiment in which static upright and inverted faces and houses were presented. Participants were assigned to one of three tasks to manipulate cognitive load and attention: (a) orientation judgment for current stimulus, (b) judgment of same/different orientation for 1-back stimulus, and (c) judgment of same/different stimulus type (face or house) for 1-back stimulus. An important methodological advance for source localization of the N170 ERP was made by the present study (Richards et al., 2018). This included high-density scalp recordings (128 Q12 electrodes), source localization algorithms based on distributed source models, individualized head models based on structural MRI scans from each participant, voxelwise segmentation of all relevant head materials, and finite element method (FEM) procedures for the source analysis. The best method for doing source analysis requires head models based on individual participants, to account for anatomical differences between individuals, and FEM procedures for identifying the location and conductivity of media inside the head (Akalin Acar & Makeig, 2013; Cho, Vorwerk, Wolters, & Knosche, 2015; Darvas, Ermer, Mosher, & Leahy, 2006; Grech et al., 2008; Hallez et al., 2007; Vorwerk et al., 2014; Vorwerk, Oostenveld, Piastra, Magyari, & Wolters, 2018). Our methods are an improvement over earlier source analysis studies that used equivalent cortical dipole analysis, low-density EEG recording, a head model based on an average template (e.g., the Montreal Neurological Institute, MNI, average template), or a nonrealistic head model such as a spherical model.

A structural MRI and a fMRI experiment were done with the same participants. We analyzed the current density reconstruction (CDR) values from the source analysis with a series of anatomical brain regions derived from the structural MRI typically linked to face processing (Richards et al., 2018). We also analyzed the CDRs with a series of ROIs from the ventral temporal-occipital pathway, for example, the temporal pole (aFFG) through the ventral occipital area (e.g., hOc1). The fMRI experiment used the same stimulus materials and similar procedures as the EEG task. The fMRI was used as a localizer task to define face-sensitive ROIs, which then were used to analyze the CDR values. The functional ROIs were defined with Gauss-gamma adaptive thresholding for each participant to have a good balance between false positive and false negative error rates (Gorgolewski, Storkey, Bastin, & Pernet, 2012; Richards et al., 2018). Our general prediction was that the mFFG and closely surrounding areas would be the major source of the N170 ERP component. Though some studies of source analysis have localized the N170 component to the pSTS, research with fMRI has distinguished between the processing of face configuration information (ventral stream, mFFG) and dynamic moving faces, or facial emotions (dorsal stream, pSTS-FA; e.g., Bernstein & Yovel, 2015; Duchaine & Yovel, 2015; Freiwald, Duchaine, & Yovel, 2016; Müller, Höhner, & Eickhoff, 2018). Our stimuli consisted of static faces or houses, so we predicted a less important role of pSTS in generating the N170 component.

2. METHOD

2.1. Participants

Participants were 34 volunteers recruited from University of South Carolina (mean age, 23.4 years; SD, 5.2 years; 13 male). All had normal or corrected-to-normal vision and no history of neurological impairments. Participants were compensated for their time and gave written informed consent as approved by the Institutional Review Board at University of South Carolina. Each participant took part in the ERP experiment, structural MRI, and fMRI, in that order, on separate days. All participants’ data were used for the ERP analyses. Two participants did not participate in the structural MRI or fMRI procedures due to exclusion criteria (pregnancy, metal implant). This resulted in structural and fMRI data for 32 participants for the analysis of the anatomical and functional ROIs for the ERP source analysis.

2.2. Faces and houses presentation procedure

2.2.1. Stimuli and experimental control equipment

Stimuli were digitized color photographs of human faces and houses. Forty-two full front view faces (25 male, 17 female) with neutral or happy expression were taken from the NimStim database (Richards et al., 2018, figure 7; Tottenham et al., 2009). All faces were cropped to an oval-shaped mask in order to hide distinguishing features (i.e., ears, hair, clothes). Forty-two symmetrical images of houses were taken from the Internet and edited to remove the background and the landscape area on the borders. The E-Prime computer program was used to control the stimulus presentation and record responses. For the ERP experiment, participants sat in an electrically shielded and sound-attenuated room, facing a 28-in. LCD monitor (Hanns.G HG281D) about 55 cm away on which stimuli were presented. A Chronus response box was used to collect responses. For the MRI experiment, the stimuli were presented to the participant via a mirror on the head coil that allowed for a view of a projection screen placed at the foot of the scanning table. A Celeritas Series Fiber Optic Button Response glove was used to collect responses.

FIGURE 7.

Top: Current density reconstruction (CDR) values at posterior fusiform gyri plotted around the peak of N170 as a function of the four conditions. Bottom: CDR for the functionally defined fusiform face area. Larger CDR values around the peak were found for faces compared to houses, with the largest amplitudes recorded at the posterior fusiform gyrus. All regions showed an inversion effect for faces

2.2.2. Procedure

The fMRI and ERP procedures involved presentations of faces and houses to the participants. Participants were presented with four categories of stimuli: upright faces, inverted faces, upright houses, and inverted houses. The four categories were randomly chosen on a trial-by-trial basis for the ERP experiment and in a blocked design for the fMRI experiment. The trials started with a fixation cross presented in the center of the screen for a duration of 500–1,000 ms, randomly chosen on each trial. The face or house stimulus was then presented for 500 ms followed by a blank screen for 750 ms. The fixation cross was presented within a lime-green square for 10% of trials, randomly chosen. On these trials, following the stimulus presentation, the participant was instructed to make a response evaluating the stimulus within 2 s. The trials were repeated for 20 s, followed by a feedback display that presented the number of trials, correct and incorrect responses, and omissions (no response within 2 s).

ERP experiment

The feedback screen lasted for a maximum of 10 s, and the trials could be restarted if the participant pressed any response button. The participants were assigned to one of three conditions for the response: (a) stimulus orientation, judging the orientation as upright or inverted regardless of stimulus type (N = 12); (b) 1-back orientation, judging whether the stimulus had the same orientation as the previous one regardless of stimulus type (N = 11); and (c) 1-back stimulus, judging whether the stimulus was the same type, face or house, regardless of orientation (N = 11).

fMRI experiment

Participants were presented with the stimuli in a block design consisting of 20 s of stimulus presentations and 15 s of rest. The delay period could not be terminated by a participant’s response. A single stimulus judgment type was done for all participants, which was the same as the stimulus orientation task in the ERP experiment. Since all stimuli in a block were of the same orientation, the same response would be used in a stimulus block. A total of 24 35-s blocks were presented with each stimulus type presented randomly in four-trial blocks (total six blocks per condition; total 14 min).

2.3. Structural MRI

2.3.1. Image acquisition

Whole-head anatomical images were obtained using a T1-weighted 3D MP-RAGE protocol (TR = 2,250 ms, TE = 4.11 ms, flip angle = 9°, FOV = 256 × 256 mm, in-plane resolution = 256 × 256 pixels, voxel size = 1.0 × 1.0 × 1.0 mm). Whole-head T2-weighted images were obtained using a T2-weighted SPC protocol (TR = 3,200 ms, TE = 567 ms, FOV = 256 × 256 mm, in-plane resolution = 256 × 256 pixels, voxel size = 1.0 × 1.0 × 1.0 mm). The first 10 participants were scanned with a Siemens Magnetom Trio 3.0 T scanner (Siemens, Erlangen, Germany) and the remainder with a Siemens Prisma scanner. The same sequences were used for the scanners.

2.3.2. MRI segmenting and anatomical ROIs

The anatomical MRI of each participant was used for segmenting of the head materials and to define anatomical ROIs for the analyses. We also used them for EEG electrode placement and as a high-resolution MRI for the fMRI analyses (later sections). The MRI was used to segment the head into scalp, skull, cerebrospinal fluid (CSF), white matter, gray matter, nasal cavity, eyes, and muscle (Richards, 2013, main text and supplemental information, for details). The gray matter and eyes were used as the source locations for the source analysis (see EEG and ERP in Method). The MRI thus provided us with a realistic head model for cortical source analysis.

Anatomical ROIs were defined based on the extracted brain from the structural MRI. First, we defined the ROIs based on anatomical stereotaxic atlases of the individual structural MRI (Fillmore, Richards, Phillips-Meek, Cryer, & Stevens, 2015; Richards, 2013, supplemental information). These ROIs were based on the LONI Probabilistic Brain Atlas (LPBA; Shattuck et al., 2008) and the Hammers atlas of MRIs from the Information Exchange for the Internet (Heckemann, Hajnal, Aljabar, Rueckert, & Hammers, 2006; Heckemann et al., 2003). These atlases were used to define 18 anatomical ROIs (Guy, Zieber, & Richards, 2016; Richards et al., 2018, table 4). For this study, we selected a subset that would be of interest to the hypotheses about the N170 sources and fusiform face area locations (e.g., source of N170 in the mFFG, fusiform face area in mFFG, occipital face area in lIOG, face sensitive areas in the pSTS). Q4 Second, we used a recent study of the anatomy of the ventral temporal-occipital visual areas (e.g., FG1, FG2, FG3, FG4; Rosenke et al., 2018). We were primarily interested in these areas because they provided an explicit anatomical location for the pFFG (e.g., FG1, FG2, pFFG). The MNI coordinates for these anatomical locations were translated into an average MRI template based on 20- to 24-year-olds (Richards, Sanchez, Phillips-Meek, & Xie, 2015; Richards & Xie, 2015) and then into the individual participant MRI. The ROIs included were mapped onto the 20–24 years average template (Richards, Sanchez et al., 2015; Richards & Xie, 2015). Figure 1 shows the anatomical locations defined from the Rosenke atlas overlaid on the mFFG area defined with the first method. Table 1 contains a list of the anatomical ROIs emphasized in this study (Richards et al., 2018). The anatomical ROIs from the structural MRI and the vcAtlas were translated into the FieldTrip format for storing MRI volumes (Oostenveld, Fries, Maris, & Schoffelen, 2011; Richards et al., 2018).

2.4. fMRI experiment

2.4.1. Image acquisition

A total of 700 volumes of transverse slices, covering the whole brain, were acquired in an ascending order starting from the bottom of the brain. Functional images were acquired using a multiband echo EPI (echo planar imaging) pulse sequence (TR = 1,200 ms, TE = 37 ms, multiband acceleration factor = 4, flip angle = 65°, FOV = 208 × 208 mm, in-plane resolution = 104 × 104 pixels, slice thickness = 2 mm, gap = 0 mm, voxel size = 2.0 × 2.0 × 2.0 mm; Xu et al., 2013). A single-band reference image (SBRef) with high resolution and increased tissue contrast was collected before the fMRI recording (e.g., Human Connectome Project, https://www.humanconnectome.org/storage/app/media/documentation/data_release/October2012 _Release_User_Guide.pdf). Two brief functional scans were collected at the end of the fMRI recording with reversed phase-encode blips, resulting in pairs of field map images with distortions going in opposite directions (anterior-posterior, posterior-anterior). These two scans were used for distortion corrections.

2.4.2. Data analysis

The fMRI preprocessing was carried out using FSL (FMRIB software library, version 5.0, www.fmrib.ox.ac.uk.fsl) and was designed by Hanayik (2018). The procedure included Q5 field inhomogeneity correction with FSL’s TOPUP tool, head motion detection with FSL’s MCFLIRT, spatial smoothing with FSL’s fslmaths. The BOLD fMRI volumes were registered to the whole head T1 image with the FSL SBRef tool. This registration was used to relate the fMRI BOLD volumes to the anatomical atlases based on the T1, provide functional ROIs, and relate the 2-mm BOLD volumes to the 1-mm T1 and 3-mm source analysis spaces.

A general linear model (GLM) using SPM 12 (http://www.fil.ion.ucl.ac.uk/spm) in MATLAB was carried out. This was done by convolving the stimuli onsets of all conditions with a canonical hemodynamic response function for each participant data. Six motion parameters from MCFLIRT were included as nuisance covariates, while removing low frequency drifts by a high-pass filter with a cutoff of 128 s and accounting for serial autocorrelations with an autocorrelation model AR (1). The following t contrasts were computed: faces versus houses, houses versus faces. The contrast images were entered in a one-sample t test, with voxelwise significance that the two contrast values were significantly different from 0, respectively. We also computed contrasts that were not analyzed with statistical tests: Faces gt rest, houses gt Q6 rest.

2.4.3. Functional ROIs

The fMRI was used as a localizer to define functional ROIs based on the fMRI data of each individual; complete details of this procedure may be found in Richards et al. (2018). We followed recent recommendations that define functional localizers on the individual fMRI rather than on group results translated to the individual (Fedorenko, Hsieh, Nieto-Castanon, Whitfield-Gabrieli, & Kanwisher, 2010; Glezer & Riesenhuber, 2013; Gorgolewski et al., 2012; Julian, Fedorenko, Webster, & Kanwisher, 2012; Nieto-Castanon & Fedorenko, 2012; Wang et al., 2014). The Gauss-gamma adaptive threshold method for voxelwise error control (Gorgolewski et al., 2012) was used by identifying a threshold for the anatomical ROI being used and using the threshold values for the individual to find the functional ROI. We defined four ROIs for each participant based on this method: face-sensitive areas with significant thresholded voxels in the middle fusiform gyrus and adjacent areas (fusiform face area), house-sensitive areas with significant thresholded voxels in the lingual gyrus and adjacent areas (lingual gyrus house area), face-sensitive areas in the occipital lobe and adjacent areas (occipital face area), and house-sensitive areas in the occipital lobe and adjacent areas (occipital house area). The functional ROIs from the fMRI were translated into the FieldTrip format for storing MRI volumes (Oostenveld et al., 2011; Richards et al., 2018). Figure 2 shows these areas translated as a mask from each participant to the adult average MRI template and summed over all subjects for each ROI.

FIGURE 2.

Coronal and lateral representation of functional ROIs (fROIs). Individual participant MRIs were transformed into the 20–24 years average MRI template, and the color bar represents the number of participants with a voxel in the fROI at that location. Face-sensitive areas were localized in the fusiform gyrus (FFA) and occipital lobe (OFA), while house-sensitive areas were detected in the lingual gyrus (LGHA) and occipital lobe (OHA)

2.5. ERP experiment

2.5.1. EEG recording

Continuous EEG was recorded using a high-density Electrical Geodesics Incorporated (EGI, Eugene, OR; Tucker, 1993; Tucker, Liotti, Potts, Russell, & Posner, 1994) recording system consisting of 128 channels (HydroGel Geodesic Sensor Net, HGSN). The EEG was recorded with 20 K amplification at a 250 Hz sampling rate with band-pass filters set from 0.1 to 100 Hz and impedances below 100 kΩ. The EEG signal was referenced to vertex during the recording and rereferenced algebraically to an average reference. The Netstation version used the correct filtering offset.

2.5.2. HGSN and 10–10 electrode locations

The locations of the HGSN electrodes were found on each participant with a Geodesic Photogrammetry System dome following the experiment (Russell, Jeffrey Eriksen, Poolman, Luu, & Tucker, 2005). Details of this procedure have been reported previously (Richards, 2013; Richards, Boswell, Stevens, & Vendemia, 2015, supplemental information; Richards et al., 2018). The GPS acquires photographic images of the head, which were then coregistered to the head mask of the anatomical MRI for each participant. We identified a set of external head locations (e.g., Nz, Iz, LMA, RMA, LPA, RPA, Vz), which were used to compute the 10–10 recording locations. The EEG data from the 128 HGSN channels were transformed into “virtual 10–10” channels using a spherical spline interpolation from the participant’s HGSN locations into the participant’s 10–10 locations with a spherical spline interpolation of the 128 channel data into 81 10–10 channel data (Richards et al., 2018).

2.5.3. ERP data analysis

EEG preprocessing, artifact detection and rejection, and ERP averaging were carried out using the EEGLAB (Delorme & Makeig, 2004) and ERPLAB (Lopez-Calderon & Luck, 2014) MATLAB toolboxes. First, a multistage robust rereferencing of the signal was performed using the PREP pipeline MATLAB toolbox (Bigdely-Shamlo, Mullen, Kothe, Su, & Robbins, 2015). This procedure provides an average reference after removing line noise, doing a 1 Hz high-pass filter, and interpolating bad channels. The EEG was digitally filtered with a band-pass Butterworth filter of 0.1–42 Hz and then segmented into epochs consisting of 100 ms of prestimulus recording and 1,000 ms of recording after stimulus onset. Waveforms were corrected relative to the 100-ms prestimulus baseline period. Trials containing EEG activity exceeding ± 200 μV were removed within the first 500 ms, and EEG activity exceeding ± 400 μV were removed within 500–1,000 ms. Extended independent component analysis (ICA; Delorme & Makeig, 2004) was done on the segmented data and was used to identify IC components representing eye movements or eyeblinks based the timing and shape of the IC activations and the ocular channels. The remaining components were used to reconstruct the data (Jung et al., 2000). All epochs were visually inspected for detecting ocular and motor movements and other artifacts. Individual trials containing bad channels with unresolved ocular or activity artifacts were identified. The channels with artifact were substituted with artifact-free channels using a spherical spline interpolation routine (Delorme & Makeig, 2004). Trials with more than 12 bad channels were rejected for further analysis.

The ERP analyses focused on the amplitude of N170 component. Peak-to-trough difference between the N170 peak and the preceding positive peak (P1) was calculated in order to control for the potential effect of slow waves over posterior lateral clusters of electrodes (PO7, PO8, PO9, PO10, P7, P8, P9, P10, TP7, TP8, TP9, TP10; see figure 7 in Richards et al., 2018). First, the median latency value of P1 component was calculated between 100 ms and 200 ms over medial posterior clusters of electrodes (Oz, Iz, O1, O2, I1, I2); then, for each stimulus category (face upright, face inverted, house upright, house inverted) the N170 peak was defined as the negative trough from the median latency of the P1 to 250 ms. The mean peak across participant did not differ based on stimulus category, F(3, 4436) = 0.50, p = 0.682 (M_{face upright} = 178.075; M_{face inverted} = 178.563; M_{house upright} = 177.793; M_{house inverted} = 177.385). A multivariate approach to repeated measures was used to analyze with a GLM approach the groups of electrodes as multiple dependent variables and the experimental factors. The grouping of the electrodes and the multivariate analysis controlled for inflated error rates due to repeated tests and heterogeneity in the covariance matrix of the electrode effects. Thus, a multivariate analysis of variance (MANOVA) was performed on the peak amplitudes using the Proc GLM of SAS (version 9.4) software (SAS Institute Inc., Cary, NC). Separate univariate ANOVAs were performed on each cluster of electrodes, in order to test the effect of significant dependent variables in each of the ROIs. All statistical tests were conducted on a .05 level of significance.

2.5.4. ERP source analysis

Source localization of neural activity was conducted with the realistic head models and the CDR technique. First, a forward model was created using a FEM mapping of the electrical conductivity of the head. The forward model was created with the FieldTrip computer program and the SimBio protocol (Oostenveld et al., 2011; Vorwerk, 2013; Vorwerk et al., 2018). Q7 The forward model included the segmented MRI volume and values of conductivity for each material. Second, we defined the source locations (source volume) for the analysis with a 3-mm grid defined on the participant’s segmented gray matter and location of eyes. The lead-field matrix was based on the locations of the electrodes on the scalp, source locations defined by the gray matter/eyes source volume, and the forward model for each individual. Third, source reconstruction was conducted with the ERP data and the lead-field matrix using eLORETA constraint (Pascual-Marqui, 2006). The peak values around N170 were used for estimation Q7 of the current density amplitudes (i.e., CDR values) for each location in the source volume model. The CDR values were stored in the FieldTrip storage model for MRI volumes (Oostenveld et al., 2011; Richards et al., 2018). The source reconstruction methods and the algorithm pipeline for the FieldTrip computer programs used in the source analysis are presented in Richards et al. (2018) and elsewhere (Buzzell et al., 2017).

3. RESULTS

3.1. ERP results

The ERPs were examined for their relationship to the experimental conditions: upright and inverted faces, upright and inverted houses. Figure 3 shows the average distribution on the scalp at the peak of the N170 for the faces and houses ERP. There was large bilateral negative activity over the posterior lateral channels for the face stimuli, which appears to be centered over the most inferior row of the 10–10 electrodes (e.g., PO10, P10, TP10).

FIGURE 3.

Topographical maps of the grand-averaged data representing the scalp distribution for faces (left) and houses (right). Electrode sites represented in green circles refer to the 128 HGSN system, whereas red circles represent the virtual 10–10 system. Maps show a more negative scalp distribution at the N170 peak for face than house stimuli at inferior posterior lateral channels

The peak of the N170 ERP component was analyzed for the inferior posterior lateral 1010 channels (PO7, PO8, PO9, PO10, P7, P8, P9, P10, TP7, TP8, TP9, TP10). A MANOVA with the electrodes as multiple dependent variables was done on the peak-to-trough value as a function of stimulus type (face, house), stimulus orientation (upright, inverted), and task group (orientation, 1-back stimulus, 1-back orientation). There were significant main effects of stimulus type, Wilks’s λ = 0.3204, F(12, 20) = 3.53, p = 0.0062, and stimulus orientation, Wilks’s λ = 0.419, F(12, 20) = 2.31, p = 0.0474. These effects were confirmed by univariate ANOVA results combining all electrodes, which revealed significant stimulus type, F(11, 22) = 3.50, p = 0.0001, and stimulus orientation, F(11, 22) = 2.85, p = 0.0014, main effects. No significant main effect or interactions involving task group factor were found. Figure 4 shows the ERP changes for the four experiment stimulus types. The N170 peaks were significantly greater for faces than houses and for inverted than upright stimuli (Figure 4, topographical scalp potential maps). Univariate tests for the individual electrodes showed significant stimulus type and stimulus orientation for all but the TP electrodes (ps < 0.0233). The parietal and parietaloccipital electrodes showed larger N170 for faces and inverted stimuli. Table 2 shows the η² of each effect for each electrode. The largest values for stimulus type and stimulus orientation were in the PO10 electrode, followed by P10, PO9, PO8 for the stimulus type effect, and by PO7, PO8 for the stimulus orientation effect.

FIGURE 4.

Top-half panel represents topographical potential maps at the N170 peak latency (177 ms) as a function of stimulus type (left pair) and stimulus orientation (right pair) across task groups. A large negative N170 ERP peak occurred for faces in the inferior posterior lateral positions, and the average N170 ERP peak occurred in similar positions when combined for stimulus orientation across stimulus types and task groups. Grand-averaged ERP waveforms elicited by all conditions (upright face, upright house, inverted face, inverted house) at inferior posterior lateral channels are depicted in the bottom-half panel

We noticed striking individual differences in the pattern of the distribution of the scalp topographical potential maps. Figure 5 shows the average distribution on the scalp at the peak of the N170 for the face stimuli, averaged across the subjects (left figure) and for six individuals. There was a large area across which the peak of the N170 distribution occurred across these participants. Such topographical differences in the N170 distribution of activity might reflect differences in the underlying source of this component, either in the location of the source or in the pattern on the scalp due to individual differences in electrical properties of underlying tissue.

FIGURE 5.

Topographical maps of average distribution on the scalp at the peak of N170 for face stimuli. Leftmost figure shows the scalp distribution for the average template of all participants (n = 32). Remaining figures highlight the differences in the N170 scalp distribution for six individuals

3.2. Cortical sources for anatomical and functional ROIs

The cortical source analysis of the N170 ERP component was examined for its relationship to the experimental conditions and ROIs. Figure 6 shows the average CDR for the face stimuli plotted on a 3D rendering of the average MRI template. The source activity of the N170 for faces was spread throughout the posterior temporal-occipital ventral surface with more activity in the right than in the left hemisphere. The largest value for the CDR in the entire brain was in the pFFG (Figure 6, right panels; −52, 33, −11 for 20- to 24-year template; −51, 31, −10 for MNI template). The CDR values surround the peak N170 activity (± 8 ms) were analyzed in relation to both anatomical regions reported in literature as sources of the N170 ERP component and functional ROIs derived from our fMRI experiment. Figure 7 compares the CDR values around the N170 peak for the four experimental conditions in the max location found in the whole brain figure (pFFG) and the face-sensitive area defined by our fMRI data (i.e., fusiform face area). Larger CDR values were found in response to faces than houses for the anatomical and functional ROIs, and larger values were found on the right than on the left hemisphere.

FIGURE 6.

3D rendering of the average CDR for faces plotted on the average template. The average was obtained by transforming individual participant CDR volumes into the 20–24 years average MRI template, smoothing with 4-mm Gaussian filter, and averaging the resulting transformed volumes. Display shows values above a one-way t test done with Monte Carlo permutations governed by a cluster strategy. Right images show the location of the maximal CDR value for faces

An analysis was done on the summed CDR activity from the two ROIs thought to be most relevant for the source of the N170 (mFFG, pFFG), a nearby ROI (lingual gyrus, LG), and an ROI that has been found to be a source of the N170 ERP component (pSTS). A univariate ANOVA was carried out to examine the CDR values in these anatomical ROIs as a function of stimulus type (face, house), stimulus orientation (upright, inverted), ROI (mFFG, pFFG, LG, pSTS), side (left, right), and task group (orientation, 1-back stimulus, 1-back orientation). There was a significant main effect of ROI, F(3, 90) = 17.57, p < 0.0001. The pFFG had the maximum amplitude (i.e., showed the largest CDR values; M = 10.10; SD = 12.21), followed by the mFFG (M = 9.80; SD = 11.87) and LG (M = 9.06; SD = 8.25), whereas the pSTS activity was the smallest (M = 4.30; SD = 3.87). The main effects of both stimulus type, F(1, 31) = 19.02, p = 0.0001, and stimulus orientation, F(1, 31) = 8.83, p = 0.0057, were qualified by a significant interaction between stimulus type, stimulus orientation, and ROI, F(3, 90) = 3.61, p = 0.0162. The univariate ANOVAs over each anatomical brain region revealed significant effect of both stimulus type, Fs(1, 33) > 9.40, ps < 0.0043, and stimulus orientation, Fs(1, 33) > 3.90, ps < 0.0556, for all the considered regions (p values of each region are reported in Table 3). Figure 7 (top panels) displays CDR values of the four conditions around the peak of N170 in the pFFG. Current density amplitude was greater in response to faces than houses and in response to inverted than upright stimuli. A similar pattern of activity but of lower magnitude was found in the LG and pSTS ROIs. The smallest activity occurred in the pSTS anatomical region (upright face: M = 4.71, SD = 3.89; inverted face: M = 5.71, SD = 4.85; upright house: M = 3.30, SD = 2.74; inverted house: M = 3.81, SD = 3.26). Simple effects were further examined through the calculation of the eta-squared values (Table 3), which revealed that the variance of both the factors was largely explained over pFFG (stimulus type η² = 0.624; stimulus orientation η² = 0.107). Moreover, the stimulus type and stimulus orientation interaction was significant only for mFFG (left: F(1, 33) = 80.857, p = 0.0050; right: F(1,33) = 91.025, p = 0.0111) and pFFG (left: F(1, 33) = 146.970, p = 0.0115; right: F(1, 33) = 151.582, p = 0.0350) anatomical regions. The post hoc analyses revealed for all the regions significant inversion effect for faces (ps < 0.0009) but not for houses (ps > 0.0802).

TABLE 3.

Eta-squared and significance level values of N170 sources of four anatomical regions over both right and left hemisphere

ROI	Stimulus type	p	Stimulus orientation	p
Left mFFG	0.192	.0003	0.099	.0017
Right mFFG	0.605	.0007	0.063	.0015
Left pFFG	0.167	.0034	0.107*	.0057
Right pFFG	0.624*	.0031	0.063	.0039
Left LG	0.237	.0027	0.103	.0195
Right LG	0.517	.0043	0.079	.0149
Left STS	0.222	.0002	0.092	.0566
Right STS	0.587	.0002	0.077	.0305

^*Maximum value of variance.

The analysis of the four functional ROIs defined by our fMRI data (Figure 2) was done on the summed activity of the CDR of the N170 ERP component for its relationship to the experimental conditions. Separate univariate ANOVAs were performed to test the current density values of each functional ROI as a function of stimulus type (face, house), stimulus orientation (upright, inverted), side (left, right), and task group (orientation, 1-back stimulus, 1-back orientation). Figure 7 (bottom panel) shows that the CDR values plotted around the peak of N170 in the fMRI-defined fusiform face area were greater for faces than houses. Both the occipital face area and the lingual gyrus house area showed a similar trend of activity, whereas there were small differences between face and house CDR values in the occipital functional face region (face: M = 6.53, SD = 4.31; house: M = 5.66, SD = 3.96). The N170 current density was significantly affected by stimulus type, Fs(1, 26) > 7.68, ps < 0.0102, and stimulus orientation, Fs(1, 28) > 5.39, ps < 0.0283, over fusiform and occipital face areas, and the lingual gyrus house area. The F values of each factor are reported in Table 4. The CDR for faces was larger than houses in these three functional ROIs and was larger for inverted than upright stimuli. These main effects were qualified by a significant interaction between stimulus type and stimulus orientation, Fs(1, 26) > 5.14, ps < 0.0310 (see Table 5). The post hoc analyses revealed significant inversion effect for faces (ps < 0.0061) but not for houses (ps > 0.1367) over all three functional brain regions.

TABLE 5.

F values of significant factors in the ANOVA for CDR values of the functional ROIs

ROI	Stimulus type	Stimulus orientation	Type × Orientation	Group
FFA	15.29†	19.76†	16.31†	2.11 ns
OFA	7.68†	5.36*	8.23†	1.86 ns
OHA	2.41 ns	1.59 ns	1.39 ns	4.45*
LGHA	11.82†	12.07†	5.14*	2.03 ns

Note. ns = not significant.

^*Significant at 0.05

^†significant at 0.01.

3.3. Current density reconstruction and BOLD contrast

Figure 8 (top panels) shows a bar chart comparing the CDR of the faces and houses for a series of ROIs. These ROIs are plotted from the anterior portion of the ventral temporal lobe (e.g., aFFG) through the ventral occipital area (e.g., hOc1). There was a gradual shift of increasing CDR from the anterior to the mFFG, and then a decrease from the pFFG to the posterior ventral occipital ROIs. Statistical analyses of these regions showed main effects of stimulus type, F(1, 33) = 16.06, p = 0.0003, stimulus orientation, F(1, 33) = 9.14, p = 0.0048, and ROI, F(15, 468) = 7.56, p < 0.0001, Stimulus Type × ROI, F(15, 68) = 5.55, p < 0.0001, and Stimulus Orientation × ROI, F(15, 68) = 3.24, p < 0.0001, interactions. ANOVAs on each region revealed an effect of both stimulus type and orientation in almost all the ROIs, with the largest variance explained by parahippocampal gyrus (PHG; stimulus type, η² = 0.324) and hOc1 (stimulus orientation, η² = 0.035), respectively. Values of p and η² of both stimulus type and orientation over the considered ROIs are reported in Table 4.

FIGURE 8.

Top: Current density reconstruction (CDR) values as a function of stimulus type for right anatomical ROIs. Bottom: BOLD contrast values as a function of stimulus type for right anatomical ROIs. Graphs show an overall larger activity for faces than houses and a more sharpened variation of the fMRI signal compared to the CDR values. Regions are plotted from the anterior portion of the ventral temporal lobe (i.e., aFFG) through the ventral occipital area (i.e., hOc1). Both CDR and BOLD show an increasing gradient from the anterior to the middle fusiform gyrus and a decrease from the posterior portion of the fusiform gyrus to the ventral occipital areas

We did not do formal quantitative comparisons between the ERP source analysis and the fMRI BOLD analysis due to the differences in methodology for quantifying the BOLD contrasts and the ERP CDR values. However, some informal comparisons were made that showed good correspondence between the two measures. Figure 8 shows the CDR response to faces and houses (top panels) and the BOLD responses in the faces > rest and houses > rest contrasts (bottom panels) as a function of a series of ventral temporal-occipital areas in the right hemisphere. There was an increase from the anterior areas to the mFFG in the contrast values across these ROIs and then a decrease in the values from the mFFG to the posterior occipital ROIs. The values of face > rest contrasts were overall larger than the house > rest contrast values. The same trend of an increase from the anterior to the mFFG and then a decrease to the posterior was found in the CDR values, with activity in response to faces larger than the activity in response to houses. However, the CDR values showed more gradual increases and decreases than the BOLD contrast values.

Figure 9 displays the average from all participants for the CDR to faces and the BOLD contrast values for the faces > rest contrast. The more compact peakedness of the BOLD contrast distribution may be seen in this figure compared to the CDR distribution, and both show larger activity on the right than on the left. The CDR and BOLD values were multiplied on a voxel-by-voxel basis and shown in the right panel of Figure 9. This resulted in an even more marked peak activity in the right hemisphere for the face stimuli. This peak of the combined CDR × BOLD values was in the same location as the peak CDR values (e.g., pFFG).

FIGURE 9.

Coronal and axial representations on the average MRI volume of the fMRI contrast (left), ERP CDR (middle). There was larger activity on the right than on the left side for both fMRI and ERP results, and the maximal value for both were in a similar location. 3D rendering representation of the fMRI × CDR shows the combined activity occurring the strongest in the posterior FFG

4. DISCUSSION

The main goal of the current study was to determine the neural origin of the N170 component and spatiotemporal characteristics of face processing in the brain. Structural MRI, fMRI, and high-density ERP data were acquired in separate sessions. The major hypothesis of the current study was that the mFFG is the primary neural source of the N170. The source analysis confirmed that the fusiform gyrus was the major contributor to the N170, with both the mFFG and pFFG showing considerable activity around the peak of the N170. This location was in a similar location as the fusiform face area found in the fMRI experiment. This was confirmed by the analysis of the CDR activity in the fMRI-defined fusiform face area ROI, and by the similar locations of the activity for the fMRI and the CDR in response to the face stimuli (e.g., Figures 7, 8, 9).

The N170 ERP analysis showed that larger amplitudes were elicited when participants were viewing faces than when viewing houses, and when viewing inverted than upright stimuli. The N170 effect was predominantly observed in posterior lateral occipital-temporal regions as illustrated in the topographical scalp potential maps (Figures 3, 4, 5). The maximum responses were mostly found in the PO10 electrode as supported by effect sizes of each effect (faces > houses or inverted > upright) for each electrode (Table 2). These findings are consistent with previous studies that found larger N170 amplitude for faces than for houses (Bentin et al., 1996; Carmel & Bentin, 2002; Eimer, 2000b; Iidaka et al., 2006; Sagiv & Bentin, 2001), as well as previous studies that found larger amplitude for inverted faces than upright faces (Bentin et al., 1996; Eimer, 2000a; Itier & Taylor, 2002, 2004a, 2004b; Jemel, Coutya, Langer, & Roy, 2009; Mohamed, Neumann, & Schweinberger, 2011; Rossion et al., 1999, 2000; Schweinberger, Kaufmann, Moratti, Keil, & Burton, 2007). There were striking individual differences in the distribution of the scalp topographical potential maps for N170. These might reflect individual differences in the source of the component. These differences also might represent idiosyncratic patterns in the electrical conduction of the signal from its neural source to the scalp due to differences in gray matter/white matter, CSF, or skull across individuals. These individual differences confirm our expectation that cortical source analysis of ERPs benefits from individualized head models.

We were particularly interested in the neural sources of N170 component. Prior research has applied cortical source analysis of N170, but mixed results have been found. The present study found a close correspondence between the location of the active areas for face processing from the fMRI and the ERP source analysis. These include similar distribution of the CDR and BOLD activity in the posterior ventral visual areas, mFFG and pFFG (Figures 6, 7, 8, 9). The results showed that the largest value for the CDR of the N170 was in the pFFG, nearly identical to the BOLD response (Figure 9). The functional relations between the face and house stimuli were similar. The CDR values in anatomical ROIs (mFFG, pFFG, LG, pSTS) were greater in response to faces than houses and to inverted than upright faces, with pFFG showing the maximum CDR values. A close quantitative parallel also was supported by the use of the functional ROI from the fMRI experiment showing similar functional relations for the CDR activity. These results were also confirmed by a voxel-by-voxel based multiplication of CDR and BOLD values, which showed consistency between the fMRI activations and the CDR values of the N170. It is plausible that the emphasis on individual structural MRIs for the head models, accurate coregistration between electrode locations and scalp, and use of the FEM for the source analysis contributed to this correspondence. Our cortical source findings are in line with our ERP findings, as well as previous studies that found the fusiform gyrus was the most probable neural generator of the N170 (Deffke et al., 2007; Rossion, Joyce, Cottrell, & Tarr, 2003).

Some studies have indicated that the pSTS is the primary source of the N170, instead of the mFFG or occipital areas (Batty & Taylor, 2003; Itier, Alain, Sedore, & McIntosh, 2007; Itier, Herdman, George, Cheyne, & Taylor, 2006; Itier & Taylor, 2004c). The functional role of pSTS in face perception has been attributed to representation of changeable aspects, such as facial expression, eye gaze, and lip movement (Fox, Iaria, & Barton, 2009; Haxby et al., 2000; Müller et al., 2018; O’Toole, Roark, & Abdi, 2002; Pitcher, Dilks, Saxe, Triantafyllou, & Kanwisher, 2011). For example, Pitcher et al. (2011) found much stronger responses for dynamic faces than static faces in the fMRI-defined pSTS face area, but not in fusiform or occipital areas, suggesting a functional dissociation between different brain regions for sensitivity of dynamic information. The difference between these studies and the current study is likely due to the use of static faces in our study and the use of dynamic and moving faces in those studies. We did find a difference in activity in the pSTS between the face and house stimuli, but this was small compared to the findings in the mFFG or pFFG. We did not find that the fMRI-defined occipital face area was substantially related to the sources of the N170 ERP component. This was consistent with our review of the literature (Richards et al., 2018) and the concept that ERP components occurring earlier than the N170 might be generated by this brain area (Pitcher, Walsh, Yovel, & Duchaine, 2007). Our findings are in line with recent models of face processing that distinguish between ventral and dorsal pathways for processing of static and dynamic face information (Duchaine & Yovel, 2015; Pitcher, Duchaine, & Walsh, 2014; Steeves et al., 2006).

There were two inconsistencies between the pattern of CDR to faces and houses and the fMRI BOLD responses. First, differential processing of faces and houses was not represented in dissociated CDR values in some brain regions. The BOLD activity in response to houses was larger in the lingual gyrus than it was to faces. This functional contrast defined the lingual gyrus house area ROI. However, CDR values for faces were also larger than houses even at this house-sensitive brain region. The inconsistent findings between the CDR result and BOLD contrast might be due to the different nature of fMRI and EEG data or some nature of the cortical source analysis method that was not assessed in our study. The clear relation between responses to the face stimuli and the CDR face-sensitive activity suggests that the N170 has a functional relation to the fMRI-defined fusiform face area but is not a house-sensitive ERP component.

Second, the CDR of the ERP indicated that a broader area surrounding the fusiform gyrus was more active to faces than houses than the fMRI BOLD response depicted. This was shown by an ROI analysis of the CDR (Figures 7, 8). Although the CDR activity showed the same functional relation to the faces and houses for the ventral visual areas (e.g., faces > houses; right faces > left faces), the pattern of localization of the fMRI activity was quite marked compared to the CDR activity in the same analysis (Figure 8). The 3D average plots of the CDR and BOLD (Figures 6, 9) were consistent with this finding. Though it is generally claimed that source analysis of ERPs has lower spatial resolution compared to fMRI, some studies have suggested a higher correspondence between the findings from LORETA source analysis and fMRI (Corrigan et al., 2009; Mulert et al., 2004; Vitacco, Brandeis, Pascual-Marqui, & Martin, 2002). It is possible that the inverse source reconstruction of the ERP signal results in an artificial spread of activity across larger areas than actually occur in the brain. The skull density acts as a low-pass spatial filter for the electrical activity reaching the scalp and smears the signal on the scalp (Hallez et al., 2007; Wolters et al., 2006). The inverse source reconstruction would thus lead to a larger spread of activity at the cortical level than actually occurred. It also may be that some aspect of the source analysis (e.g., anisotropy of the skull conductivity or white matter anisotropy, the eLORETA constraint) results in smearing of the modeled source reconstruction. Alternatively, it is possible that the techniques applied to the fMRI during processing result in more localized areas than are actually represented in individuals. This would be supported by the larger range of the fMRI activity in the individually defined functional ROIs (Figure 2) than in the average BOLD response (Figure 9). Our emphasis on individual analyses of the ERP source data compared to the average representations of the fMRI result may exacerbate this comparison.

The present study examined the neural sources of face processing. We found a consistent pattern of results for activation to faces > houses, and inverted > upright in ERP, cortical source analysis of the N170 on anatomical ROIs, and cortical source analysis of the N170 on functional ROIs of fMRI. These results support the fusiform gyrus, including both the mFFG and pFFG, as the most significant neural generator of the N170. The direct comparison of the fMRI ROI localizer provides additional evidence for the important role that the fusiform face area plays in generating the N170 component. A contribution that the current study makes to the literature is that cortical source analysis can also be conducted on functional ROIs identified with fMRI. This provided a direct quantitative comparison between the functional ROI defined by fMRI and the CDR activity found in ERP source analysis. This is one of the first studies using individualized anatomical head models to examine the neural sources of the N170 component. We believe that this contributes to the close association of cortical source generators for the N170 and the face-sensitive areas found with fMRI. This study provides further insights into the neural origin of the well-studied face-sensitive N170 component using improved methods and highlights the need to use individual-based methods in examining the neural correlates of face processing.