Electrophysiological correlates of face-evoked person knowledge

Abstract

Face recognition includes identifying a face as perceptually familiar and recollecting biographical information, or person-knowledge, associated with the face. The majority of studies examining the neural basis of face recognition have confounded these stages by comparing brain responses evoked by novel and perceptually familiar famous faces. Here, we recorded EEG in two tasks in which subjects viewed two sets of faces that were equally perceptually familiar, but which had differing levels of associated person-knowledge. Our results dissociated the effects of person-knowledge from perceptual familiarity. Faces with associated biographical information elicited a larger ~600 ms centroparietal positivity in both a passive viewing task in which subjects viewed faces without explicitly responding, and an active question-answering task in which subjects indicated whether or not they knew particular facts about the faces. In the question task only, person-knowledge was associated with a negative ERP difference over right posterior scalp over the 170–450 ms interval which appeared again at long latency (>900 ms).

An essential faculty in human social interaction is the ability to recollect biographical facts that compose an individual’s identity — such as his or her name, profession, place of origin, etc. — on the basis of that person’s physical attributes. The most evocative of such physical attributes is that person’s face. A psychological model by Bruce and Young (1986) divides this person recognition process into two main stages: first, a perceptual analysis stage in which a particular viewpoint of a face is matched with a viewpoint-invariant representation of that face, and a person identification stage that links the face to semantic information related to that person. But how does the brain implement these stages? An extensive body of research suggests that face processing involves the coordinated activity of a distributed network of brain areas (Haxby et al., 2000; Ishai, 2008). Several neuroimaging, electrophysiology, and neuropsychological studies have linked the perceptual analysis of faces to discrete regions of ventral occipitotemporal cortex (VOTC) and lateral occipitotemporal cortex (LOTC) (Allison et al., 1994; McCarthy et al., 1997; Kanwisher et al., 1997). The storage of person-knowledge (such as biographical facts) about individuals has been linked to regions of the ventral anterior temporal lobe (vATL) (Puce et al. 1999; Leveroni et al., 2000; Gainotti et al., 2010).

The low temporal resolution of neuroimaging and neuropsychological methods limits their value in investigating the temporal dynamics of face-recognition, and the timescale of these processes is rapid. In behavioral studies, the minimum reaction time (RT) measured by button press for judging that a face is familiar has been reported as ~360 ms (Barragan-Jason et al., 2013) and RTs for distinguishing faces from non-faces occur at least 100 ms earlier (see review by Fabre-Thorpe, 2011). Studies of fast saccades reveal that eye movements towards a face in a visual display can occur in as little as 100–110 ms (Crouzet et al., 2009). RTs include motor preparation and execution latencies, and almost certainly overestimate neural processing time. Intracranial EEG studies from subdural electrode strips and penetrating depth probes in epilepsy patients have identified face-specific event-related potentials (ERPs) whose latencies provide some limits on neural processing time (Allison et al. 1994; Allison et al. 1999; Puce et al., 1999). The N200 ERP, so named due to its mean peak latency of ~200 ms, has been recorded from the fusiform gyrus and nearby ventral occipital temporal cortex (VOTC) and exhibits categorical specificity for faces without sensitivity to person identity. The P350 ERP, with mean peak latency of ~350 ms, has been recorded over the ventral anterior temporal lobe (vATL) and does show sensitivity to person identity, as evinced by name-face priming tasks (McCarthy et al., 1999).

While the intracranial ERP studies are greatly informative, the placement of electrodes in patients is determined by clinical considerations and thus brain activity is not systematically sampled across individuals and across all cortical regions. This frustrates a finer spatio-temporal analysis of face processing. Studies of scalp-recorded ERPs afford the opportunity to investigate the time course of face processing non-invasively with systematic electrode placement across samples of neurologically normal individuals. This systematicity comes, however, at the cost of spatial resolution when compared to intracranial ERPs. Nevertheless, scalp ERP studies have discovered several face-selective ERP components over overlapping temporal intervals that exhibit different sensitivity to face perception and person-knowledge.

One of the best-studied face-selective scalp-recorded ERPs is the N170, a negative-voltage deflection with a mean latency of ~170 ms that is maximal in amplitude over right temporoparietal scalp (Bentin et al., 1996). The N170 has been shown to exhibit greater adaptation when the subject was shown the same face from two different angles compared to when the subject was shown two different faces from those same angles. This suggests that the structural encoding of a face — that is, the construction of a viewpoint-invariant representation of a face from a given view of a face — occurs at least as early as 170 ms (Caharel et al, 2009). However, other work has suggested that the N170 is not affected by a subject’s familiarity with a face, implying that while the N170 might be correlated with the structural encoding of a face, it does not correspond to the matching of the encoded face representation with a stored representation in memory (Amihai et al., 2011).

Two ERPs with an inferior temporal topography and a latency of approximately 250 ms, the N250 and N250r, have been studied in the context of face processing. The N250r is a negative deflection elicited by repeated presentations of faces (but not non-faces) that has a larger magnitude to familiar faces than to unfamiliar faces (Schweinberger et al., 2002; Schweinberger et al., 2004; Martens et al., 2006; Bindemann et al. 2008. Zimmerman and Eimer, 2013; Schweinberger, 2011). The N250, by contrast, is a potential elicited outside of the context of a stimulus repetition task that is more negative to faces than non-faces, and is more negative in response to personally familiar faces than unfamiliar faces (Nasr and Esteky, 2009; Caharel et al., 2014; Gosling and Eimer, 2011; Tanaka et al., 2006; Kaufmann and Schweinberger, 2012). Although it is possible that these ERPs represent different underlying neural processes, a potential explanation of both these potentials is that they reflect the activation of a memory trace for a given face (in contrast with the N170, which appears to reflect face perception rather than face memory). In the context of repetition tasks, this would explain why famous faces elicit a larger negativity when repeated than unfamiliar faces. For unfamiliar faces, the second appearance of the face activates a weak memory trace—that is, the single appearance of the face from several seconds earlier—whereas for familiar faces the memory trace is much stronger, reflecting not just the recent exposure to the face but any long-term memories associated with the face. Outside of repetition tasks, the logic is simpler; seeing a known face will trigger a memory trace, whereas seeing an unknown face will not.

One caveat of these studies, however, is that faces of familiar people are perceptually familiar while also having associated person-knowledge. Thus, it is not clear whether this early difference corresponds to perceptual familiarity or to differences in person-knowledge between familiar and novel faces. Several studies have attempted to disambiguate the effects of perceptual familiarity and semantic knowledge on the magnitude of the N250, but results have been mixed. Gosling and Eimer (2011) found that an N250 difference emerges for faces that are explicitly recognized but not for faces that merely appear visually familiar, suggesting that the N250 may be sensitive to semantic knowledge. However, Abdel-Rahman and Sommer (2012) report no N250 difference in response to faces with equal perceptual familiarity but differing levels of semantic knowledge. Further research, then, will be required to clarify the precise conditions over which N250 familiarity differences occur.

Other studies have examined longer latency ERPs in face recognition studies. Several investigators have identified a centroparietal ERP over the 300 – 500 ms post-face onset interval (sometimes referred to as N400f) that was more negative for famous compared to novel faces (Eimer, 2000; Bentin and Deouell, 2000). However, as noted earlier, famous faces are both perceptually familiar and rich in semantic associations. The reliance of these studies on famous faces leaves unclear whether these differences reflect perceptual familiarity or person knowledge. Similar to the N250, the N400f has also been reported to exhibit repetition effects to familiar faces; a common interpretation is that the N250 repetition effect reflects priming at a perceptual level while the N400f repetition effect reflects priming at a conceptual level, although the confounds related to famous faces mentioned previously make it difficult to establish this distinction with certainty (Neumann & Schweinberger, 2008; Jemel et al., 2003). It is important to distinguish N400f from N400, a negative ERP with a similar latency that has been associated with semantic processing, most often in the language domain. N400s are largest for content words whose meanings have not been primed by sentence context or by related words (Kutas and Federmeier, 2011; Nobre and McCarthy, 1994).

A positive ERP in the 400–600 ms range (P600f) with a maximum distribution over centroparietal scalp does appear to be sensitive to person knowledge distinct from perceptual familiarity. As summarized in Boehm and Paller (2006), this potential is more positive for famous compared to novel faces, and persists even when attention is diverted or when the face is task-irrelevant. This potential is often studied by comparing the brain’s response to famous and novel faces, but other approaches have been used. Curran and Hancock (2007) dissociated perceptual familiarity from semantic knowledge by employing a familiarity versus recollection paradigm. Subjects first learned face-occupation pairings, then 30 seconds later, were shown the faces again along with distractors and were asked whether they had seen each face before (a measure of recall, or episodic familiarity). If they indicated familiarity, they were then asked if they could remember the occupation associated with that face (a measure of recollection). Within the set of faces reported as familiar, a centroparietal ERP over 400–600 ms was more positive for faces for which the subjects successfully recalled the occupation than for faces for which the subjects did not. In this study, the learning and retrieval phases occurred closely in time and so it is unclear whether the memory representations were encoded episodically or semantically. A more naturalistic face-learning task may have more conclusively established whether the ERP differences task reflected semantic knowledge.

Adopting the distinctions among processing stages identified in the Bruce and Young (1986) model, a common claim is that the N250/N250r reflects the face recognition unit of face recognition, whereas the P600f and N400f both reflect the activation of person identity nodes (Boehm & Paller, 2006). One reason that the N250/N250r is generally taken to reflect the face recognition unit stage of face recognition is that it has been shown to be sensitive to the mere perceptual familiarity of faces independently of any semantic knowledge, since even repeating an entirely novel face will elicit an N250r modulation (Schweinberger et al., 2002; Schweinberger et al., 2004; Martens et al., 2006; Bindemann et al. 2008; Zimmerman and Eimer, 2013; Schweinberger, 2011). As the face familiarity effects found in the P600f and N400f occur temporally subsequent to the N250/N250r, and as they have a roughly similar latency and topography to potentials implicated in semantic knowledge in the context of language processing, they are generally taken to reflect a semantic rather than perceptual stage of processing.

As noted previously, many face recognition studies compared the brain’s response to famous and novel faces, and thus confounded perceptual familiarity and associated semantic knowledge. However, several studies have avoided this confound. Curran and Hancock (2007) dissociated perceptual familiarity from semantic knowledge in the context of short-term face learning, finding that faces with associated semantic knowledge elicited a larger centroparietal ERP over 400–600 ms than faces judged as merely visually familiar. This strengthens the inference that the P600f reflects non-perceptual knowledge (although in this particular case, it is unclear whether the knowledge is episodic or semantic). Rahman and Sommer (2012) employed a design that equated subjects’ perceptual familiarity to two sets of faces while varying the level of semantic knowledge in the context of long-term memory. They found a N400f and P100 difference between faces with associated semantic knowledge and faces that were merely perceptually familiar. No such differences occurred for the N170, N250, or P600f potentials. Kaufmann et al. (2009) employed a design in which they showed subjects faces of a set of individuals from several angles. For some of these faces, semantic information such as a name was also presented, whereas no such information was presented for other faces. The authors found no significant difference at the latencies of the N170, N250, N400f, or P600f ERPs between faces that had, or had not, been associated with semantic information (although they did find a difference between these faces over inferior temporal scalp over the 700–900 ms interval). This study differed from typical famous face studies in that the person knowledge was acquired over a short span of time. This may explain why semantic knowledge did not affect the N400f or P600f ERPs, which perhaps reflect knowledge that had been consolidated into long-term memory. Taken together, these studies provide important insight regarding how these various potentials might be differentially implicated in long-term versus short-term, and semantic versus perceptual face memory. However, given the relatively few studies that have carefully dissociated perceptual and semantic knowledge, further work is needed to clarify the precise sensitivity of various ERP components to the degree and type of person-knowledge associated with a perceived face.

In the current study, we investigated ERPs previously associated with face processing to determine when person-knowledge was accessed. Our focus was upon N170, N250, N400f, and P600 ERPs. We compared the brain’s response to two sets of faces to which subjects had equal perceptual exposure but differing levels of person knowledge. In this way, we sought to avoid the confounding of perceptual familiarity and semantic knowledge evident in many prior studies.

Person-knowledge is acquired over time and stored in long-term memory. To achieve a measure of ecological validity, we trained subjects over the course of several days with two sets of artificial faces to which they were initially naïve. For one, subjects learned unobservable biographical facts - name, occupation, and home state - associated with each face. For the other set, subjects learned outwardly visible physical facts. After training, subjects were tested in two tasks. In the Question task, subjects viewed a picture of each face, and then were shown a question about the face and had to indicate with a button-press whether or not they knew the answer. In the Passive-Viewing task, subjects simply viewed each face one at a time and were asked to reflect on what they knew about the face without making an explicit response.

Methods

Subjects

Sixteen right-handed, healthy young adults (9 males, average age 19.8 years old) with normal or corrected vision participated in exchange for compensation. All subjects were current university students or recent university graduates. Three additional participants participated on the first day, but did not meet the performance criteria (described below) and so were excluded from the remaining sessions of the study.

All procedures were approved by the Yale University Institutional Review Boards. All subjects provided informed consent and were paid for their participation.

Stimuli

Seventy-two face stimuli were generated with FaceGen 3.5 software (Singular Inversions, Toronto, Canada). Four facial features were systematically varied to produce faces with each possible combination of gender (male, female), race (African-American, Caucasian, Asian), age (young, middle-aged, old), and facial expression (slightly happy, slightly sad). Two distinct faces were generated for each of these 36 combinations (e.g., there were two middle-aged, happy, Asian males, and two older, sad, Caucasian females, etc.). For each participant, one face from each pair was assigned to the biographical-facts condition (Biographical), and one was assigned to the physical-facts condition (Physical). The biographical/physical assignment was flipped within each face pair for half the subjects so that, across subjects, each face served in each condition.

The biographical facts for each individual consisted of a name, location of origin (home state), and occupation. For the names, 18 popular male names and 18 popular female names were used. Thirty-six American states were arbitrarily chosen for home state, and thirty-six common occupations were chosen. These biographical facts were then randomly combined to create 36 combinations of name, state and occupation, which were then randomly associated with each pair of generated faces (with the constraint that the genders matched). The physical facts consisted each face’s gender, race, and eye-color, and matched the features of each individual face. Example faces for each fact set are shown in Figure 1.

Figure 1.

Example faces with (A) physical facts and (B) biographical facts as presented during the training period.

Procedures

The complete study consisted of five sessions conducted on different days: (1) a baseline EEG acquisition and initial behavioral training session, (2–3) two behavioral training sessions, (4) post-training fMRI acquisition, and (5) post-training EEG acquisition. Sessions were generally spaced no more than 48 hours apart. Due to scheduling constraints, two subjects had the order of their EEG and fMRI sessions switched, and two subjects had either one or two additional behavioral training sessions to prevent overly long (greater than 48 hours) gaps from preceding the post-training EEG session. The results of the fMRI study will be described in another paper, and will not be discussed further here.

The baseline EEG session was conducted prior to any behavioral training, and thus all of the faces were equally unfamiliar. Each of the 72 faces was displayed for 1500 ms. At face offset, a question appeared on the screen for a duration randomly chosen from a uniform distribution between 1300 – 1800 ms that directed the subject to recall either that face’s gender, race, or eye color (Figure 2). The purpose of this task was to direct subject’s attention to the faces. The subject had to indicate with a button-press whether they did or did not know the answer to the question. At the offset of the question, an inter-trial interval (ITI) randomized between 1500 ms and 2000 ms occurred before the next face was shown.

Figure 2.

Schematic of first pre-learning EEG session: The subject viewed all 72 novel faces in random order. After each face offset, the subject was immediately presented with a question about the gender, race, or eye color of the face and simply had to press the “f” key if he or she knew the answer to the question, and the “j” key if not. The face was presented for a fixed 1500 ms. The duration of the question was 1300–1800 ms (randomized uniformly over that interval). The subsequent face occurred after an inter-trial interval (ITI) of 1500–2000 ms (randomized uniformly). The subject could respond any time during the question or ITI. Every five trials a rest screen was presented, during which the subject could blink freely and continue at will by pressing any key.

During the behavioral training sessions, participants were trained on the facts associated with the 72 faces. Each training session was divided into six rounds. During each round, participants learned the facts associated with 12 faces (six Biographical and six Physical) by viewing a video in which each face was presented alongside the associated facts (see Figure 1) for 10 seconds three times each in a random order. Subjects then completed a self-paced quiz in which they sequentially viewed each face from that round and were asked to verbally state the three associated facts. A research assistant kept track of responses. During the quiz, each face had a visible label indicating whether it was a face associated with biographical or physical facts. By providing this label, we relieved subjects of the need to remember from which fact set a given face came. Following the quiz, participants were shown a review video with the correct answers for that round, consisting of the face alongside the associated facts.

After completing six rounds in this manner, participants were given a self-paced final test for the session in which they viewed all 72 faces sequentially and had to recall the associated facts for each one. For the first training session, participants had to meet a criterion of recalling 40 out of the 108 biographical facts in order to proceed with the study (as the physical facts were obvious from inspecting the face, they were not counted towards this criterion). Three of the 19 subjects who entered the study were unable to reach this threshold criterion and were dismissed from the study.

The final EEG session occurred after a minimum of three behavioral training sessions. There were two tasks. In the Passive-Viewing task, which was always conducted first, subjects viewed each of the 72 faces and were asked to think about what they knew about each face without making an overt response (Figure 3). Faces were presented in random order for 1750 ms with an offset-to-onset interval randomized between 6500–7500 ms. For the Questions task (Figure 4), each trial commenced with a visually presented word cue indicating whether the face they were about to see was from the Biographical or Physical fact set. This category cue was displayed for 1750 ms and was immediately followed by a face from the indicated set that was also displayed for 1750 ms. A visually presented recall prompt appeared for 1750 ms at face offset that asked about one of the three items of biographical or physical information concerning that particular face.

Figure 3.

Schematic of the Passive-Viewing task: The subject’s task viewed each face and was asked to think about what he or she knew about that face. The 72 faces were presented in a random order. Each face was presented for 1750 ms with an ITI of 6500–7500 ms. A rest screen was presented every five trials during which the subject could blink freely. The subject resumed the run by pressing any key.

Figure 4.

Schematic of the Questions task: Each trial began with a word cue (’Biographical’ or ‘Physical’) informing the subject of the fact category from which the upcoming face was drawn. The word cue was presented for 1500 ms. At the cue’s offset, the face was then presented for 1500 ms, immediately followed by a visually presented recall prompt. This prompt, presented for 1500 ms, directed the subject to recall one fact about that face. The subject’s task was to press the “f” key if he or she was able to remember the fact directed by the prompt, and “j” if not. The 72 faces were presented in a random order. An inter-trial interval of 3000–4000 ms then occurred before the onset of the next trial.

If the cue word cue was ‘Biographical’, the prompt directed the subject to recall one designated fact - either the name, occupation, or home state of the depicted face. If the cue was Physical, the prompt directed the subject to recall either the gender, race, or eye color of the depicted face. The subject was instructed to press the “f” key on a keyboard if he or she was able to remember the fact requested by the prompt, and the “j” key otherwise. In both conditions, the cue focused the subject’s attention to the face. However, a potential drawback to this design is that the subjects were cued about the nature of the information they would be asked to recall and that this may have altered their mental state prior to the onset of the face, thereby potentially altering baseline brain activity and complicating interpretation of the ERPs. However, the subject did not know the specific face that would be presented or the specific fact on which they would be questioned. Furthermore, the inclusion of the Passive-Viewing task allowed us to test for differential effects attributable to cuing in the Questions task.

Following the EEG session, participants were given another test of their knowledge of all 72 faces as in the training days. This final test allowed us to assess the extent to which subjects actually remembered the pertinent facts, since it was possible that subjects would misjudge their degree of knowledge during the task itself.

Summing across the initial EEG session, the three behavioral training sessions, and the fMRI session, each of the 72 faces was viewed at least 21 times prior to the final EEG session. As described above, some subjects received one or two additional training sessions. Critically, each subject saw each face from the Biographical and Physical fact set an equal number of times.

Data Acquisition and Analysis

EEG data was acquired continuously using a Compumedics Synamps RT amplifier system (Compumedics USA, Charlotte, NC). The EEG was recorded from Ag/Ag-Cl electrodes using a 64-channel Neuroscan Quik-Cap (consisting of all the electrodes in the 10–20 system along with interposed electrodes). The subject’s nose was used as a reference. Each EEG channel was amplified with a gain of 2200, and digitized at 500 Hz with 24-bit resolution and passed through a 0.05–100 Hz band-pass filter. Additional horizontal and vertical electrooculogram (EOG) channels were placed to monitor eye movements and blinks.

Single trial epochs beginning 100 ms before the presentation of each face to 2000 ms after the presentation of each face were extracted offline from the continuous EEG records The average of the 100-ms pre-stimulus period was subtracted from all data points for that epoch, and thus served as a baseline for amplitude measurements. Epochs contaminated with eye movements and blinks were identified using Matlab (MathWorks, Nattick, MA) scripts incorporating functions from the Fieldtrip analysis library (Oostenveld et al. 2011). Trials with a vertical or horizontal electrooculogram peak-to-peak amplitude over 200 μV, or a z-score over 4, or with any EEG channel’s z-score over 20 within the first 1000 ms post-stimulus were rejected from further analysis. A total of 12.3% of all trials (across EEG sessions 1 and 2) were rejected using these criteria.

As reviewed in the introduction, past research has identified particular ERP components with reported sensitivity to face familiarity and person-knowledge including the N170 and N250 components, an extended negativity beginning at ~170 ms over temporoparietal scalp, and two long duration potential differences over the 400–700 ms interval (N400f and the P600f centroparietal positivity). These ERPs and temporal intervals constituted our a priori measurement intervals. The average amplitudes for each component and across each interval was computed for each condition, electrode and subject. The difference in the average amplitude between the Biographical and Physical conditions measured for each subject was assessed with a two-tailed t-test at each electrode. The N170 was measured as the average voltage from 160–180 ms. Similar to Tanaka et al (2006), we measured the N250 as the average voltage from 220–320 ms. The N400f was measured across the latencies 320–400 ms and 400–480 ms, as per Eimer (2000). The P600f was measured as the average voltage from 550–650 ms.

Correcting for multiple comparisons in ERP data with many scalp electrodes is complex due to the issue of non-independence. A single neural generator contributes voltage to each electrode in a nonlinear manner whose magnitude and polarity of the contribution depends upon the location and predominant orientation of the neurons that compose the generator. Thus, each electrode measures every generator and therefore does not constitute an independent test. Here we took advantage of the scalp distributions of the difference waves over the measured intervals as previously reported for the ERPs discussed in the introduction to limit our a priori tests, restricting our a priori analyses to electrodes and latencies for which previous studies have noted ERPs related to face processing rather than testing all electrodes and latencies for a significant effect, a strategy that would have increased the likelihood of a type 1 error. The ERP data was unsmoothed for all measurements described below, and for all of the figures.

Results

Behavior

During the final EEG session, subjects responded ‘yes’ they knew the biographical fact requested by the recall cue on 92.9% of trials, and ‘yes’ they knew the physical fact requested on 96.2% of trials. This difference was not statistically significant (two-tailed t-test, p=.25). The subjects were also tested at the conclusion of the final EEG session (after the electrode cap was removed) and recalled 91.9% of all biographical facts and 100% of all physical facts, a difference that was statistically significant (two-tailed t-test, p=.0261). We note, however, that the biographical facts required memorization whereas the physical facts only required observation, so accuracy levels on these tasks are not directly comparable. Furthermore, although subjects performed slightly worse on the physical fact condition during the task than in the post-test, this can be attributed to the fact that they were under time pressure during the task and may not have had sufficient time to note fine physical details of the faces, such as eye color, whereas they were under no such time pressure during the post-test. Moreover, subjects performed roughly equivalently on the biographical fact condition between the task and the post-test, suggesting that their responses during the task accurately reflected their knowledge of the face facts.

Early interval including N170 and N250

An N170 ERP with a canonical right posterior temporoparietal scalp focus was clearly evident in the initial (Figure 5C) and final EEG sessions. However, we found no significant difference for N170 that differentiated faces from the Biographical and Physical fact sets in either the Question or Passive Viewing tasks (all electrodes p > .05 for either task).

Figure 5.

(A) Grand average time course of electrode TP8 for Questions condition and average topographic distribution of difference wave across all electrodes for 220–320 ms post-stimulus. Thick dashed line denotes face onset. Dotted lines at 220 ms and 320 ms denote the interval over which the ERP responses to the Biographical (blue line) and Physical (red line) faces were compared using a two-tailed t-test (p value indicated). The y-axis denotes amplitude in microvolts. The difference wave (green line) computed over this interval was averaged and the scalp topography of this average is shown in the topographic map. The location of TP8 on topographic map is denoted by black dot with white outline and the color bar represents average microvolts. (B) Same as A, but for Passive-Viewing condition. (C) Similar to A and B but for ERP responses to faces prior to the participant being trained. Here, the categories Biographical and Physical correspond to how the subject will be trained; however, these ERPs were collected prior to training when all faces were novel. The topographic map depicts the distribution of the N170 ERP (averaged across all faces) measured at 160 ms post-stimulus, as denoted by the dotted line. Thus, no significance values are represented. The green waveform depicts the difference wave as described above.

The N250 measurement (220–320 ms) showed significant difference at the TP8 electrode where faces from the Biographical fact set elicited a significantly more negative response than those from the Physical fact set (p=0.037), but for the Question task only. The topography of the difference wave (Biographical minus Physical) revealed a posterior temporoparietal scalp focus similar to that reported for N250, although more strongly right sided (Figure 5A). The distribution for N250 was also similar to that measured here for N170 (compared topographic maps in Figures 5A and 5C). The difference wave (green line, Figure 5A) was prolonged, beginning at ~170 ms and persisted until ~470 ms – similar to that observed by Caharel et al. (2014). No significant differences in this interval were observed for the Passive Viewing condition (all ps > .05; Figure 5B).

Late interval (400–700 ms)

The difference wave between ERPs evoked by faces from the Biographical and Physical fact sets revealed a centroparietal difference with a greater positive voltage evoked by the faces from the Biographical set (P600f, Figure 6A–B). This widespread positive difference was evident for both the Questions (Figure 6A) and Passive Viewing (Figure 6B) tasks, although the onset of the difference wave occurred ~200 ms earlier for the Questions task (duration of positive difference from ~300–700 ms) than for the Passive Viewing task (duration of positive difference from ~500–800 ms). Over the P600f measurement interval of 550–650 ms, the positive difference for Biographical fact set faces was significant in both the Questions task (p< 0.05 for 23 electrodes over the occipital, parietal, and central scalp) and Passive-Viewing task (p<0.05 for 10 electrodes over central and parietal scalp). Comparing the two tasks, the difference waves have qualitatively similar scalp topographies over this measurement interval, although the topography of the Passive-Viewing task is shifted slightly to the right hemisphere. Not surprising, the ERPs evoked by the Biographical and Physical faces before training exhibit nearly identical ERPs over this interval (Figure 6C). By contrast, we found no evidence for a centroparietal N400f. There were no significant differences between the Biographical and Physical face sets over the interval 320–400ms in either the Questions task or the Passive-Viewing task. Although there were significant differences over the 400–480ms interval for the Questions task, these were in the opposite direction of the previously reported N400f (with the Biographical faces eliciting a more positive response than the Physical faces) suggesting that these differences reflect the onset of P600f.

Figure 6.

(A) Grand average time courses for electrode Pz for the Questions condition and average topographic distribution of difference wave across all electrodes for 550–650 ms post-stimulus. Thick dashed line denotes face onset. Dotted lines at 550 ms and 650 ms denote the interval over which the ERP responses to the Biographical (blue line) and Physical (red line) faces were compared using a two-tailed t-test (p value indicated). The y-axis denotes amplitude in microvolts. The difference wave (green line) computed over this interval was averaged and the scalp topography of this average is shown in the topographic map. Location of Pz on topographic map is denoted by black dot with white outline and the color bar represents average microvolts. The black dots on the topographic maps refer to electrode sites where the average difference wave was significant (p<.05). (B) Same as A, but for Passive-Viewing condition. (C) Similar to A and B but for ERP responses to faces prior to the participant being trained. Here, the categories Biographical and Physical correspond to how the subject will be trained; however, these ERPs were collected prior to training when all faces were novel. Here, the topographic map indicates the average amplitude to all faces, and not the difference wave. Thus, no significance values are represented.

Very late interval (900–1500 ms)

Although we did not have a priori measurements planned for the late ERP interval, the Biographical minus Physical difference wave for the Questions task revealed a remarkably consistent difference beginning at ~900 ms and growing in amplitude until the end of the face presentation. Figure 7A illustrates the scalp topography for this difference over the interval 1500 – 1600 ms for the Questions task, where the difference was significant at p<.05 at 34 electrodes across occipital, parietal, and central scalp. The difference wave topography was negative over both left and right temporoparietal scalp with a more extensive negative distribution over the right scalp. There were no significant differences in this same interval for the Passive Viewing task (all ps > .05) (Figure 7B).

Figure 7.

(A) Grand average time course of electrode TP8 for Questions condition and average topographic distribution of difference wave across all electrodes for 1500–1600 ms post-stimulus. Thick dashed line denotes face onset. Dotted lines at 1500 ms and 1600 ms denote the interval over which the ERP responses to the Biographical (blue line) and Physical (red line) faces were compared using a two-tailed t-test (p value indicated). The difference computed over this interval was averaged and the scalp topography of this average difference is shown in the topographic map. Location of TP8 on topographic map is denoted by black dot with white outline. The black dots on the topographic maps refer to electrode sites where the average difference wave was significant (p<.05) Location of TP8 on topographic map is denoted by black dot with white outline. (B) Same as A, but for Passive-Viewing condition. The average difference was not significant at any electrode site.

Result Summary

Figure 8 summarizes the central findings of our study in the form of Biographical condition minus Physical condition difference waves for left temporoparietal, centroparietal, and right temporoparietal electrodes. Within the Question task, the right temporoparietal (TP8) electrode illustrates all three findings of our study: an initial N250 difference, a P600f difference and the late difference. The P600f and late difference are evident at all three electrodes, with the P600f difference maximal at the centroparietal electrode, CPz, and the late difference larger at both the left and right temporoparietal electrodes. Table 1 summarizes the p-values associated with all of the effects that we identified.

Figure 8.

The difference waves (Biographical – Physical) for the Questions (left) and Passive Viewing (right) tasks for the TP7 (left temporoparietal), CPz (centroparietal), and TP8 (right temporoparietal) electrodes. Moving from the left to right, the dotted lines within each waveform plot indicate the N250, P600f, and late difference intervals. The areas under the curves are shaded to emphasize the measurement intervals. The y-axis denotes amplitude in microvolts.

Table 1.

Significant (or trending) p-values for the two-sided t-test comparing the ERP response to the biographical versus physical condition faces.

Task	Measured Latency (ms)	Electrode P-Values
Question	220–320ms	TP8: 0.0371, P8: 0.0739
	550–650ms	FC4: 0.0428, T7: 0.0386, C3: 0.0234, C1: 0.0094, Cz: 0.0152, C2: 0.0178, C4: 0.0212, CP5: 0.0034, CP3: 0.0054, CP1: 0.0030, CPz: 0.0058, CP2: 0.0072, CP4: 0.0120, CP6: 0.0302, P7: 0.0237, P5: 0.0406, P3: 0.0053, P1: 0.0242, Pz: 0.0105, P2: 0.0320, P4: 0.0245, PO5: 0.0474, PO3: 0.0098, I1: 0.0241, O1: 0.0181, Oz: 0.0117, O2: 0.0439, I2: 0.0437
	1500–600ms	FC3: 0.0436, FC4: 0.0450, C5: 0.0306, C3: 0.0321, C2: 0.0421, C4: 0.0367, C6: 0.0189, T8: 0.0134, TP7: 0.0158, CP5: 0.0402, CP3: 0.0498, CPz: 0.0284, CP2: 0.0208, CP4: 0.0083, CP6: 0.0109, TP8: 0.0108, P7: 0.0393, P3: 0.0245, P1: 0.0179, P2: 0.0104, P4: 0.0287, P6: 0.0045, P8: 0.0073, PO7: 0.0135, PO5: 0.0437, POz: 0.0357, PO4: 0.0078, PO6: 0.0054, PO8: 0.0046, I1: 0.0423, I1: 0.0423, O1: 0.0473, Oz: 0.0437, O2: 0.0139
Passive-Viewing	550–650 ms	C1: 0.0319, C2: 0.0382, C4: 0.0422, CPz: 0.0447, CP2: 0.0326, CP4: 0.0326, M2: 0.0412, P1: 0.0322, Pz: 0.0462, P2: 0.0441

Discussion

In this study, we dissociated perceptual familiarity and person knowledge for three ERP correlates of face processing evident in scalp recordings, while failing to dissociate these processes for other face-related ERPs previously described in the literature. Subjects were shown initially novel and artificial faces that were associated with either a triplet of biographical or physical facts over three or more training sessions. The age, race, gender, and facial expression of the novel faces were balanced (in pairs) across fact-sets, and care was taken to ensure that the subjects saw each face the same number of times. Although the biographical facts were more taxing to learn, subjects performed at near ceiling on the biographical facts recall task following the final EEG session. Importantly, performance did not differ between the biographical and physical fact-sets during EEG acquisition for the Questions task.

Consistent with prior studies (Amihai et al., 2010; Gosling and Eimer, 2011), we did not observe a difference attributable to person knowledge in the amplitude of N170, an ERP with a (primarily right hemisphere) temporoparietal scalp distribution that has been reliably associated with face processing, and which has been proposed to represent the structural encoding of faces (Bentin et al. 1996). However, while we observed no significant difference in N170 proper, we did observe differences beginning immediately afterwards in the form of a focally distributed, right-hemisphere temporoparietal difference wave between the Biographical and Physical faces. This difference started at ~170 ms and persisted until about 470 ms and was only present for the Questions task. This difference wave was similar in latency and topography to the N250 (Schweinberger et al, 2002; Tanaka et al, 2006), and so it is reasonable to consider whether it represents the same phenomenon.

The N250 has typically been reported in tasks that compare repeated to non-repeated faces and exhibits greater modulation for repeated faces. In such tasks, the N250 effect rapidly diminishes over time between the first and second face repetitions (Schweinberger et al., 2002). Why might we see such a difference, then, if the two categories of faces were viewed the same number of times, and if face repetitions do not occur within each of our two tasks? Our preferred explanation is that the N250 is sensitive to person-knowledge about individuals, and does not reflect purely perceptual processing. However, we must consider the possibility that the Biographical faces were more deeply encoded perceptually as well as semantically throughout training, since the task associated with them (recalling relevant facts from memory) was likely more demanding than the task associated with the Physical faces (recalling physical characteristics of the faces). Some evidence consistent with this explanation can be found in Tanaka et al. (2006), who found that repeated exposures of an initially-novel face could cause an N250 to evolve over time, but only if the face was task-relevant; i.e., task-irrelevant faces that were viewed the same number of times did not cause an N250 to evolve. In our task, however, faces from both fact-sets were task relevant and required a response about a physical or biographical fact when the face was no longer present on the screen.

One pertinent aspect of our observed N250 difference between the ERPs evoked by faces from the Biographical and Physical fact sets is that it occurred in the Questions task, but not the Passive Viewing task. In the Questions task, the preceding word cue informed the subject whether the upcoming face would have associated biographical facts and, compared to the Passive Viewing task, this could have caused the biographical information to be retrieved earlier in time. Caharel et al. (2014) identified a recognition effect with a similar topography and latency as reported here, by comparing novel faces to faces that were personally familiar to the subjects. Our results, then, potentially clarify their findings by establishing that the effect they observed does not arise from mere differences in perceptual exposure to the two classes of faces, since perceptual exposure was equated in our study. Also consistent with this interpretation is a study by Gosling and Eimer (2011) in which famous faces that the subjects explicitly recognized elicited a more negative potential at this latency and with similar topography than famous faces that were judged as merely familiar. We note that in the present study, as in those studies just mentioned, the face familiarity difference occurred outside the context of the immediate repetition tasks used to identify the N250r (Schweinberger et al., 2002; Schweinberger et al., 2004; Martens et al., 2006; Bindemann et al. 2008. Zimmerman and Eimer, 2013; Schweinberger, 2011). This provides further evidence that immediate repetition is not essential to produce familiarity effects at this latency and spatial location.

We observed a second significant difference between the ERPs evoked by the faces from the Biographical fact set compared to the Physical fact set over the roughly 400–700 ms time window with a peak difference occurring near 600 ms. Unlike the N250 discussed above, this later difference wave was largest over centroparietal scalp and was significant for both the Questions and Passive Viewing task. Our observation corroborates past work identifying a positive centroparietal potential in the 300–700 ms range (P600f) (Curran and Hancock, 2007; Boehm and Paller, 2006; Eimer, 2000). Here P600f was larger for faces for which biographical facts were known, consistent with prior findings indicating that P600f was larger when subjects viewed faces with associated semantic information. In the present study, each face had the same degree of perceptual familiarity, and thus we conclude that P600f is sensitive to person-knowledge about faces.

Although the P600f difference was significant for both the Passive Viewing and Questions tasks, we noted that the difference wave between the ERPs evoked by the Biographical and Physical faces rose earlier in the Questions than Passive Viewing task. One likely reason for this task difference is that, in the Questions task, the face was preceded by a cue that indicated whether or not the face was from the Biographical or Physical fact-set. No such cue was provided for the Passive Viewing condition. Thus, when viewing a face in the Passive Viewing condition, the face itself was the cue for its fact set membership, and this additional processing would presumably influence the latency of the difference wave.

The P600f difference in the Passive Viewing task occurred despite no requirement for the subject to make an overt response. We did, however, ask subjects to reflect upon their knowledge of each face, and thus our data do not speak to the automaticity of access of person-knowledge. However, prior studies that employed famous faces in which person-knowledge was not task-relevant also elicited a voltage difference in this time range (Eimer, 2000). This is at least suggestive that the access of some person-knowledge is evoked incidentally by that person’s face. Also relevant to our interpretation are the behavioral studies of Ellis et al. (1990) who found that the identity associated with a face appears to be processed in an obligatory manner, regardless of the task context. Thus, our P600f difference for the Passive Viewing task may represent the neural basis of this automatic activation of person identity information. One important difference between our study and many studies that have examined the P600f is that our study compared faces with equal amounts of perceptual familiarity but differing levels of associated semantic information, whereas other studies compared famous faces (that were both semantically and perceptually familiar) to novel faces. Thus, whereas for many past studies it has remained unclear whether any ERP differences reflect semantic or perceptual familiarity, in the case of this study we can conclude with more certainty that the process indexed by the P600f is semantic in nature.

The P600f was maximally distributed over centroparietal scalp. It is notable that the latency and topography of this effect corresponds roughly with a commonly-observed variant of the difference-due-to-memory, or Dm, ERP, which is revealed for items in a list that were subsequently-remembered when compared to items in that same list that were subsequently forgotten (Paller, et al., 1988) and which is thought to represent differences in the strength of encoding. Dm can have a varying topography and timecourse depending on the nature of remembered material and the precise task the subjects perform, but often has a centroparietal distribution at the latency we observed (Otten and Rug, 2001; Wilding and Sharpe, 2003). Moreover, the Dm effect is not specific for faces, but occurs in subsequent memory experiments using stimuli of different modalities (Guo et al, 2004; Lian et al., 2002). However, relevant to our study, Guo et al. (2005) identified Dm potentials with distinct topography depending on whether just faces, just names, or face-name pairings were being remembered, suggesting a degree of specificity in the way this different information is encoded. Additionally, Yovel and Paller (2004) performed a study in which they had subjects learn both the faces and occupations of various individuals, later assessing whether the subjects remembered the occupation and found the face familiar, merely found the face familiar without remembering the occupation, or did not remember the face at all. They found that during encoding, trials in which subjects subsequently remembered the occupation of the face being presented elicited a significantly greater centroparietal positivity at 600–800ms (resembling the “canonical” Dm effect) compared to trials in which the subjects did not remember the faces at all, with the familiar-versus-unremembered contrast revealing a smaller though still present effect. Although their measurement was made during encoding, this finding bears an interesting parallel to our result obtained during retrieval; i.e., that semantic knowledge increased centroparietal positivity at ~600 ms. Future studies might examine the extent to which successful encoding and successful retrieval of person knowledge involve the same underlying neural processes.

We failed to identify a centroparietal difference wave corresponding to N400f in our data (Eimer, 2000; Bentin and Deouell, 2000; Boehm and Paller, 2006). One possible explanation is that N400f does not differentiate semantic knowledge from perceptual familiarity; i.e., faces with more semantic information also happened to be those that were perceptually familiar. Thus, previously reported N400f differences (which typically compared famous faces to novel faces) may have corresponded to differences in perceptual familiarity, which we did not observe since we equated subjects’ perceptual exposure to the two sets of stimuli (note, however, even some work using famous faces failed to find an N400f effect; see Gosling and Eimer, 2012). This interpretation is consistent with the findings of a study by Curran and Hancock (2007), who found that a potential at 400 ms distinguishes between perceptually familiar and unfamiliar faces, but does not distinguish between perceptually familiar faces that have associated semantic information from those that do not. However, the ERP difference identified at N400f by these researchers had a frontal distribution, in contrast with the centroparietal distribution of the N400f reported by other researchers, and so it is a matter of some continuing debate whether the former potential (commonly referred to as the FN400) represents the same phenomenon as the latter (Bridger et al., 2012; Voss and Federmeier, 2011). Since we did not find either a centroparietal or frontal negative difference, however, we cannot contribute to this debate. Eimer (2000) suggests that the N400f may only arise in response to faces that are task-relevant and that are the focus of attention; however, since the faces in our study were always task-relevant, this does not explain the absence of the N400f in our study. It is important to note that several studies have found that the N400f effect decreases with repeated presentations of a face (Neumann & Schweinberger, 2008; Jemel et al., 2003). This introduces a potential confound into our study; since the Passive-Viewing task always preceded the Question task, and so during the Question task each subject was seeing each face for a second time within the same approximately 20-minute period. However, at least one study that has explicitly examined repetition effects on an N400f difference between novel and famous faces has found that any reduction in the N400f magnitude happens gradually rather than precipitously; one exposure to a face is not sufficient to obliterate the effect entirely (Eimer, 2000). Thus, it is unlikely that our ability to find an N400f effect is due to the one-time repetition of face stimuli between the Passive-Viewing and Question tasks. As a final note of caution, it is possible that repeated exposures to the faces over the five days of the study may constitute a sort of priming that could have eliminated a potential N400f effect, but to our knowledge no positive evidence supports the possibility that N400f reduction effects occur over that interval.

The final positive ERP identified in this study was a late difference between the Biographical and Physical faces began at ~900 ms post-face onset and grew larger in amplitude until the face offset. This difference was maximal over temporoparietal scalp, a similar scalp distribution to the N250 discussed above, although more bilateral. Like the N250 difference, this late difference was only significant in the Question task. To our knowledge, this late ERP difference has been previously unreported in the context of face recognition. Unlike the other ERP differences reported herein, this late difference was not predicted a priori and thus could be an adventitious finding. However, the difference was quite prominent in amplitude and was statistically significant at 34 scalp electrode sites, suggesting it may be a real phenomenon.

We believe that this difference likely reflects neural activation associated with either continued access to, or rehearsal of, biographical facts. The subject is unaware of what single biographical fact of the triplet will be requested by the prompt appearing at face offset. Thus, this late difference may reflect either the persistence of an access process initially reflected by N250 or continued rehearsal of the relevant facts. Interestingly, this difference closely resembles a hemifield-dependent negativity potential that was identified by Schweinberger et al. (1994) and shown to be dependent on the cognitive task being performed rather than being obligatory and stimulus-triggered.

In conclusion, we observed robust differences between ERPs evoked by faces associated with biographical facts compared to equally perceptually familiar faces that were associated with physical facts. These differences were task-dependent, in that the early and very late negative differences over temporoparietal scalp were significant only in the Questions task, which required subjects to recall the biographical facts associated with the presented face in order to respond to a memory prompt at face offset. However, a significant difference in P600f over centroparietal scalp occurred for both the Questions and Passive Viewing conditions, although the onset of the difference wave was later in the Passive Viewing condition when subjects were not cued whether the face to be presented was associated with biographical facts. Taken together, these results strongly suggest that person knowledge retrieval is dependent upon task – particularly the timing of that retrieval. This also indicates that Passive Viewing tasks, as used here and which are popular in ERP studies of face processing, are not ideal for studying memory access associated with faces.

Despite this inadequacy, the Passive Viewing task revealed a vexing complication in our results. The results derived solely from the Questions task can be explained as follows: The N250 difference suggests the initiation of a retrieval process for biographical information associated with the current face. The success of this process is presumably reflected in the P600f. The rehearsal of this information to guide the response to the memory prompt at face offset is reflected in the very late difference that reaches its peak just before the memory prompt. However, if the N250 difference reflects retrieval of biographical information and P600f the success of that process, how can we observe a (albeit delayed) P600f in the Passive Viewing task without an antecedent N250?

We suspect that the cue preceding the face in the Questions task synchronized the retrieval process to face onset, and thus facilitated our observation of this difference. In the Passive Viewing task, the retrieval process may have had a variable onset, and perhaps for some faces, it was not initiated at all. As a consequence, the (larger amplitude) P600f difference would have a similarly variable onset. The temporal variability of the N250 and subsequent P600F would smear and attenuate the amplitude of the N250, and result in a later apparent onset of P600f, as we observed. While this explanation is not entirely satisfactory, it is consistent with our observations. Indeed, the shape of the topography for the N250 interval in the Passive Viewing task is very similar to that of the Questions task with a right temporoparietal minimum. Consistent with the interpretation that the cognitive process indexed by the N250 is present regardless of task, Boehm and Sommer (2005) found that the N250 difference between novel and famous faces emerges regardless of whether explicitly recognizing the face is necessary for the task. Thus, the fact that we did not find an N250 difference for the Passive Viewing condition may be a byproduct of differences in neural timing between the two tasks we used rather than an indication that the process indexed by the N250 did not occur.

Highlights.

• Compared ERP responses to faces with or without biographical info

• Faces were equated for perceptual familiarity through training

• Faces with biographical info elicited more positive ~600 centroparietal ERP

• These faces caused an early and sustained right posterior negativity in one task

• We failed to find an N400 or N170 difference between the face sets