Structural face encoding: How task affects the N170’s sensitivity to race

Abstract

The N170 event-related potential (ERP) component differentiates faces from non-faces, but studies aimed at investigating whether the processing indexed by this component is also sensitive to racial differences among faces have garnered conflicting results. Here, we explore how task affects the influence of race on the N170 among White participants. N170s were larger to ingroup White faces than outgroup Black faces, but only for those required to attend to race, suggesting that attention to race can result in deeper levels of processing for ingroup members. Conversely, N170s were larger to Black faces than White faces for participants who attended to the unique identity of the faces, suggesting that attention to identity can result in preferential recruitment of cognitive resources for outgroup members. Taken together, these findings suggest that race can differentially impact face processing at early stages of encoding, but differences in processing are contingent upon one’s goal state.

Faces provide some of the most critical stimuli from which we glean social information. Perhaps not surprisingly, electrophysiological studies consistently reveal a face-sensitive event-related potential (ERP)—the N170—that is larger to human faces than to objects and non-human faces (Bentin et al., 1996). Initial research found N170 amplitude insensitive to variations among faces such as at the level of unique identity (Bentin and Deouell, 2000; Eimer, 2000), facial expressions (Eimer et al., 2003; Holmes et al., 2003; Ashley et al., 2004; Balconi and Lucchiari, 2005; Holmes et al., 2005; O’Connor et al., 2005) or social category distinctions, such as race (Caldara et al., 2003, 2004; Ito et al., 2004; Vizioli et al., 2010). From these results it was concluded that the N170 reflects structural face encoding but not mechanisms that differentiate among faces (Bentin et al., 1996; Bentin and Deouell, 2000; Eimer, 2000). More recent studies have challenged this conclusion by showing variations in the N170 to different kinds of faces (e.g. faces that differ in identity; Jacques and Rossion, 2006; Jacques et al., 2007).

This disparity of findings begs the question of when and how the N170 is sensitive to variations among faces. The present study examines this question with respect to face race, specifically examining how differences in perceivers’ goals influence the way in which race modulates the N170. While many studies find that the N170 (or its assumed dipole, the vertex positive potential) does not differentiate among faces of different races (James et al., 2001; Caldara et al., 2003, 2004; Ito et al., 2004), other studies find larger N170s to racial ingroup members (Ito and Urland, 2005), and still others show larger N170s to outgroup members (Herrmann et al., 2007; Gajewski et al., 2008; Stahl et al., 2008; Walker et al., 2008; He et al., 2009; Balas and Nelson, 2010; Brebner et al., 2011; Caharel et al., 2010; Stahl et al., 2010). A review of relevant research actually predicts such variability, suggesting that the way in which race affects N170s should depend on processing goals. We thus investigate how differences in perceivers’ goals influence the way in which race modulates the N170, which in turn bears on two broader theoretical issues. The first is to better understand face perception, particularly the degree to which structural face encoding precedes and is functionally separate from the encoding of other aspects of face identity. The second goal is to better understand the mechanisms of race perception.

N170 and race: Past research

One possible effect of race on the N170 can be derived from research on perceptual expertise. Larger N170s have been found to non-face stimuli with which participants have particular expertise (Caharel et al., 2002). For example, Tanaka and Curran (2001) found that dog experts showed enhanced N170s to dogs, but not to birds, while the opposite was true for bird experts. These results have been interpreted as reflecting a tuning of the perceptual system to the structural properties of ecologically relevant stimuli (Bentin et al., 1999). Such larger N170s to stimuli with which individuals are more familiar may specifically reflect the subordinate or individual-level processing that typically accompanies the development of expertise (Gauthier et al., 1999). Because perceivers commonly have greater contact and experience interacting with racial ingroup than outgroup members (Allport, 1954), perceptual tuning should result in larger N170s to ingroup than outgroup faces, a pattern that has been obtained by Ito and Urland (2005).

By contrast, a typicality-based account predicts larger N170s to racial outgroup faces. This prediction is derived from studies showing that N170 amplitude is increased in response to faces made atypical through digital editing (Halit et al., 2000) or inversion (Rossion et al., 1999), effects understood to reflect increased demands upon the extraction of configural information under unusual conditions. With respect to race, outgroup faces are generally processed in a less configural manner relative to ingroup faces (Michel et al., 2007), so if the task requires it, processing outgroup faces at more subordinate levels may demand increased recruitment of face processing resources. Consistent with this conceptualization, several studies have found increased N170s to racial outgroup faces (Herrmann et al., 2007; Gajewski et al., 2008; Stahl et al., 2008; Walker et al., 2008; He et al., 2009; Balas and Nelson, 2010; Caharel et al., 2010; Stahl et al., 2010; Brebner et al., 2011).

Although the expertise- and typicality-based accounts are seemingly inconsistent, the findings are actually complementary if perceiver goals are considered. Lacking any particular motivation to deeply encode outgroup faces, individual-level judgments are more likely for ingroup than outgroup members (Levin, 1996, 2000). This notion predicts that when task demands are low, perceivers will tend to process ingroup members more deeply than outgroup members, which the expertise perspective suggests will produce larger N170s to the ingroup. However, if the task requires subordinate-level processing of outgroup members, doing so should increase structural encoding demands (because they are not normally processed in this manner). This demand should result in a concomitant increase in the N170 for outgroup faces, in accord with prior studies on face typicality.

This analysis suggests that varying the motivation to attend to outgroup faces at a subordinate level will affect how race impacts the N170, a conclusion that is also consistent with extant research. The study producing larger N170s to racial ingroup members involved a relatively superficial processing goal (racial categorization) (Ito and Urland, 2005), whereas studies that require attention to subordinate-level features have yielded larger N170 amplitudes to outgroup faces (Herrmann et al., 2007; Stahl et al., 2008; Walker et al., 2008; Stahl et al., 2010). To date, however, the effects of race on face processing have been assessed with tasks that have varied between studies; no single study has examined these differences simultaneously, preventing a direct assessment of the potential for task goals to modulate the effect of race on the N170. In the present study, White participants viewed ingroup White and outgroup Black faces while attending to them either at a relatively superficial level (matching at the level of race) or an individual level (matching at the level of the unique individual). Our objective in selecting these tasks was to create one condition that facilitated differences in the processing of ingroup and outgroup faces (with ingroup faces processed at the subordinate level and outgroup faces processed more superficially at the category level) and another condition in which the task demands required both ingroup and outgroup members to be processed at the subordinate level. We also included a condition requiring only a face/non-face distinction largely for descriptive purposes because this type of task is frequently used in face-perception research (e.g. Bentin et al., 1996; Bentin and Deouell, 2000; Sagiv and Bentin, 2001; Caldara et al., 2003; Gajewski et al., 2008). We were uncertain how participants would encode the faces in the absence of any explicit instructions and so we had no strong predictions about race differences in the N170 in this condition.

METHODS

Participants

Fifty-four White undergraduate students (29 male) from the University of Colorado Boulder participated for course credit.

Materials

Five color pictures each of Black male faces, White male faces and butterflies were used. The Black and White faces were selected to be equal in attractiveness on the basis of normative attractiveness judgments obtained from a separate sample of participants (n = 31–57) (z-scores of attractiveness ratings M_Blacks = 0.76 and M_Whites = 0.54), t(8) = 0.89, P = 0.40, with equally high agreement in their racial categorization (M = 98.85% of pretest participants saying the face was ‘Black’ for Black faces and M = 99.35% saying the face was ‘White’ for White faces), t(8) = .51, P = 0.62. Faces were front-view with smiling expressions and no distinctive features. All stimuli were cropped to only show internal face features by placing a black oval over them and were displayed on a black background.

Procedure

Participants were randomly assigned to one of three task conditions: (i) an identity 1-back task requiring a manual response whenever the same exact person repeated over successive trials, adapted from Walker et al. (2008) (n = 18), (ii) a race category 1-back task requiring a response whenever two members of the same race repeated over successive trials (n = 18), and (iii) a butterfly task requiring a response whenever a butterfly appeared on the computer screen (n = 18).

The experimental task consisted of 195 total trials presented in a different order for each subject with the following constraints. Each unique stimulus was presented 12 times (180 trials), which included five instances each of Black faces, White faces and butterflies of a different identity following presentation of a stimulus of the same type (i.e. category repeats). Additionally, each unique stimulus appeared directly after an identical stimulus once (15 trials, i.e. identity repeats). This yields 165 trials on which the stimulus differed from the preceding one both in category and identity. Before beginning the experimental task, participants viewed 12 practice trials containing four of each stimulus type.

Each trial consisted of a fixation point presented at the center of the screen for 1000 ms followed by a centered face or butterfly stimulus for 500 ms. Trials were separated by an interstimulus interval randomized between 1000 and 2500 ms that occurred immediately after a response was made or 500 ms elapsed without a response. Participants were instructed to respond as quickly and accurately as possible to relevant stimuli.

Psychophysiological data collection and reduction

ERP data were recorded continuously from 58 scalp sites using tin electrodes sewn into an elastic cap (Electro-cap International, Eaton, OH, USA), referenced to a nose electrode. Additional electrodes were placed above and below the left eye and on the outer canthus of each eye to record vertical and horizontal eye movements. Electrode impedances were kept below 5 KΩ at all sites. ERP recordings were amplified with a gain of 500 by NeuroScan Synamps model amplifiers (Compumedics, Charlotte, NC, USA) with a bandpass of 0.1–30 Hz and were digitized at 1000 Hz.

The data were submitted off-line to a regression procedure to remove the effects of eye movements from the ERP (Semlitsch et al., 1986). Epochs, created beginning 200 ms before stimulus onset and continuing for 1000 ms post-stimulus, were visually inspected to remove remaining ocular or other artifacts due to movements. If an artifact was detected at any of the scalp sites, data from all of the sites were removed from further analysis for that trial. An average of 46 trials were retained per condition (Ms = 82.53–84.85%); this did not vary significantly as a function of stimulus type or task condition (all Fs < 0.50).

Figure 1 presents grand average waveforms as a function of task and stimulus type at P8, with the N170 to faces clearly visible (mean latency = 170 ms). N170 amplitudes are typically maximal over lateral temporal areas (Botzel et al., 1995; George et al., 1996; Eimer, 2000). A similar topography was obtained here (Figure 1, panel D), so the N170 was scored for each participant at P7 and P8 by identifying the largest negative deflection within 140–200 ms.

Fig. 1.

Grand average ERP waveforms to Black and White faces in the identity (A), racial category (B), and butterfly (C) tasks at P8. (D) Topography from 140 to 200 ms averaged across all face stimuli and tasks.

RESULTS

We first conducted a preliminary 3 (Stimulus: Black, White, butterfly) × 3 (Task Condition: identity, category, butterfly) × 2 (Laterality: P7, P8) mixed model analysis of variance (ANOVA) to verify that we replicate larger N170s to faces than non-faces. We obtained the expected main effect of stimulus type F(2, 50) = 18.73, P < 0.001, = 0.43, showing larger N170 amplitudes to both Black faces (M = −5.11 µV) and White faces (M = −4.96 µV) than butterflies (M = −2.38 µV), Fs(1, 51) = 16.46 and 13.76, Ps < 0.001 and s > 0.35, respectively. As we expected, participants in all conditions were able to readily perform the tasks, as shown by high overall accuracy rates (Ms = 93%, 97% and 99% in the identity, category and butterfly conditions, respectively). A one-way ANOVA showed no accuracy differences as a function of task condition, F(2, 51) = 2.24, P = 0.12. Of particular importance, a 2 (Race: Black, White) × 2 (Task Condition) ANOVA on only the identity and category conditions showed no task differences on accuracy of responses to Black and White faces (all Fs < 1.5, Ps > 0.25). Similar analyses were done on response latency. The one-way ANOVA showed a main effect of Task Condition [F(2, 51) = 7.09, P < 0.01, = 0.22], with faster responses in the butterfly (M = 406.72 ms) than identity (M = 507.06 ms) [F(1, 51) = 9.31, P < 0.01, = 0.15) or category conditions (M = 519.72 ms), [F(1, 51) = 12.29, P = 0.001, = 0.19]. This difference is to be expected from the category verification literature examining responses at different levels of categorization. Basic-level categorizations, the task being performed in the butterfly condition, are typically performed more quickly than subordinate-level categorizations (Rosch et al., 1976), what participants in the identity and category tasks are doing as they make distinctions within the category of faces. Focusing just on the conditions in which participants responded to faces, a 2 (Race) × 2 (Task Condition) ANOVA revealed no task differences on response latencies to Black and White faces, (all Fs < 1.0, Ps > 0.35).

Having verified the basic face/non-face N170 effect and our expectation of similar behavioral difficulty when responding to Black and White faces, all subsequent analyses focused only on responses to faces, using a 2 (Race: Black, White) × 3 (Task Condition) × 2 (Laterality) multivariate ANOVA (MANOVA). Analysis focused exclusively on trials that differed from the preceding trial on both category and identity. This was done to ensure that the effects of task were evaluated based on responses to the exact same stimuli from all participants, unconfounded by task differences in responses required to specific stimuli (e.g. identity and race repeat trials required responses from some but not all participants).¹ Analyzing only data from trials that did not require a response also eliminates any possible effects due to response-related activity (see also Walker et al., 2008).

Consistent with previous research showing a laterality effect of right hemisphere dominance (Small, 1983; Bentin et al., 1996), the N170 was larger at P8 (M = −4.53 µV) than at P7 (M = −3.78 µV); F(1, 51) = 3.98, P = 0.05, = 0.07. More importantly, the predicted Race × Task Condition interaction was significant, F(2, 51) = 6.06, P = 0.004, = 0.19 (Table 1). As predicted, in the identity condition, N170s were bigger to Blacks than to Whites, F(1, 51) = 4.07, P < 0.05, = 0.07.² The opposite was true in the category condition, with larger N170s to Whites than to Blacks, F(1, 51) = 6.57, P = 0.01, = 0.11. By contrast, race had no effect for participants in the butterfly condition, F(1, 51) = 1.65, P > 0.05, = 0.03.³

Table 1.

Mean N170 amplitude by task and stimulus race

	Race
Task	Black	White
Identity	−4.63 (3.71)a	−3.42 (5.10)b
Category	−4.79 (2.90)a	−6.32 (3.34)b
Butterfly	−5.92 (6.13)a	−5.15 (5.21)a

All values are in µV. Standard deviations are reported in parentheses. Means within the same row with different subscripts differ at P < 0.05.

We also conducted an additional control analysis to determine whether there were any low-level visual differences between our Black and White stimuli, and whether responses to such differences varied as a function of task. For this, we focused on the lateral occipital P1, which is sensitive to visual properties such as luminance (e.g. Handy and Khoe, 2005; Rousselet et al., 2008; cf. Brebner et al., 2011). The P1 was scored for each participant by identifying the largest positive deflection within 80–120 ms at O1/O2, PO5/PO6 and PO7/PO8 (mean latency = 113 ms). A 2 (Stimulus: Black, White) × (Task Condition: identity, category, butterfly) × 2 (Laterality: left, right) × 3 (Site: occipital, parietal occipital, lateral parietal occipital) ANOVA revealed no effects of race or task, suggesting that systematic physical differences between our Black and White faces—or their interaction with task—do not account for our N170 effects.⁴

DISCUSSION

The results reported here demonstrate that race can affect the N170, but its particular effects depend on task goals, a conclusion that converges with both extant theory and past research. When we systematically varied task, we find that White perceivers focusing at the level of individual identity show N170s that are larger to racial outgroup Black than ingroup White faces, but when focusing at the level of racial category, N170s are larger to ingroup White than outgroup Black faces. No effect of race on the N170 is found when the focus is on the distinction between faces and non-faces.

Implications for race perception

These results have several important implications. They first provide insight into the mechanisms that affect race perception. Converging lines of research show meaningful differences in our default approaches to ingroup and outgroup faces. Ingroup faces are processed in a more configural manner (Michel et al., 2007) and are more likely to be encoded at the level of unique identity, whereas outgroup members are encoded at the level of social category (Levin, 1996; 2000; Hugenberg et al., 2007). Overall, this reflects a deeper, more expert, more subordinate mode of processing ingroup as opposed to outgroup members. The results from our category task converge with this past research. Making racial categorization decisions is a relatively easy task, lacking any special demands that should discourage participants from these typical differences in processing. In this context, the N170—an ERP component known to increase with perceptual expertise and subordinate-level processing—was larger to ingroup White than outgroup Black faces.

The identity condition results further support the conclusion that superficial, category-level encoding is typical for outgroup faces. If ingroup and outgroup faces were both usually processed in a way that readily supports subordinate-level judgments, then the perceptual demands associated with responding at the level of identity should be the same for ingroup and outgroup faces. However, the typically more superficial processing of outgroup faces results in a greater need for processing resources to be selectively recruited when they must be processed at the subordinate level. Consistent with this, we find larger N170s from White participants to outgroup Black faces in the identity condition. These results also show that while individuation may not be the norm for outgroup face processing, it can be accomplished when the context demands it (cf. Hugenberg et al., 2007; Young and Hugenberg, 2012). At the same time, it is worth noting that while the identity task required attending at the subordinate level, it was still a relatively easy task, a point reinforced by the high level of accuracy displayed by participants performing it. There may be a limit to how well perceivers are able to overcome the chronic differences in the processing of ingroup and outgroup faces through the selective recruitment of additional perceptual resources when encoding outgroup faces, which might be revealed with more difficult tasks (e.g. an n-back task in which n > 1).

Although we tested participants from only one racial group, we do not think this limits our ability to assess how task affects responses to members of different races. Having participants from a single race can be problematic when assessing some effects of race because it confounds target group and ingroup/outgroup status (i.e. in our study, only White faces can be ingroup faces). While it is the case that faces of Blacks and Whites can differ in ways that affect perceptual processes (e.g. in low-level visual features, in the stereotypes that are activated), such differences do not provide a parsimonious account of our results. This is because target group effects would likely manifest as a race main effect, whereas our results show that the effect of race depends on task. This is not to say that target group effects have no impact on early perceptual processes, but rather that the effect of race on the N170 is not simply a function of a target group difference, and instead depends on the way in which faces of different races are being perceived. There is also no empirical evidence that low-level perceptual differences between our Black and White faces are responsible for our N170 effects. This was shown through the analysis of the P1 (cf. Vizioli et al., 2010; Brebner et al., 2011), a component sensitive to physical differences such as luminance that revealed no race differences here. It would, nevertheless, be interesting to confirm these effects in participants of other races.

This study was not designed to specifically address race effects on memory, but the results may be relevant to understanding the other-race effect (ORE), in which memory is typically better for racial ingroup than outgroup faces (e.g. Malpass and Kravitz, 1969; Meissner and Brigham, 2001). The greater perceptual expertise assumed to drive larger N170s to White faces from our White participants in the race categorization condition has also been used to explain the ORE (Rhodes et al., 1989; Tanaka et al., 2004). Similarly, the larger N170s to Black than White faces in our identity condition mirror improvements in memory for outgroup faces under individuation instructions (cf. Hugenberg et al., 2007; Young and Hugenberg, 2012). These parallels suggest the possibility that the racial differences in structural encoding observed in the present study may contribute to subsequent memory differences, although this remains to be examined directly.

Among participants encoding faces in terms of identity, N170s were larger to Blacks than Whites, whereas among those encoding in terms of race, N170s were larger to Whites than Blacks, directly in line with our predictions. One somewhat surprising aspect of the results, however, is that when responses to faces of the same race are compared across condition, it is the N170s to the White faces that appear to differ more as a function of task. We might expect processing of White faces to be more similar across task conditions, but individuation to increase encoding of Blacks. We suspect deviations from this pattern in our data are a product of the between-subject manipulation of task, which allows atheoretical between-subject differences to contribute to differences between tasks. We considered a within-subjects task manipulation, but worried about both fatigue and carryover effects (e.g. participants instructed to encode at the individual level might be sensitized to identity even when subsequently told to attend to another aspect of the stimuli). We thus chose a between-subjects manipulation, and in such a design, the most appropriate test of our predictions is from the within-subjects race effects that, as we noted, fully support our predictions.

While extant theory suggests clear predictions for the identity and category conditions, expectations for the butterfly condition were less obvious because we were uncertain how participants might process the faces in the absence of any specific direction from us about how to do so. Our inclusion of this condition was both more exploratory and descriptive, providing a link with past N170 studies that often evaluate responses to faces during a face/non-face task (e.g. Bentin et al., 1996; Bentin and Deouell, 2000; Sagiv and Bentin, 2001; Caldara et al., 2003; Gajewski et al., 2008). One possible expectation is that in the absence of a strong motive to individuate outgroup Black faces our White participants would devote more attentional resources to ingroup White faces. Instead, N170s were equally large to the Black and White faces, and if anything, N170s were directionally larger to outgroup Black faces (this difference was reliable in the ancillary analysis that included the target and repeat trials). These results might therefore be interpreted to indicate that left to their own devices, White perceivers more deeply encode outgroup Black faces, but such a conclusion is inconsistent with the large ORE literature (Maclin and Malpass, 2001). We can only speculate on the tendency for larger N170s to the Black faces, but one explanation consistent with extant research is that it represents slower habituation to the less familiar outgroup faces. That is, the superiority at individual-level processing for ingroup White faces may have allowed our participants to more quickly determine that they were seeing the same Whites faces repeated multiple times, whereas the same determination may have been slowed for the Black faces (essentially reflecting a cross-race effect). If this occurred, it would result in participants orienting to the Black faces across more of the trials, increasing the overall mean N170 to them.

It can be difficult to determine precisely what psychological process drives results in a condition like the butterfly one because it is characterized by a lack of strong instruction to engage in a particular type of encoding with respect to the faces, so our explanation is only speculative at this point. We do note that the absence of N170 race effects when faces are not task relevant is consistent with prior research (Caldara et al., 2003). Regardless of the pattern of results in the butterfly condition, what we think is most important is that in the two conditions in which we are more certain about how participants were encoding the Black and White faces, race differences confirm our predictions.

Implications for face processing

A second important implication of these results is to question the assumption that structural face encoding and the encoding of face features necessarily occur sequentially.

For nearly three decades, a cognitive architecture model originally posited by Bruce and Young (1986) has guided research on face perception. According to the model, face perception begins with the construction of an abstract invariant representation of the face. This structural code is then used in subsequent processing stages to extract socially relevant information, including identity-related and social categorization information (cf. Haxby et al., 2000). Because of the sequential ordering of perceptual operations, this model has been interpreted to mean that features differentiating among faces, such as race, do not affect initial structural encoding. The present results instead indicate that the earliest neural markers of face perception can be modulated by racial group membership. It may be that the Bruce and Young model is typically interpreted too narrowly. The model clearly states that featural information is extracted during the initial structural encoding stage, with such information serving as the basis upon which subsequent social processing operates. The present results indicate that this featural information may be acted on earlier versus later depending on task goals. N170s did not differ to ingroup and outgroup faces in the butterfly condition, suggesting that the assumption that operations sensitive to social category occur subsequent to structural encoding might be specific to situations in which differentiations among faces are not task relevant.

In sum, the N170 literature has been characterized by inconsistent effects of race and competing theoretical perspectives. The present analysis suggests that the findings and theories are not necessarily inconsistent, but rather depict responses under different goal states. Depending upon one’s focus of attention, race can significantly impact the perception of the same set of faces, with the same exact physical properties, in different ways. Moreover, research on out/ingroup person perception has historically given attention to what we do not or cannot do with outgroup faces, with less consideration of what we can do when motivated. The present study provides necessary insights regarding this latter, often unstudied area of research.

Conflict of Interest

None declared.

This work was supported by the National Institutes of Health Grant 1R01 MH071257 to T.A.I.