Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Mar 23.
Published in final edited form as: J Cogn Neurosci. 2006 Aug;18(8):1343–1358. doi: 10.1162/jocn.2006.18.8.1343

Modulation of face-sensitive event-related potentials by canonical and distorted human faces: the role of two visual structural properties

Viola Macchi Cassia 1, Alissa Westerlund 2, Dana Kuefner 1, Charles A Nelson 2,3
PMCID: PMC3063065  NIHMSID: NIHMS236734  PMID: 16859419

Abstract

The study examined the sensitivity of early face-sensitive ERP components to the disruption of two structural properties embedded in faces, namely up-down featural arrangement and vertical asymmetry. Response times and ERPs were recorded as adults made an orientation judgment on canonical faces and distorted faces that had been manipulated for either or both of the mentioned properties.

The P1, the N170 and the VPP allexhibited a similar linear increase in amplitude or latency as the properties were disrupted in the order of 1) up-down featural arrangement, 2) vertical symmetry, and 3) both up-down featural arrangement and vertical symmetry. Exceptions to this finding were seen for the amplitudes of the N170 and VPP, which were larger for the stimulus in which vertical symmetry was disrupted. Interestingly, the enhanced amplitudes of the N170 and VPP are consistent with impaired behavioral performance on the orientation judgment observed for this stimulus.

Keywords: ERP, faces, structural properties

Introduction

Human faces form a class of complex, visually homogeneous stimuli, all sharing a basic structure; the same set of features arranged in the same fixed layout (i.e., two horizontal eyes above a centrally located vertical nose and horizontal mouth). Over the past 20 years, a great deal of research has been focused on attempting to identify the neurocognitive operations involved in the sophisticated ability humans show in detecting such a unique layout within their visual world.

Event Related Potential (ERP) studies have identified three visual components related to the early encoding stages of face processing: the P1 (Taylor, 2002), N170 (after Bentin, Allison, Puce, Perez, & McCarthy, 1996), and the VPP (Vertex Positive Potential; after Jeffreys 1989). The P1 is a visual positive component generated from the striate and extrastriate cortex (Gomez et al., 1994), and appears at occipital leads around 100-120 ms following stimulus onset. Albeit scarcely investigated in studies on face processing, the P1 has been recently found to be significantly larger to faces than to different categories of objects (Itier & Taylor, 2004-a). These findings led to the claim that the P1 may reflect an early global response to faces, perhaps reflecting a holistic processing stage enabling the perception/detection of first order properties, those which define a face as a face (Itier & Taylor, 2002, 2004-b). Nevertheless, the contribution of low-level visual properties (see discussion in Itier & Taylor, 2004-a) or attentional, task-dependent factors (see Taylor, 2002) to the observed differences between faces and other visual objects at the P1 still remains controversial.

Much more frequently measured and consistently observed, the N170, a negative component occurring between 140 and 200 ms over occipito-temporal regions, is widely considered as the earliest reliable marker of a processing difference between faces and objects, being systematically larger and often faster to faces than to a variety of other object categories (Bentin et al., 1996; Botzel, Schulze, & Stodieck, 1995;, Itier & Taylor, 2004-a; Rossion, et al., 2000). Because the N170 is unaffected by face familiarity (Bentin & Deouell, 2000) and resistant to selective attention (Cauquil, Edmonds, & Taylor, 2000), it has classically been thought to reflect the “structural encoding” stage of face processing (Bruce & Young, 1986) at which face components are detected and initially processed independently of recognition of personal identity or facial expression (after Bentin et al., 1996).

The VPP, a large positive potential appearing at centro-frontal sites over the same time window to the N170, also appears to respond differentially to faces, being larger and sometimes faster to face than non-face visual objects (Botzel et al., 1995; Jeffreys, 1996; Rossion, Joyce, Cottrell, & Tarr, 2003). Because of its temporal synchronicity with the N170, its opposite polarity and surface localization, and its remarkable functional similarity (e.g., Itier & Taylor, 2002, 2004-b; Rossion et al., 1999), some authors have argued that the VPP is the positive counterpart of the N170, the two components being part of the same dipolar complex (Rossion et al., 2003; Joyce, & Rossion, in press).

The demonstration that the N170 and the VPP, and according to some authors (Itier & Taylor, 2004-a) also the P1, discriminate between faces and non-face objects led to the description of these components as “face-sensitive”. Nevertheless, their specificity to faces is a more debated issue. For example, not much is known about the extent to which these early components are tuned to the specific geometry of the face, specifically, the typical spatial arrangement of the inner facial features.

The most consistent evidence concerning the impact of configural changes on early face-sensitive ERP components relates to the effects produced by stimulus inversion. Face inversion consistently and reliably delays the latency of the N170 (Bentin et al., 1996; de Haan, Pascalis, & Johnson, 2002; Eimer, 2000; Itier & Taylor, 2002, 2004-a,b; Linkenkaer-Hansen et al., 1998; Rossion et al., 1999; Rossion et al., 2000; Rossion et al., 2003; Sagiv & Bentin, 2001), the VPP (Itier & Taylor, 2002, 2004-a; Jeffreys, 1993; Rossion et al., 1999; Rossion et al., 2003), and the P1 (Itier & Taylor, 2002, 2004-a,b; Linkenkaer-Hansen et al., 1998; Taylor, 2002; Taylor, Edmonds, McCarthy, & Allison, 2001) Face inversion also increases the amplitude of these three components (de Haan et al., 2002; Eimer, 2000; Itier & Taylor, 2002, 2004-a,b; Rossion et al., 1999; Rossion et al., 2000; Sagiv & Bentin, 2001 for the N170; Rossion et al., 1999 for the VPP; Itier & Taylor, 2004-a,b; Linkenkaer-Hansen et al., 1998 for the P1). These data have been tentatively interpreted as reflecting a slower and longer-lasting neuronal response associated with the increased difficulty in facial encoding due to inversion, or as a consequence of inverted faces recruiting more general brain areas involved in object recognition in addition to the typical face-sensitive areas (see Rossion & Gauthier, 2002).

Nevertheless, it is important to highlight the fact that the type of configural change investigated in face-inversion ERP studies does not directly disrupt the facial configuration, but rather changes the overall orientation of the entire face stimulus. Pictures of canonical upright faces were compared with upside-down images created simply by a 180° rotation (or a vertical flipping) of the whole face, including the inner, as well as the outer features (i.e., the external contour, the neck and the hair). Therefore, ERP studies on face inversion provide limited evidence on the extent to which the observed electrophysiological face-sensitive responses at the level of the P1, N170 and VPP are tuned to the specific visual geometry of the human face, that is to the specific internal organization of the features within a face.

To our knowledge, only three studies have directly investigated the effects produced by disruption of the spatial integrity of inner facial features on the early face-sensitive responses typically observed at the level of the N170 and VPP, and none has tested the effects of such manipulations on the earlier P1. These few studies compared canonical faces to distorted, scrambled faces created by displacing and rearranging the internal facial features while keeping the outline in its canonical orientation (George et al., 1996; Eimer & McCarthy, 1999; Yamamoto & Kashikura, 1999). Although not always concordant, findings showed that scrambled faces evoked a later (Eimer & McCarthy, 1999; George et al., 1996) and larger (George et al., 1996) N170, as well as a later VPP (George et al., 1996; Yamamoto & Kashikura, 1999). However, in addition to the paucity of direct evidence that these studies provide on the effects produced by featural displacement, they did not systematically investigate the effects produced by the selective disruption of specific aspects of the face geometry. More precisely, in these studies the rearrangement of the internal features in the scrambled versions of the face was done without reference to any specific criterion. For example, in the study by George et al. (1996) scrambling was obtained by reversing the position of the eyes and nose, retaining vertical symmetry. However, no comment was made as to which specific aspects of the face geometry this rearrangement disrupts, or how the effects of this rearrangement would differ from those produced by other types of featural scrambling. These limits render unclear the interpretation of the observed effects.

A more strict control of these variables is present in a recent series of studies reported in the developmental literature, which were aimed at investigating the specificity of the well-known newborns' face-preference phenomenon. Such phenomenon consists in the observation that both realistic (Macchi Cassia, Turati, & Simion, 2004) and highly schematized face-like configurations (Goren, Sarty, & Wu, 1975; Johnson & Morton, 1991; Valenza, Simion, Macchi Cassia, & Umiltà, 1996) spontaneously capture newborns' visual attention more than other, equally complex, visual objects. These observations have been classically taken as a demonstration that faces already represent a special class of stimuli at birth because humans are born with a specific, innate, built-in mechanism selectively tuned to the face geometry (Johnson & Morton, 1991). Yet more recently, newborns' face preference has been related to a number of nonspecific attentional biases toward a set of general structural properties embedded in faces, one of which is for visual configurations presenting more patterning in their upper as compared to their lower part (i.e., top-heavy patterns; Simion, Macchi Cassia, Turati, & Valenza, 2001, 2004). Recent research suggests that this structural property (i.e., up-down asymmetry) that faces share with nonface stimuli, rather than “facedness” per se, plays the crucial role in attracting newborns' attention toward schematic (Turati, Simion, Milani, & Umiltà, 2002) as well as veridical face images (Macchi Cassia et al., 2004).

The rationale upon which this conclusion was drawn was that of altering the spatial integrity of inner facial features so as to selectively break one of the structural rules embedded in faces, and testing the effect of such manipulation on newborns' face-specific attentional response. We believe that an identical, systematic approach of decomposing the structral properties of face stimuli would also be valuable in order to better understand what factors are driving the amplitude and latency modulations found at the level of early face-sensitive ERP components when dirupting the facial organization of features. I

It is important to stress that the visual geometry of the human face is the emerging product of the co-occurrence of a number of general visual structural properties, only one of which is the up-down arrangement of the inner features. In fact, faces can be described as top-heavy, congruent, vertically symmetrical stimuli delimited by a curvilinear contour. In addition to having a larger proportion of high-contrast features in the upper (eyes, eyebrows) compared to the lower part (mouth) (i.e., up-down featural arrangement), faces also have rounded edges and a congruent relation between the spatial distribution of the inner features and the shape of the outer frame, with the greater number of features located in the widest, upper portion of the oriented, triangular-shaped contour (i.e., congruency; see Macchi Cassia, Valenza, Pividori, & Simion, 2002). Moreover, faces are bilaterally symmetrical with reference to their vertical axis, in that the left and right halves of the face contain the same number and type of elements, which are normally equidistant from the axis (i.e., vertical symmetry). Very little is known about the contribution provided by each of these visual structural properties to the observed neurocognitive face-sensitive responses in adults.

The goal of our study was to examine this issue by testing the effects produced by the disruption of two of the structural properties embedded in faces on three early face-sensitive ERP components: P1, N170 and VPP. Specifically, in the current study the inner facial features were displaced and rearranged in such a way as to selectively break two of the visual structural rules that faces encapsulate, namely up-down featural arrangement and vertical symmetry. We compared symmetrical top-heavy canonical faces (ST) to three different types of distorted, scrambled faces, each created by selectively breaking one or both of the two selected structural rules. The selective violation of the first rule (up-down featural arrangement) led to the creation of symmetrical bottom-heavy scrambled faces (SB), and the selective violation of the second rule (symmetry in the vertical axis) led to the creation of asymmetrical top-heavy scrambled faces (AT). Finally, the contemporary violation of the two rules led to the creation of asymmetrical bottom-heavy scrambled faces (AB) (see Figure 1). Note that, because we wanted to treat stimulus inversion as one possible type of featural displacement, we kept the outline of the face in its canonical upright orientation for all three of the distorted faces, and manipulated exclusively the arrangement of the internal features. Thus, the symmetrical bottom-heavy scrambled faces that we used differ from the inverted faces employed in studies on face inversion in that, in the current study, inversion relates exclusively to the inner features, and not to the outline of the face.

Figure 1.

Figure 1

Open in a new tab

Examples of stimuli from the four categories used in the study. ST. Symmetrical Top-heavy canonical face. SB. Symmetrical Bottom-heavy scrambled face. AT. Asymmetrical Top-heavy face. AB. Asymmetrical Bottom-heavy face.

Through the comparison of the ERP responses elicited by canonical faces and the three types of distorted faces, we intended to shed light on the role played by two of the general structural properties embedded in faces in the mature adult face recognition system. If both of these properties play a role in tuning the neurocognitive operations involved in face detection, their disruption should have an additive effect on the latency and/or amplitude of the analyzed ERP components. That is, there should be a gradient of sensitivity at the P1, N170 and/or VPP to the four stimuli we used, with ST faces being at one end of a continuum, AB scrambled faces being at the other end, and SB and AT scrambled faces lying along the continuum between the two extremes. Based on the limited available evidence reported in the literature (George et al., 1996; Eimer & McCarthy, 1999; Yamamoto & Kashikura, 1999), we expected to find progressive latency and amplitude increases of the examined early face-sensitive components, varying as a result of an increase in the perceptual distance from the canonical face geometry. Moreover, it could also be hypothesized that violation of the two structural rules manipulated in the present study would differentially modulate the P1, N170 and VPP, each component being more sensitive to one property or the other, or the co-occurrence of the two.

An alternative view is that electrophysiological face-sensitive responses are driven by the unique face geometry, rather than by the visual general structural properties that faces encapsulate. According to this view, the prediction would be that any of the three scrambled stimuli not presenting the typical geometry of the face are differentiated in the same way when compared to canonical faces, thus modulating latency and/or amplitude of the P1, N170 and/or VPP in the same way. If this were the case, we would expect to find canonical faces evoking P1, N170 and/or VPP responses of shorter latency and/or smaller amplitude than all three of the distorted faces, which in turn should not be differentiated from each other.

Results

Behavioral (accuracy rates, correct response times, and inverse efficiency scores) and electrophysiological (peak latencies and amplitudes of components) measures were analyzed by means of 2×2 repeated-measures analyses of variance (ANOVA) using Greenhouse-Geisser adjusted degrees of freedom. Within-subjects factors were vertical symmetry (two levels) and up-down featural arrangement (two levels). For electrophysiological analyses, hemisphere (two levels) was added to within-subjects factors for the P1 and N170, and electrode (three levels) was added for the VPP. Post-hoc paired t-tests were also performed when necessary using Bonferroni corrections for multiple comparisons.

Behavioral data

The mean overall accuracy in the detection task ranged from 90% (AT scrambled faces) to 97% (ST canonical faces); mean response times ranged between 546 ms (ST canonical faces) and 639 ms (AT scrambled faces). The 2×2 ANOVA was performed on the accuracy rates, correct response times (RTs) and inverse efficiency scores (Akhtar & Enns 1989; Goffaux, Hault, Michel, Vuong, & Rossion, 2005; Townsend & Ashby, 1983). The inverse efficiency scores (expressed in milliseconds) were computed separately for each condition and each subject by dividing the mean response times for each condition by the proportion of correct responses for that same condition, so that lower values on this measure indicate better performance on the orientation judgment task. This measure was used to compensate for possible criterion shifts across conditions or speed-accuracy tradeoffs in performance.

The ANOVAs performed on the 18 subjects who also provided ERP data revealed a significant main effect of vertical symmetry for both accuracy rates (F1,17 = 5.06; p < 0.05) and RTs (F1,17 = 60.74; p < 0.001), due to lower accuracy rates and slower RTs for asymmetrical as compared to vertically symmetrical faces. For RTs, there was also a main effect of up-down featural arrangement (F1,17 = 9.82; p < 0.01), with orientation judgments being slower for bottom-heavy as compared to top-heavy faces. Interestingly, the interaction between vertical symmetry and up-down featural arrangement was significant for RTs (F1,17 = 30.57; p < 0.001) and marginal for accuracy rates (F1,17 = 3.79; p = 0.07). This interaction qualifies the main effects reported above. Post-hoc comparisons showed that the ST canonical faces induced the fastest and most accurate response in the orientation judgment task, in that RTs were faster and accuracy rates were higher to the ST faces as compared to both the SB (RTs: p < 0.001; accuracy rates: p < 0.05) and the AT scrambled faces (RTs: p < 0.001; accuracy rates: p < 0.05). The post-hoc also interestingly revealed a marginal trend for slower RTs (p = 0.12) to the AT in comparison to the AB scrambled faces and a similar, although not significant trend for accuracy rates (p = 0.39) (see Figure 2).

Figure 2.

Figure 2

Open in a new tab

Mean efficiency scores for the four stimuli (n = 29). ST canonical faces gave rise to the most efficient performance and AT scrambled faces gave rise to the least efficient performance in the orientation judgment task. Error bars represent standard errors of the mean.

A similar pattern of results emerged from the ANOVA performed on the inverse efficiency scores, which revealed a main effect of vertical symmetry (F1,17 = 20.38; p < 0.001) and up-down featural arrangement (F1,17 = 8.59; p < 0.01), as well as a significant interaction between these two factors (F1,17 = 15.91; p = 0.001). A posteriori tests showed that performance on the orientation judgment was best (i.e., lowest inverse efficiency score) for the ST canonical faces (ST vs SB and ST vs AT; p < 0.001) and showed a marginal trend toward being worst (i.e., highest inverse efficiency score) for the AT scrambled faces (AT vs ST; p < 0.001 and AT vs AB; p = .14) (see Figure 2).

In order to verify if this trend holds true with a larger sample size, a fourth ANOVA was performed on the inverse efficiency scores provided by an additional 11 subjects, for a total sample size of 29 subjects. This ANOVA revealed the same significant effects as the previous one, showing that subjects' performance was worse for the AT scrambled faces as compared to both the AB scrambled faces (p = 0.032) and the ST canonical faces (p < 0.001). Also, as in the previous ANOVA, the ST canonical faces gave rise to the most efficient performance, in that the inverse efficiency score for this class of stimuli was also lower than the inverse efficiency score for the SB scrambled faces (p < .001).

Electrophysiological Data

P1

No effect of hemisphere, vertical symmetry or up-down featural arrangement was found for P1 latency (see Figure 3), which was maximal around 135 ms.

Figure 3.

Figure 3

Open in a new tab

A. Grand-averaged ERPs, showing the P1 component at the left (O1) and right (O2) occipital electrodes for ST canonical faces and SB, AT, AB scrambled faces. B. Bar graph displays the overall mean amplitudes of the P1 component displayed for the four stimulus categories at O1 and O2. Note the linear increase in the voltage amplitude of the component for SB, AT and AB scrambled faces as compared to ST canonical faces.

For P1 amplitude there was a main effect of vertical symmetry (F1,13 = 12.66; p < 0.005)1, qualified by a significant interaction between this factor and the factor hemisphere (F1,13 = 5.06; p < 0.05). Asymmetrical faces elicited a larger P1 than symmetrical faces at both left (p = 0.016) and right (p < 0.005) electrode sites. There was also a main effect of up-down featural arrangement (F1,13 = 10.32; p < 0.05), due to a larger amplitude of the P1 to bottom-heavy as compared to top-heavy faces (see Figure 3). As previous studies on face inversion have found larger P1 amplitudes for inverted (i.e., bottom-heavy) than upright (i.e., top-heavy) faces (Itier & Taylor, 2004-a,b; Linkenkaer-Hansen et al., 1998), we performed a pair-wise comparison with only symmetrical faces to verify if the effect we found holds true when only the ST and SB faces are compared. The comparison failed to show a significant difference between these two stimuli, although there was a marginal trend (t13 = 2.07; p = 0.12). We also performed a test of within-subjects contrasts that showed a significant linear increase in P1 amplitudes for the four stimuli, with ST canonical faces at one end of the linear trend, AB scrambled faces at the other end, and SB and AT scrambled faces in between (F1 = 16.07; p < 0.001) (see Figure 3).

N170

N170 latency showed a main effect of vertical symmetry (F1,17 = 48.98; p < 0.001), with symmetrical faces peaking sooner than asymmetrical faces, and a main effect of up-down featural arrangement (F1,17 = 7.80; p = 0.01), due to component for top-heavy faces peaking sooner than that for bottom-heavy faces (see Figure 4). Again, we ran a pair-wise comparison on the N170 latency including only top-heavy and bottom-heavy symmetrical faces. In line with numerous findings reported in the literature on face inversion (e.g., Bentin et al., 1996; Rossion et al., 2003), we found that SB scrambled faces elicited longer N170s than ST canonical faces (t17 = 2.51; p < 0.05). A test of within-subjects contrasts revealed a significant linear change in N170 latencies, with ST canonical faces and AB scrambled faces at the two extremes of the continuum, and SB and AT scrambled faces lying in between (F1 = 58.94; p < 0.001), as shown in Figure 4.

Figure 4.

Figure 4

Open in a new tab

A. The N170 obtained in response to ST canonical faces, SB, AT and AB scrambled faces at mastoid and temporal electrodes on left (A1 and T5) and right (A2 and T6) hemispheres. B. The overall mean latencies of the N170 for the four stimulus categories at the four electrodes. Note the linear increase in the latency of the component for SB, AT and AB scrambled faces as compared to ST canonical faces. C. The mean amplitudes of the N170 at the mastoid and temporal electrodes (A2/T6) on the right hemisphere. Note the larger N170 component for AT scrambled faces compared to the other three stimuli.

For N170 amplitude, both main effects of vertical symmetry (F1,17 = 8.03; p = 0.01) and up-down featural arrangement (F1,17 = 5.06; p < 0.05) were found, with asymmetrical faces showing larger amplitude than symmetrical faces, and top-heavy faces being larger than bottom-heavy faces. These effects were qualified by a significant three-way interaction between the two factors and the factor hemisphere (F1,17 = 6.63; p < 0.05). Post-hoc comparisons showed that N170 amplitudes were larger for AT scrambled faces as compared to AB scrambled faces (p < 0.005) and ST canonical faces (p < 0.05) over the right hemisphere, as shown in Figure 4. A main effect of vertical symmetry over the left hemisphere (F1,17 = 9.44; p < 0.01) was due to asymmetrical faces producing larger amplitudes than symmetrical faces.

VPP

A main effect of electrode (F1,17 = 5.47; p < 0.05) reflected that VPP latencies peaked faster at the midline electrode (Cz). Moreover, as with the N170, there were main effects of vertical symmetry (F1,17 = 52.64; p < 0.001) due to earlier peak latency for symmetrical faces, and of up-down featural arrangement (F1,17 = 6.52; p < 0.05) due to delayed latencies for bottom-heavy faces. A marginal difference between top-heavy and bottom-heavy faces was found when only symmetrical faces were considered (ST vs SB; t17 = 1.90; p = 0.15). Again, we performed a test of within-subjects contrasts that showed a linear latency increase of the VPP corresponding to an increase in perceptual distance from the ST canonical face, with SB scrambled faces preceeding AT scrambled faces along the continuum (F1 = 52.21; p < 0.001; Figure 5).

Figure 5.

Figure 5

Open in a new tab

A. Grand-average waveforms showing the VPP component at the left (C3), medial (Cz) and right (C4) central leads for the four stimulus categories. B. The overall mean latencies of the VPP for the four stimulus categories at the three electrodes. Note the linear increase in the latency of the component for SB, AT and AB scrambled faces as compared to ST canonical faces. C. The overall mean amplitudes of the VPP for the four stimulus categories at the three electrodes. The VPP component is larger for AT scrambled faces compared to the other three stimuli.

For VPP amplitude, there was a main effect of electrode (F1,17 = 21.75; p < 0.001) due to the peaks elicited by the four stimuli being largest at the midline electrode. There was also a marginal interaction between the factors vertical symmetry and up-down featural arrangement (F1,17 = 4.13; p = 0.058), similar to that found in the N170's amplitude. This was due to a marginal trend for a larger VPP in response to AT scrambled faces compared to ST canonical faces (p = 0.06), and a significantly larger VPP to AT faces compared to AB scrambled faces (p < 0.05) (Figure 5).

Discussion

The main purpose of this study was to investigate whether the general visual structural properties embedded in faces play a role in tuning the electrophysiological brain responses to this stimulus category. This was done by testing the effects produced by the selective disruption of two specific aspects of face geometry, namely vertical symmetry and up-down featural arrangement, on the response properties of three early ERP components that are reported to be face-sensitive: the P1, N170 and VPP.

Behavioral results showed that the ST canonical faces were the easiest stimuli to be detected and judged as upright, and that the AT scrambled faces were the most difficult. The finding that SB faces were detected less efficiently than the ST canonical faces is a replication, but also an extension of the common finding that orientation judgments are made more quickly for upright as compared to inverted faces (e.g., Rossion et al., 2000; Rossion et al., 2003). Our data extend these findings by showing that an inversion of solely the internal features of the face -as in the SB scrambled faces-is sufficient to produce this same effect. Most interestingly for the purpose of the present study, the current data show that not only a disruption of the up-down arrangement of the inner features, but also the violation of another visual structural property of faces, vertical symmetry, can have a similarly impairing effect on subjects' performance. The impairing effect was particularly strong for AT scrambled faces, which led to the worst performance. This finding may suggest that subjects tended to encode the “facedness” of the stimuli with reference to the up-down disposition of the features (i.e., the eyes at the top, above the nose and the mouth), but at the same time they could not easily treat the ST scrambled stimuli as faces because of the asymmetrical distribution of the features along the vertical axis. Interestingly, the impairing effect produced by displacing the inner features did not appear to be magnified when both structural properties of the face were disrupted, thus lending no data to support the idea that disrupting both properties would have an additive impairing effect on orientation judgment responses. Instead, these behavioral findings suggest that the alteration of even one property of the face geometry, regardless of which property, is enough to render the judgment more difficult, and that the modification of one property in particular, symmetry on the vertical axis, can even further enhance the difficulty of the judgment.

Whereas behavioral data suggest that the face geometry seems to play an “all or nothing” role, electrophysiological evidence suggests otherwise. Overall, the major finding of the present study is that the two investigated structural properties inherent to faces affected the three analyzed ERP components, and that for all components the size of the effects produced by disruption of the two properties lie along a linear continuum. Within this continuum, canonical faces and scrambled faces with both properties disrupted lie at the two extremes. The violation of vertical symmetry has a more detrimental effect than the violation of up-down featural arrangement rendering the SB scrambled faces before the AT scrambled faces at the two inner points on the continuum.

The P1 has been reported as having increased amplitude to inverted as compared to upright faces (Itier & Taylor, 2004-a,b). The current data expand on these findings, showing increased amplitude to bottom-heavy scrambled faces presenting more inner elements in the lower part, regardless of the vertically symmetrical arrangement of such elements. We also found that the disruption of vertical symmetry modulated the amplitude of the P1, which was larger to asymmetrical as compared to symmetrical faces, regardless of the up-down distribution of their inner elements. As a whole, these results provide the first demonstration of the effects produced by rearrangement of the internal facial features on the P1. Most crucially, the absence of significant interaction between the two factors of vertical symmetry and up-down featural arrangement suggests that each investigated structural property plays its own individual role in tuning the P1. Yet, the presence of a significant linear increase in P1 amplitude, as shown by the test of within-subjects contrasts, points to a gradient of sensitivity of this early component to the two investigated properties. Mean P1 amplitudes to the four stimuli were revealed to lie along a linear continuum with ST canonical faces and AB scrambled faces at the two opposite extremes, and SB and AT scrambled faces lying between the two extremes. This suggests that disruption of the vertically symmetrical arrangement of the inner facial features had a stronger impact on P1 amplitude than disruption of the up-down arrangement of the features, and that the simultaneous disruption of the two properties has an additive effect on the response properties of the P1 component.

Overall, a similar pattern of results was found for the latency of the N170. The violation of each of the two investigated structural properties independently affected N170 latency, which peaked later to asymmetrical than to symmetrical faces, and was also later to bottom-heavy as compared to top-heavy faces. Importantly, the N170 showed a significant latency delay for SB stimuli as compared to ST canonical faces. This latter finding replicates and extends previous observations of a latency delay for inverted faces (e.g., Bentin et al., 1996; Rossion et al., 2003), by showing that this same effect is present when only the internal portion of the face is inverted, and the face outline is kept in its canonical upright orientation. Additionally, as we did for P1 amplitude, we also found a significant linear increase in N170 latency, varying as a result of an increase in the perceptual distance from the canonical face geometry. Similarly to P1 amplitude then, N170 latency seems to show a gradient in sensitivity to the violation of the two investigated visual structural rules embedded in faces. In particular, disruption of the vertically symmetrical arrangement of the inner facial features produces a larger latency shift than disruption of the up-down arrangement of the features, suggesting the the first property plays a more prominent role than the latter in modulating the response properties of the face-sensitive N170.

This argument is further supported by the finding that AT scrambled faces, differing from canonical faces exclusively for the asymmetrical disposition of the features, elicited the largest N170 over the right hemisphere. This result is congruent with the above mentioned behavioral observation of a drop in efficiency of the orientation judgment for AT scrambled faces, and may likely reflect the larger effort required to encode these stimuli as compared to the other three. The presence of more elements in the upper rather than in the lower part may have influenced subjects to try to encode these stimuli as canonical faces, but, at the same time, the lack of a vertically symmetrical disposition of the elements rendered the stimuli very different from a face. Interestingly, this specific amplitude increase to AT stimuli was observed only over the right hemisphere, where the face-specific N170 response has been reported to be most prominent (e.g., Bentin et al, 1996; Rossion et al., 2003), whereas a more generalized amplitude increase to both asymmetrical faces was present over the left hemisphere. Finally, it is worth citing that amplitude of the N170 was not modulated by the up-down arrangement of facial features. Therefore, our findings did not replicate previous observations of an amplitude increase of the N170 to inverted faces (e.g., Rossion et al., 2000; Sagiv & Bentin, 2001). However, although reported in the literature, this effect is less consistent than the latency shift produced by stimulus inversion (see Rossion & Gauthier, 2002) which was observed also in the present study.

In comparison to the P1, the N170 findings seemed to reflect a finer discrimination between canonical and distorted faces for two reasons. First, it peaked later to SB scrambled faces than to ST canonical faces, thus showing the classic inversion effect of delayed latency, which was not present at the level of the P1. Second, it was enhanced to the AT scrambled faces for which behavioral orientation judgments were less efficient.

As for the VPP, latency and amplitude effects were identical to those observed in the N170, although for amplitude these effects were marginal. These findings are in accord with previous observations of a high degree of functional similarity between the response properties of the N170 and the central VPP (see Joyce & Rossion, in press; Rossion et al., 2003). Given the argument of the VPP and the N170 being manifestations of the same underlying neural generators (Jeffreys, 1989; Joyce & Rossion, in press; Rossion et al., 2003), the lack of statistical significance of the observed amplitude effects could be explained as a result of the VPP being more distant from the underlying sources than the N170, as is suggested by the observed small shift in peak latency between the two components (e.g., Bentin et al., 1996; George et al., 1996). The demonstration that the VPP can be modulated by alterations of the face configuration is particularly important as, to our knowledge, only two studies currently exist that examined the effects of featural displacement on this component, and these two studies provided conflicting results (George et al., 1996; Yamamoto & Kashikura, 1999).

As a whole, the obtained results showed that the alterations we introduced in the face geometry through disruption of two structural properties intrinsic to faces modulated the response properties of the P1, the N170 and the VPP. Most commonly, these modulations were observed as a linear increase in amplitude or latency from ST canonical faces to AB scrambled faces with SB and AT scrambled faces, respectively, at the inner positions on the continuum. An exception to this linear trend was observed for the amplitude of the N170 and the VPP, for which the AT faces produced the largest response. The fact that this exception occurred at the level of the N170 but not the earlier P1 is congruent with the behavioral finding of lower efficiency in the orientation judgment of the AT scrambled faces. As previously mentioned, the amplitude effect of the N170 could reflect a larger effort required to encode these stimuli as compared to the other three, caused by the peculiar arrangement of the inner features. This increase of resources at the face encoding stage, reflected by the amplitude enhancement of the N170, is suggested by may have produced the observed drop in behavioral efficiency of the orientation judgment. Indeed, it has already been proposed that the origin of the differential behavioral performance for upright and inverted faces may come from the face encoding stage reflected by the N170 (Rossion et al., 1999). Another possibility is that the orientation judgment requiring the subject to classify the AT stimuli using the same upright key as the ST canonical faces may have forced the subjects to try to encode the AT scrambled faces as canonical faces. This forced unnatural categorization could have resulted in the exagerated N170 amplitude as well as in the drop in behavioral efficiency of the orientation judgment. If this were the case, it could be reasoned that if subjects were tested in a passive viewing task with the same stimuli, the amplitude difference for the AT scrambled faces at the N170 would be no larger than that of the other two non-canonical face stimuli.

Although more research is needed to fully understand the cause of the increased electrophysiological response elicited by the AT scrambled faces and its relationship to behavioral responses, these data show that both of the two investigated structural properties inherent to faces play a crucial role in modulating the neurocognitive operations involved in face detection.

Methods

Participants

Forty-one subjects (16 females) ranging in age from 18 to 39 years (mean age = 23.2 years) took part in the study. Eighteen subjects participated in both the ERP and behavioral aspects of the study, and an additional 11 took part in only the behavioral orientation judgment task. They were healthy, right-handed students recruited from an undergraduate population at the University of Minnesota, and were either paid or received course credit for their participation. Subjects had normal or corrected-to-normal vision, and reported taking no medication and having no history of neurological, ophthalmological or systemic disease. Informed written consent was obtained from all participants, in accordance with the Institutional Review Board of the University of Minnesota. Twelve subjects were excluded from the analyses because of eye movements that resulted in too many EOG artifact (n = 5), behavioral performance below 25% accuracy (n = 4), or experimenter error/equipment failure (n = 3). Thus, the final sample consisted of 29 subjects.

Stimuli

Twenty-five high-quality grayscale photographs of young female faces were digitally modified so as to create 4 versions of each face differing exclusively in the spatial positioning of the inner features, the outline contour being equal, for a total of 100 stimuli. The original photographs served as one of the 4 versions of the stimuli, namely the symmetrical top-heavy canonical face (ST). From ST, the other 3 versions of the stimuli were created by disrupting either one or both visual structural properties manipulated in the study, namely up-down featural arrangement (top-heavy vs bottom-heavy) and vertical symmetry (symmetrical vs asymmetrical). These manipulations resulted in the generation of a symmetrical bottom-heavy scrambled face (SB), an asymmetrical top-heavy scrambled face (AT) and an asymmetrical bottom-heavy scrambled face (AB; see Figure 1). The models were photographed in a frontal pose with a neutral expression. The faces were cropped right below the neck, and the hair and ears were removed. Stimuli were presented on a black background centered on a computer screen, and subtended a visual angle of approximately 11×15°.

Apparatus and Task procedure

Following electrode application, subjects were seated on a comfortable chair in a dimly lit room, approximately 75 cm from a computer screen, and were tested in an active discrimination task involving an orientation judgment. Subjects were instructed to visually fixate the center of the screen during the presentation of two consecutive blocks of 100 trials each (25 images×2 vertical symmetry×2 up-down featural arrangement), with about a one minute pause between blocks. Each trial consisted of a 100 ms baseline, a 500 ms stimulus presentation, and a post-stimulus recording of 1100 ms; the inter-stimulus interval was randomized between 1000 and 1500 ms. All of the stimuli were randomized within a block with the constraints that each unique image in the set appear once before any was repeated and that the same up-down featural arrangement (top-heavy or bottom-heavy) was not repeated more than three times in succession. All subjects viewed the same succession of stimuli. The subject's task was to provide an orientation judgment by pressing a key if the stimulus was “upright” and another key if the stimulus was “inverted”, basing their discriminative response on the up-down arrangement of the inner facial features. The side of the response was balanced across subjects. Subjects were instructed to respond as accurately and as quickly as possible; accuracy rate and response times for correct responses were recorded.

Electrophysiological recording and data reduction

ERPs were recorded from 31 scalp electrodes mounted in a close-fitting cap (Electro-Cap International) using a modified 10-20 system. The electrodes comprised Fz, Pz, T3, T4, T5, T6, C3, C4, F3, F4, F7, F8, O1, O2, FC1, FC2, FC5, FC6, CP1, CP2, CP5, CP6, P3, P4, PO3, PO4, PO7, PO8, plus the left and right mastoids (A1, A2) and a ground electrode. Cz was the reference lead during acquisition. An average reference was used that was calculated off-line prior to data reduction and analysis. Electrooculogram (EOG) was recorded from bipolar miniature electrodes placed vertically above and below the right eye for the purpose of artifact detection. Impedance for all scalp and EOG electrodes was kept below 5kΩ. EEG and EOG were acquired using a Grass Neurodata Acquisition System and amplified using Model 12A5 amplifiers with a gain of 50,000 for scalp leads and 5,000 for EOG. The bandpass was 0.1-30 Hz, and a 60-Hz notch filter was engaged. Data were sampled every 5 ms (200 Hz). ERP data were digitized on-line and then edited by computer algorithm. Prior to averaging, trials with excessive artifact (i.e., EEG > ±100 mV) were rejected. Data were then re-referenced to an average reference, and eye movement-related artifact was corrected (Gratton, Coles, & Donchin, 1983). Using 100 msec prior to stimulus onset as the baseline, individual trials were baseline corrected and then averaged for each participant within each stimulus type. Each subject contributed an average of 41 trials per lead in each condition.

After visual inspection of the grand average in all conditions, peak latency and amplitude values of the P1, N170 and VPP were automatically extracted. For the P1, these values were extracted at the maximum (positive) amplitude point between 75 and 150 ms at a single occipital electrode site in the left and right hemisphere (O1 and O2; see Figure 3). For the N170, latency and amplitude values were measured at the most negative point between 120 and 270 ms at the mastoid and temporal sites in the two hemispheres (A1/T5 and A2/T6; see Figure 4). The VPP measurements were made on the maximum positivity occurring within the same time window at the left, medial and right central leads (C3, Cz, C4; see Figure 5).

Acknowledgments

This work was supported in part by a National Institutes of Health R01 grant to Charles A. Nelson (NS32976). We also wish to thank Kim Pearson, Jeff Benson and Art Gorr for programming, Robert Shannon and Michelle Stein for testing subjects, and Jim Williams for technical assistance.

Footnotes

1

The ANOVAs on the latency and amplitude of the P1 were performed on 14 of the 18 subjects. Four subjects were removed from these analyses because of bad electrodes at O1 and O2.

References

  1. Akhtar N, Enns JT. Relations between covert orienting and filtering in the development of visual attention. Journal of Experimental Child Psychology. 1989;48:315–334. doi: 10.1016/0022-0965(89)90008-8. [DOI] [PubMed] [Google Scholar]
  2. Bentin S, Allison T, Puce A, Perez A, McCarthy G. Electrophysiological studies of face perception in humans. Journal of Cognitive Neuroscience. 1996;8:551–565. doi: 10.1162/jocn.1996.8.6.551. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bentin S, Deouell LY. Structural encoding and identification in face processing: ERP evidence for separate mechanisms. Cognitive Neuropsychology. 2000;17:35–54. doi: 10.1080/026432900380472. [DOI] [PubMed] [Google Scholar]
  4. Botzel K, Schulze S, Stodieck SR. Scalp topography and analysis of intracranial sources of face-evoked potentials. Experimental Brain Research. 1995;104:135–143. doi: 10.1007/BF00229863. [DOI] [PubMed] [Google Scholar]
  5. Bruce V, Young A. Understanding face recognition. British Journal of Psychology. 1986;77:305–328. doi: 10.1111/j.2044-8295.1986.tb02199.x. [DOI] [PubMed] [Google Scholar]
  6. Caldara R, Seghier M, Rossion B, Lazeyras F, Michel C, Hauert CA. The fusiform face area is tuned for non-face head-shaped patterns with a greater number of elements in the upper visual field. Poster presented at the 10th annual Cognitive Neuroscience Society Conference; New York, New York. 2003. Mar, [Google Scholar]
  7. Cauquil AS, Edmonds G, Taylor MJ. Is the face-sensitive N170 the only ERP not affected by selective attention? NeuroReport. 2000;11:2167–2171. doi: 10.1097/00001756-200007140-00021. [DOI] [PubMed] [Google Scholar]
  8. de Haan M, Pascalis O, Johnson MH. Specialization of neural mechanisms underlying face recognition in human infants. Journal of Cognitive Neuroscience. 2002;2:199–209. doi: 10.1162/089892902317236849. [DOI] [PubMed] [Google Scholar]
  9. Eimer M. Effects of face inversion on the structural encoding and recognition of faces. Evidence from event-related brain potentials. Cognitive Brain Research. 2000;10:145–158. doi: 10.1016/s0926-6410(00)00038-0. [DOI] [PubMed] [Google Scholar]
  10. Eimer M, McCarthy RA. Prosopagnosia and structural encoding of faces: evidence from event-related potentials. Neuroreport. 1999;10:255–259. doi: 10.1097/00001756-199902050-00010. [DOI] [PubMed] [Google Scholar]
  11. Farah MJ, Rabinowitz C, Quinn GE, Liu G. Early commitment of neural substrates for face recognition. Cognitive Neuropsychology. 2000;17:117–123. doi: 10.1080/026432900380526. [DOI] [PubMed] [Google Scholar]
  12. George N, Evans J, Fiori N, Daviddoff J, Renault B. Brain events related to normal and moderately scrambled faces. Cognitive Brain Research. 1996;4:65–76. doi: 10.1016/0926-6410(95)00045-3. [DOI] [PubMed] [Google Scholar]
  13. Goffaux V, Hault B, Michel C, Vuong QC, Rossion B. The respective role of low and high spatial frequencies in supporting configural and featural processing of faces. Perception. 2005;34:77–86. doi: 10.1068/p5370. [DOI] [PubMed] [Google Scholar]
  14. Goren CC, Sarty M, Wu PY. Visual following and pattern discrimination of face-like stimuli by newborn infants. Pediatrics. 1975;56:544–549. [PubMed] [Google Scholar]
  15. Gratton G, Coles MGH, Donchin E. A new method of off-line removal of ocular artefact. Electroencephalography & Clinical Neurophysiology. 1983;55:468–484. doi: 10.1016/0013-4694(83)90135-9. [DOI] [PubMed] [Google Scholar]
  16. Itier RJ, Taylor MJ. Inversion and contrast polarity reversal affect both encoding and recognition processes of unfamiliar faces: a repetition study using ERPs. Neuroimage. 2002;15:353–372. doi: 10.1006/nimg.2001.0982. [DOI] [PubMed] [Google Scholar]
  17. Itier RJ, Taylor MJ. N170 or N1? Spatiotemporal differences between object and face processing using ERPs. Cerebral Cortex. 2004-a;14:132–142. doi: 10.1093/cercor/bhg111. [DOI] [PubMed] [Google Scholar]
  18. Itier RJ, Taylor MJ. Face recognition and configural processing: a developmental ERP study using upright, inverted and contrast-reversed faces. Journal of Cognitive Neuroscience. 2004-b;16:1–15. doi: 10.1162/089892904322926818. [DOI] [PubMed] [Google Scholar]
  19. Jeffreys DA. A face-responsive potential recorded from the human scalp. Experimental Brain Research. 1989;78:193–202. doi: 10.1007/BF00230699. [DOI] [PubMed] [Google Scholar]
  20. Jeffreys DA. The influence of stimulus orientation on the vertex positive scalp potential evoked by faces. Experimental Brain Research. 1993;96:163–172. doi: 10.1007/BF00230449. [DOI] [PubMed] [Google Scholar]
  21. Jeffreys DA. Evoked potential studies of face and object processing. Visual Cognition. 1996;3:1–38. [Google Scholar]
  22. Johnson MH, Morton J. Biology and cognitive development The case of face recognition. Oxford, England: Basil Blackwell; 1991. [Google Scholar]
  23. Joyce C, Rossion B. The face-sensitive N170 and VPP components manifest the same brain processes: The effect of reference electrode site. Clinical Neurophysiology. doi: 10.1016/j.clinph.2005.07.005. in press. [DOI] [PubMed] [Google Scholar]
  24. Linkenkaer-Hansen K, Palva JM, Sams M, Hietanen JK, Aronen HJ, Ilmoniemi RJ. Face-selective processing in human extrastriate cortex around 120 ms after stimulus onset revealed by magneto- and electroencephalography. Neuroscience Letters. 1998;253:147–50. doi: 10.1016/s0304-3940(98)00586-2. [DOI] [PubMed] [Google Scholar]
  25. Macchi Cassia V, Turati C, Simion F. Can a nonspecific bias toward top-heavy patterns explain newborns' face preference? Psychological Science. 2004;15:379–383. doi: 10.1111/j.0956-7976.2004.00688.x. [DOI] [PubMed] [Google Scholar]
  26. Macchi Cassia V, Valenza E, Pividori D, Simion F. Facedness versus non-specific structural properties: is crucial in determining face preference at birth. Poster presented at the International Conference on Infant Studies; Toronto, Canada. 2002. Apr, [Google Scholar]
  27. Rossion B, Delvenne JF, Debatisse D, Goffaux V, Bruyer R, Crommelinck M, Guèrit Spatio-temporal localization of the face inversion effect: an event-related potentials study. Biological Psychology. 1999;50:173–189. doi: 10.1016/s0301-0511(99)00013-7. [DOI] [PubMed] [Google Scholar]
  28. Rossion B, Gauthier I. How does the brain process upright and inverted faces? Behavioral and Cognitive Neuroscience Reviews. 2002;1:62–74. doi: 10.1177/1534582302001001004. [DOI] [PubMed] [Google Scholar]
  29. Rossion B, Gauthier I, Tarr MJ, Despland PA, Bruyer R, Linotte S, Crommelink M. The N170 occipito-temporal component is enhanced and delayed to inverted faces but not to inverted objects: an electrophysiological account of face-specific processes in the human brain. Neuroreport. 2000;11:69–74. doi: 10.1097/00001756-200001170-00014. [DOI] [PubMed] [Google Scholar]
  30. Rossion B, Joyce CA, Cottrell GW, Tarr MJ. Early laterlization and orientation tuning for face, word, and object processing in the visual cortex. Neuroimage. 2003;20:1609–1624. doi: 10.1016/j.neuroimage.2003.07.010. [DOI] [PubMed] [Google Scholar]
  31. Sagiv N, Bentin S. Structural encoding of human and schematic faces: Holistic and part-based processes. Journal of Cognitive Neuroscience. 2001;13:937–951. doi: 10.1162/089892901753165854. [DOI] [PubMed] [Google Scholar]
  32. Simion F, Macchi Cassia V, Turati C, Valenza E. The origins of face perception: Specific versus non-specific mechanisms. Infant and Child Development. 2001;10:59–65. [Google Scholar]
  33. Simion F, Macchi Cassia V, Turati C, Valenza E. Non-specific perceptual biases at the origins of face processing. New York: Nova Science Publishers; 2004. [Google Scholar]
  34. Taylor MJ. Non-spatial attentional effects on P1: critical factors. Clinical Neurophysiology. 2002;113:1903–1908. doi: 10.1016/s1388-2457(02)00309-7. [DOI] [PubMed] [Google Scholar]
  35. Taylor MJ, Edmonds GE, McCarthy G, Allison T. Eyes first! Eye processing develops before face processing in children. Neuroreport. 2001;12:1671–1676. doi: 10.1097/00001756-200106130-00031. [DOI] [PubMed] [Google Scholar]
  36. Townsend JT, Ashby FG. The Stochastic Modelling of Elementary Psychological Processes. Cambridge: Cambridge University Press; 1983. [Google Scholar]
  37. Turati C, Simion F, Milani I, Umiltà C. Newborns preference for faces: What is crucial? Developmental Psychology. 2002;6:875–882. [PubMed] [Google Scholar]
  38. Valenza E, Simion F, Macchi Cassia V, Umiltà C. Face preference at birth. Journal of Experimental Psychology: Human Perception and Performance. 1996;22:892–903. doi: 10.1037//0096-1523.22.4.892. [DOI] [PubMed] [Google Scholar]
  39. Yamamoto S, Kashikura K. Speed of face recognition in humans: an event-related potentials study. Neuroreport. 1999;10:3531–3534. doi: 10.1097/00001756-199911260-00013. [DOI] [PubMed] [Google Scholar]

RESOURCES