Do facially disfiguring features influence attention and perception of faces? Evidence from an antisaccade task

Abstract

Facial disfigurements can influence how observers attend to and interact with the person, leading to disease-avoidance behaviour and emotions (disgust, threat, fear for contagion). However, it is unclear whether this behaviour is reflected in the effect of the facial stigma on attention and perceptual encoding of facial information. We addressed this question by measuring, in a mixed antisaccade task, observers’ speed and accuracy of orienting of visual attention towards or away from peripherally presented upright and inverted unfamiliar faces that had either a realistic looking disease-signalling feature (a skin discolouration), a non-disease-signalling control feature, or no added feature. The presence of a disfiguring or control feature did not influence the orienting of attention (in terms of saccadic latency) towards upright faces, suggesting that avoidance responses towards facial stigma do not occur during covert attention. However, disfiguring and control features significantly reduced the effect of face inversion on saccadic latency, thus suggesting an impact on the holistic processing of facial information. The implications of these findings for the encoding and appraisal of facial disfigurements are discussed.

Introduction

During social interaction, attention to a person’s face is important as the dynamic changes in eye gaze and expression provide information about the person’s mood and intentions. Faces also contain other visual cues that may influence an observer’s interaction with other people. These include features related to the person’s identity and gender, and also to their age, physical attractiveness, biological fitness, and health. Physical facial features may interfere with the observer’s attention to communication-relevant facial aspects such as eye gaze and expression. For example, when a face contains marks that signal disease (e.g., a scar, spots, or a birth mark), these can capture attention and hence influence the way in which the person bearing the mark is being perceived (Ishii et al., 2009; Meyer-Marcotty et al., 2010).

Individuals with a facial disfigurement can experience considerable negative responses from others: being stared at, avoidance, prejudice, discrimination, and stigmatisation (McGrouther, 1997; Shaw, 1981). These reports have been corroborated by evidence that facially anomalous features may be associated with contagious disease and elicit emotional responses such as disgust (Shanmugarajah et al., 2012), avoidance, and stigmatisation (Oaten et al., 2009, 2011). Because observers tend to minimise or avoid contact even when they know that the other person’s facial disfigurement (e.g., birth mark) is non-contagious, their avoidance behaviour suggests an implicit predisposition to avoid disease (Oaten et al., 2009, 2011). Avoidance behaviour can even extend towards objects that the person with the disfigurement handled. For example, Ryan et al. (2012) found that observers avoid close physical contact with objects (e.g., oral contact with a cup) that had previously been handled by confederates who had a simulated facial disfigurement (a birth mark) or who simulated symptoms of influenza. Avoidance of these objects was also accompanied by overt facial expressions of disgust.

The prevalence of avoidance responses towards individuals with facial disfigurements suggests that disfigurements affect initial perception and attention towards the face. However, the precise impact of perceived facial stigma on attentional and perceptual mechanisms is unclear. Given the speed with which observers form first impressions from faces (within ~100 ms; Willis & Todorov, 2006) and given that cognition and attentional control can be modulated by emotional stimuli (Lundqvist & Öhman, 2005), it is highly plausible that disfiguring features affect the way in which observers attend to faces, and that this impact may at least partially account for subsequent cognitive and behavioural responses to facial disfigurement.

Evidence that facial disfigurements influence visual attention comes primarily from eye tracking studies of observers’ eye gaze during free exploration of photographs of faces—predominantly with a configural facial disfigurement such as cleft lip and palate (Ishii et al., 2009; Meyer-Marcotty et al., 2010). These studies showed an influence on oculomotor fixation and scan paths, resulting in less frequent and shorter fixations on the eyes and more frequent and longer fixations on the mouth and nose regions. While these studies compared observers’ eye movements to faces with and without disfiguring features, they could not rule out the possibility that attentional capture might have been caused by the presence of an unusual feature, regardless of its nature (i.e., whether it signalled disease or not). We addressed this issue by comparing oculomotor responses with face images digitally manipulated to contain either a realistic looking skin deformity (a “portwine stain”) or a control feature that was partially occluding the face (Boutsen et al., 2018; Figure 1). Faces with the simulated disfiguring feature attracted fewer fixations on the eyes and incurred a higher number of recurrent fixations compared with faces with a control feature. This suggests a differential effect of the disease-signalling nature of the facial feature in that it draws attention away from the eye region which is preferentially inspected in typical faces.

Figure 1.

Example of face stimuli containing either (a) no added feature, (b) a disfiguring feature, or (c) an occluding feature.

While the above studies using free visual exploration provide measures of the focus of attention within the face, they are somewhat limited in their capacity to answer the question to what extent faces with anomalous features bias the initial directing of attention, that is, the initial covert orienting of attention to a peripherally presented face. However, there is strong evidence that faces, and facial expressions in particular, influence covert attention.

First, typical upright faces attract attention preferentially over non-face objects, as demonstrated in tasks using visual search (Hershler & Hochstein, 2005; VanRullen, 2006), spatial cueing (Bindemann et al., 2005), and antisaccades, that is, saccadic eye movements away from the stimulus location (Gilchrist & Proske, 2006; Morand et al., 2010). Attentional biases to peripherally presented face stimuli in these tasks have been demonstrated by faster face detection during visual search, by enhanced attention to stimulus locations previously occupied by a face in a spatial cueing task, and by slower suppression of saccadic eye movements towards a face when instructed to look away from it (i.e., produce an antisaccade). This bias may even be limited to upright faces only (Gilchrist & Proske, 2006; Hershler & Hochstein, 2005), reflecting holistic face processing.

Second, emotional facial expressions depicting fear or anger increase attention and delay disengagement of attention towards other stimuli (Belopolsky et al., 2011; Calvo et al., 2006; Fox et al., 2002; Lundqvist & Öhman, 2005). Indeed, there is substantial evidence that threat-related and fearful stimuli (faces and non-facial objects) modulate attentional processing. For example, fearful and angry facial expressions can facilitate the processing of subsequently presented stimuli at their location, as in the emotional dot-probe task (MacLeod et al., 1986). Likewise, angry faces may be detected more rapidly in visual search (e.g., Fox et al., 2000; Horstman & Bauland, 2006), although this angry superiority effect has not been replicated in other studies (Calvo & Nummenmaa, 2008). The preferential processing and detection of threat-related emotional faces (as well as threat-related non-facial stimuli, for example, spiders and snakes) can be interpreted in the context of the adaptive significance of threat detection for an organism (Dolan, 2002; Öhman & Mineka, 2001). Convergent evidence from neuropsychology (Adolphs et al., 2005), functional neuroimaging (Surguladze et al., 2003; Vuillemier et al., 2001), and electrophysiology (Eimer & Holmes, 2007) supports the notion of dedicated and enhanced neural structures and pathways for emotional processing. The amygdala in particular is implicated in the neural structures mediating detection of fear and threat (Adolphs et al., 2005; Vuillemier, 2005). For example, event-related potentials as early as 100 ms following stimulus presentation are modulated by attention to fearful facial expressions (Holmes et al., 2003).

Interestingly, among threat-related stimuli, those that elicit disgust rather than fear appear to be processed by a neural pathway distinct from that of fear, involving the insula rather than the amygdala (Adolphs et al., 2005; Calder et al., 2001). Disgust stimuli also have a distinct effect on attention, suppressing rather than enhancing it (Krusemark & Li, 2011; Santos et al., 2008). Even though both represent a threat which an observer would eventually avoid interacting with, a scene, object, or person eliciting disgust is likely to trigger an immediate avoidance response, while a fearful or angry scene, object, or person would capture attention initially to extract further information about the nature of the threat (cf. by attending to gaze direction on a fearful or angry face). For example, during a visual search task using task-irrelevant background images depicting disgusting, fearful, or neutral scenes, Krusemark and Li (2011) reported that the amplitude of the early (~100 ms following stimulus onset) posterior occipital event-related potential component for fearful (compared with neutral) scenes was enhanced (reflecting increased attentional processing), while for scenes eliciting disgust it was suppressed.

In light of the above evidence, we ask to what extent attentional engagement to a face is affected by the presence of disfiguring features. We used a mixed antisaccade task in which on every trial the observer was instructed by a centrally presented cue to make a saccadic eye movement either towards (i.e., a prosaccade) or away (i.e., an antisaccade) from a laterally presented face. By randomly interleaving pro- and antisaccade trials, rather than presenting them in separate trial blocks, we attempted to better equate the executive and inhibitory requirements for each saccade type, as correct responses in both trial types require correct goal monitoring through the interpretation of the task cue (Irving et al., 2009). The mixed antisaccade task (Morand et al., 2010) permits, on prosaccade trials, to measure the speed and efficiency of covert attentional engagement with a peripheral stimulus prior to the execution of a saccadic eye movement towards that stimulus. On antisaccade trials, however, covert attentional engagement with the stimulus needs to be interrupted and disengaged to suppress a reflexive saccadic eye movement towards the stimulus and instead execute an endogenously driven eye movement away from the stimulus (typically in the opposite direction horizontally). On prosaccade trials, the saccadic onset latency—the time between the onset of the stimulus and the onset of the saccade—reflects the attentional engagement with the stimulus as well as the time to programme the saccadic landing position, and thus providing a meaningful measure of covert orienting of attention. On antisaccade trials, the saccadic onset latency additionally reflects the time to inhibit a saccade to the stimulus by disengaging covert attention to the stimulus and subsequently redirecting attention and programming a saccadic eye movement away from it; for this reason, the latency of an antisaccade typically is longer than that of a prosaccade (Kristjánsson, 2007).

Using the mixed antisaccade task, we ask whether disfiguring features suppress rather than enhance attention to the face—given the potential association between facial stigma and emotional responses of disgust. Under this assumption, a disfigured face would hold attention for less time but might also facilitate disengagement of attention from the face. Alternatively, if facial disfigurements are interpreted as eliciting threat, we might expect them to be subject to a similar attentional bias as other threat-related stimuli such as angry or fearful faces, and expect them to hold attention and delay the disengagement of attention (cf. Belopolsky et al., 2011) compared with typical faces or faces with a non-threatening (i.e., not disease-signalling) control feature. The finding of a modulatory effect on attention (i.e., suppression or enhancement) by disfiguring facial features would support the notion that early attentional and perceptual processing of facial information can be affected by the detection of disease-related visual features. It should be noted that the mere presence of any anomalous salient feature on a face could enhance attentional engagement, irrespective of its perceptual interpretation—that is, whether disease-signalling or not. If that were the case, then we would expect attentional engagement (on prosaccade trials) to be enhanced, and disengagement (on antisaccade trials) to be delayed for faces with an added disfigured or control feature in equal measure, and likewise observe these effects similarly for both upright and inverted faces.

If facially disfiguring features influence attention, a related question is whether they influence face perception—in particular here the encoding of facial information. Perceptual representations of facial information are thought to be holistic or configural, meaning that the spatial relations between the face parts are encoded, as well as the parts (Maurer et al., 2002).This configuration-based encoding process contrasts with a feature- (or part-) based encoding of visual information, which is predominant in non-face object recognition. Empirical evidence for the holistic encoding of facial information comes from the face inversion effect, in which perceptual judgements and recognition (regarding identity or expression) of faces are impaired when the face is presented upside down (for a review, see Rossion, 2008). The face inversion effect is interpreted as the result of a switch from a holistic to a feature-based encoding strategy, and as such can be seen as an index of holistic face encoding. Here, we hypothesise that salient facial features might promote a feature-based (instead of holistic) encoding of facial information, and we test this by measuring the effect of face inversion on attention to the face. Thus, if the presence of a disfiguring feature on the face interferes with the (holistic) encoding of the face configuration, then its impact on attention to the face might decrease when the face is inverted (due to the reduced holistic processing). This should result in a reduced face inversion effect, expressed as the difference in saccadic onset latency and accuracy between upright and inverted faces. ¹

Materials and methods

Participants

Sixty healthy adults (43 females, aged 18–25 years, from various ethnic backgrounds) took part in the study. Of these, 57 participants were included in the data analysis (see “Prosaccade trials” section). All participants reported normal or corrected vision and colour vision. All participants gave written informed consent prior to taking part, and the study was approved by the local research ethics committee.

Equipment, stimuli, and design

Equipment

The experiment was run on a PC using E-Prime 2.0 Professional. Stimuli were presented on a 22-inch Iiyama ProLite LCD monitor at a resolution of 1,920 × 1,080 pixels and with a 60 Hz vertical retrace rate. All stimuli were presented in colour on a white background. During each trial, the right-eye position of each observer was monitored at 1 kHz using a desktop-mount EyeLink 1000 Eye Tracker (SR Research Ltd., Canada) with a chin/forehead rest positioned at a viewing distance of 80 cm.

Stimuli

Face stimuli were created from a set of 80 Caucasian faces (40 males and 40 females) photographed with a neutral expression and from a frontal viewpoint. Five exemplars were created of each face: a typical (unchanged) face and four exemplars with a digitally added disfiguring or occluding feature to the left or right cheek area (Figure 1). Furthermore, inverted (upside down) versions of each exemplar were created by flipping it across its horizontal midline; this ensured the added feature remained in the same absolute location. Thus, 10 images were created from each face. Details on the faces and on the construction of the added features can be found in Boutsen et al. (2018). Each image was 600 pixels (10.86°) wide and between 703 and 1,007 pixels (12.53–17.88°) tall; aspect ratio was preserved.

Design

Each participant was presented with 160 trials—80 prosaccade and 80 antisaccade trials. On each trial, one face image was shown in one of five conditions: unchanged (no added feature), left- and right-sided disfiguring feature, and left- and right-sided occluding feature. There were 32 faces per condition, and within each condition, all faces were of different, randomly sampled identities, with an equal number of male and female faces and an equal number of upright and inverted faces. Faces appeared with equal probability in the left or right visual field (centred horizontally at 610 pixels from the edge of the screen) on both prosaccade and antisaccade trials. Each face image was positioned 10.43° from the centre of the screen, with the nearest edge to the screen centre at 6.22°.

Procedure

Each participant was tested individually in a session lasting ~50 min. A detailed briefing explained the course of a typical trial and examples of the faces with an added disfiguring and occluding feature. Following this briefing, the participant was seated in front of the eye tracker, and a 9-point calibration was performed. This was followed by 20 practice trials (which were discarded from the analysis) and then by 160 experimental trials in eight blocks of 20 trials each. A drift correction was applied every 20 trials.

Each trial consisted of the following sequence of events. First, a blue or red dot (saccade cue) appeared on screen instructing the participant of the type of saccade required: a prosaccade or an antisaccade, respectively. The participant then initiated the trial by pressing the space bar on a standard keyboard. At the start of the trial, the saccade cue changed into a fixation cross of the same colour, which remained on screen until the participant had fixated it for 1 s. After this fixation period, a target face was presented immediately (there was no delay between the 1-s fixation cross and the face appearing) either to the left or the right visual field. The face remained on screen until the participant’s gaze shifted in the direction indicated by the saccade cue and remained in that area for 2 s. No restrictions were imposed on the target location of the saccade. When making a prosaccade, the participant was free to look anywhere on the face, and likewise anywhere in the visual field opposite to the face when making antisaccade. The verbal instruction given to the participant regarding an antisaccade trial was “to move your eyes as quickly as possible in the direction opposite to the face’s location”; regarding prosaccade trials, the instruction was “to move your eyes as quickly as possible towards the face.” When the participant’s gaze had dwelled 2 s in the instructed location, the target face disappeared from the screen and was replaced by the saccade cue of the next trial, awaiting initiation of the trial by the participant. Each participant was encouraged to avoid eyeblinks once they had initiated a trial. Self-paced breaks between trials and blocks were encouraged, and drift correction errors during the experiment were monitored by the experimenter.

Post-experiment questionnaire

After the experiment had finished, the participant was asked to describe in writing what the added features (skin discolouration, occluding feature) “looked liked.” All participants described the face with the disfiguring feature as if it had a “disfigurement,” “scar,” “skin condition,” or “burn,” while the face with the occluding feature was described as being an added feature or spot to the image, or a “zoomed-in patch” (i.e., as if revealing an enlarged patch of skin on the face). When questioned about the integrality of the feature within the face, all of the participants stated that the disfiguring feature was seen as part of (i.e., integral with) the face, while the occluding feature was seen as separate from the face.

Data selection

Data from 57 participants were analysed; the remaining data from 3 participants who had <50% valid first eye movements were excluded. The accuracy and the latency of the first saccade following the onset of the face were analysed as a function of face type and orientation. Only first saccades that contained no blink, that were initiated from the fixation region of interest, that had a latency of between 80 and 700 ms, and that had an amplitude of at least 2° were included in the analysis. Together these criteria led us to retain 7,048 of 8,624 (81.17%) eligible first saccades for the analysis. Of these, there were 3,615 prosaccades and 3,433 antisaccades.

Analysis

We used linear mixed models (LMMs; Baayen et al., 2008; Meteyard & Davies, 2020) to evaluate, on prosaccade and antisaccade trials separately, the effect of face condition on the accuracy and speed of orienting of attention (i.e., directional accuracy and onset latency of the first directionally correct saccade). These analyses were performed in R (v.3.4.0, R Core Team, 2020) using the lme4 package and the lmer (on the latencies) and glmer (on saccade accuracy) commands (Bates et al., 2015); the bobyqa optimiser algorithm was used to reduce failures to converge. All models were performed on unaggregated data, and saccadic onset latencies were log-transformed.

Fixed effects in the model were as follows: face location (left vs. right visual field), facial feature (typical—that is, no added feature—vs. disfiguring vs. occluding), facial feature location (left vs. right face half), face orientation (upright vs. inverted), and all of their interactions. The random effects were by-participants and by-item (face identity) intercepts, as well as their slopes as a function of the fixed effects. We started from this initial model with a maximal random effects structure (Barr et al., 2013); we then simplified the model to remove the perfect correlations (of 1.00 or –1.00) between random intercepts and random slopes. The model reported here contained random intercepts by-participants and by-items, as well as all fixed effects. Effects on saccadic latency and directional accuracy were interpreted as statistically reliable when |t| > 1.96.

Results

Figure 2 and Table 1 show, respectively, the average onset latency of the first correct saccade, and the proportion of directionally correct first saccades, as a function of the face condition and face orientation, and collapsed across the visual field location of the face. The output of each of the four analyses of latencies (Tables A1 and A2) and accuracy (Tables A3 and A4) can be found in the supplemental appendix. We also inspected saccadic onset latency differences in the face conditions relative to performance with upright normal faces, and show these for each saccade type in Figure 3.

Figure 2.

Latencies (mean and 95% confidence intervals) of the first correct saccade as a function of saccade type, face type, and face orientation: (a) prosaccade trials and (b) antisaccade trials.

Table 1.

Accuracy (proportion correct) of the first saccade as a function of saccade type, feature type, and face orientation.

Face condition	Upright	Inverted	Inversion effect
Prosaccade trials
Typical face	.968	.983	.016
Disfiguring feature	.960	.978	.018
Occluding feature	.968	.962	–.005
Antisaccade trials
Typical face	.838	.868	.031
Disfiguring feature	.851	.847	–.003
Occluding feature	.838	.851	.013

Figure 3.

Saccade latency differences between each face condition and the latency for upright typical faces (set to 0 ms; filled marker), on (a) prosaccade trials and (b) antisaccade trials.

Note. Positive values reflect delays in latency (relative to upright typical faces), and negative values reflect facilitations.

As is typical for this paradigm, across stimulus conditions, the onset latency of the first saccade was shorter on prosaccade trials than on antisaccade trials; likewise, there were more correct prosaccades than antisaccades. In the following, we describe the effects of the face conditions separately for each saccade type.

Prosaccade trials

Saccadic onset latency

The saccadic onset latencies towards typical, disfigured, and occluded faces did not differ reliably (Figure 2a). Inspection of the latency differences between upright and inverted faces, however, revealed an effect of face orientation, with faster onset latencies to upright than to inverted faces. This effect, however, depended on the face condition: It was present for typical faces (11 ms), but absent for occluded faces and reduced for faces with a disfigurement (0 and 7 ms, respectively). The LMM, reported in Table A1 (supplemental appendix), confirmed the above observations: There was a reliable main effect of face orientation, which was qualified by a reliable three-way interaction with the type and location of the feature; there were no other reliable effects.

Saccadic directional accuracy

The saccadic directional accuracy data (Table 1) showed a pattern similar as the onset latencies with more accurate saccades to inverted faces; this inversion effect was present for typical and disfigured but not for occluded faces. The LMM (Table A3 of supplemental appendix) confirmed this, showing a reliable effect of face orientation and a marginally reliable (p = .051) interaction between face orientation and feature type.

Antisaccade trials

Saccadic onset latency

On antisaccade trials, the onset latency of saccades away from the face was similar across feature type. However, there was a reliable interaction between feature type and face orientation (Figure 2b)—in that for typical faces, the antisaccade latency was slower (by 21 ms) for inverted than for upright faces, and this effect was absent for disfigured and occluded faces (–1 and –3 ms). The LMM (Table A2 of supplemental appendix) confirmed this, showing a reliable effect of face orientation and a reliable interaction between face orientation and feature type; there were no other effects.

Saccadic directional accuracy

The accuracy data showed a similar pattern as the latencies (Table 1), but here the LMM (Table A4 of supplemental appendix) did not reveal statistically reliable effects.

Saccade onset latencies relative to upright typical faces

To more closely inspect the nature of the above-reported inversion effects, we inspected the difference in prosaccade and antisaccade onset latencies between that of the upright typical face (acting as a baseline) and that of each of the remaining conditions. ² They are visualised in Figure 3 for prosaccade and antisaccade trials. They show the pattern of delays or facilitations in the onset latencies relative to the onset latency towards upright typical faces. Depending on the particular condition, the latency differences reported here may reflect effects of feature condition, face orientation, or both.

Prosaccade trials

Inspection of Figure 3A shows marginal effects of disfiguring/occluding features that appeared to be little affected by inversion. The presence of the disfiguring feature in the upright face facilitated (by 2 ms) the onset latency when the face was upright, but delayed it (by 5 ms) when the face was inverted. Thus, the reduced face inversion effect reported with disfigured faces (compared with typical faces; see Figure 3A appears to be related to a reduction in the latency cost by inversion rather than to an increased delay when the face was upright. The presence of the occluding feature caused a delay (by 3 ms) in onset latency (relative to that with upright typical faces), both when the face was upright and inverted. Thus, face inversion did not seem to affect the onset latency over and above the effect of the occluding feature.

Antisaccade trials

The presence of the disfiguring or occluding feature had similar effects on the onset latency of the antisaccade relative to that with upright typical faces (Figure 3B). Across both upright and inverted faces, the presence of the features delayed the onset latency of the antisaccade by a similar amount, although this delay was larger with occluded (10–13 ms) than disfigured (6–7 ms) faces. As a result, the inversion effects with the feature-bearing faces disappeared compared with typical faces. This pattern of latency differences suggests that the lack of inversion effects with disfigured and occluded faces is related to a delayed onset latency across face orientation.

Discussion

Using a mixed antisaccade task, we examined (1) the nature of initial covert attentional engagement by peripheral faces containing disfiguring features, and (2) the effect of disfiguring features on perceptual encoding of faces, as indexed by the effect of face inversion. First, given the potential of facially disfiguring features to elicit initial avoidance responses and negative emotions, we asked to what extent these responses may be driven by changes in covert orienting towards disfigured faces, as measured through the speed and directional accuracy of the first saccade. Second, we evaluated the impact of disfiguring features on initial perceptual encoding of the face during initial covert orienting by measuring the effect of face inversion.

In brief, our results suggest that the presence of a disfiguring or a control feature did not influence the orienting of attention (in terms of the first correct saccadic onset latency) towards upright faces, suggesting that avoidance responses towards facial stigma do not occur during covert attention. However, disfiguring and control features significantly reduced the effect of face inversion on saccadic latency, suggesting an impact on the holistic processing of facial information. In the following sections, we expand on this interpretation of our results in detail. Because directional accuracy was high, our interpretation is guided by the analyses of the latencies of first saccades.

Attentional (dis)engagement by facial features

The effect of the facial features on attentional (dis)engagement can be most purely estimated by examining the results with upright faces—that is, excluding any possible influence of face inversion. The onset latencies of the first saccade to peripheral upright faces presented no evidence that initial attention was engaged differently with the feature-bearing faces compared with typical faces. On prosaccade trials, the presence of a salient feature—disfiguring or occluding—did not influence the speed of attentional engagement compared with a typical upright face, showing neither an enhancement (a latency reduction) nor a suppression (a latency delay) of attention. On antisaccade trials, the presence of the facial features did increase attentional engagement with the—to be avoided—face, as suggested by a delay in onset latency relative to upright typical faces. However, this effect was qualified by an interaction with face orientation and therefore may not reflect a pure impact of the facial features in an upright face.

The lack of a consistent impact across saccade types suggests that covert attention to the peripherally presented face was not by default affected by these added facial features nor by their type (i.e., whether disfiguring or occluding). This finding is consistent with evidence from our previous study with the same stimuli, in which both peripheral and central faces failed to differentially influence covert attention (Boutsen et al., 2018; Experiments 2 and 3). However, this finding may still surprise as our observers consistently interpreted the disfiguring faces as disease-signalling and as affecting the appearance of the face, more so than they did towards faces with an occluding feature. The finding of an attentional modulation was premised on the perceived disfigured faces eliciting some level of negative emotional response—in particular, disgust or threat. However, because we did not explicitly measure the level of threat or disgust that our stimuli elicited in our observers, it remains to be tested how strong the emotional response to our disfigured face stimuli might have been. Indeed, our findings do not preclude the possibility that facial stigma that do induce strong emotional responses may affect covert attentional orienting in preparation of overt responses (cf. Llama-Alonso et al., 2020).

Facial features and face inversion effects

The presence of a disfiguring or occluding facial feature reduced or abolished the effect of face inversion on saccadic onset latency, compared with the effect of inversion found with typical faces in the absence of the features. On both pro- and antisaccade trials, the saccadic onset latency with a typical inverted face was slower compared with the saccade to an upright typical face (cf. Gilchrist & Proske, 2006; Hershler & Hochstein, 2005). On prosaccade trials, this inversion effect disappeared when the face had an occluding feature, and was reduced when the face had a disfiguring feature. On antisaccade trials, the inversion effect found with typical faces disappeared when the face had a disfiguring or an occluding feature. Here, with disfigured and occluded faces, saccadic latencies were slower than for upright typical faces, but faster than inverted typical faces. Our comparison of saccade onset latency differences between upright typical faces and each of the other face conditions (cf. Figure 3) confirms that the reduction or abolishment of the inversion effect with disfigured and occluded faces relates to an effect of the added features that appears to override the impact of face inversion. For instance, on antisaccade trials, the delay in disengagement of attention from a disfigured or occluded upright face remains the same when the face is inverted; its absence, however, makes the face subject to an inversion effect.

The differential inversion effects by disfiguring and occluding facial features may seem difficult to account for merely in terms of their salience (cf. Calvo & Nummenmaa, 2008), for in that case one would expect the same impact of inversion for either feature type. In the case of prosaccade trials, we suggest that because the disfiguring feature is perceived as embedded in, that is, intrinsic to, the face (as our post-experiment questionnaire indicated), its disruptive impact on holistic face processing—as indexed by the size of the face inversion effect—might be smaller than that for the occluding feature which was perceived as extrinsic to the face. On antisaccade trials, however, the abolishment of the face inversion effect that we observed for both disfiguring and occluding faces might well be accounted for in terms of visual salience: Here, initial attention had to be directed away from the face, and it is plausible that the mere presence of the added feature, rather than its specific perceptual interpretation, was relevant.

What do these findings reveal about the impact of disfiguring features on perceptual encoding of the face? Similar to the interpretation of face inversion effects, we hypothesised that disfiguring or otherwise perceptually salient facial features may disrupt the encoding of the global face configuration (as in holistic encoding; Rossion, 2008), making perceptual encoding of the face instead reliant upon featural information. ³ As a consequence, feature-bearing upright faces may be subject to a similar disruption to holistic encoding as inverted faces. With regard to saccadic latencies, our findings do support our hypothesis, in the sense that we did observe delays to saccadic latencies with upright feature-bearing faces. However, the reduction or abolishment of the face inversion effect was not merely related to this delay: The presence of the features on inverted faces also reduced the saccadic latency relative to inverted typical faces, as suggested by our inspection of latency differences. It is noteworthy that this reduction could not be accounted for by the concurrent increase in saccade latency with upright faces. Indeed, in all but one of the face conditions, the delay in saccadic latency (relative to upright typical faces; Figure 3 remained similar across upright and inverted faces.

Based on the above observations, we suggest that the effect on perceptual encoding by the facial features and by face inversion may differ qualitatively. While both may reduce holistic encoding, they appear to do so in different ways, by virtue of the differences in the information they represent. Our suggestion is based on the evidence that facilitated attentional orienting to faces—compared with non-face objects—can be driven by low-level visual information such as spatial frequency or amplitude spectrum, rather than by face orientation (Crouzet et al., 2010; Little et al., 2021). For example, Little et al. (2021) observed in a saccadic choice task that saccade latencies to faces were unaffected by inversion. While we did observe inversion effects on saccadic latencies to typical faces, we speculate that the detrimental impact of face inversion may be attenuated when the faces contain features representing salient low-level visual information.

Limitations and further research

Our findings raise some limitations as well as questions that can be addressed in further research. One important limitation to our interpretation of the effects on attention of disfiguring and occluding facial features is that they are based on the possible association, found in the literature, between facial stigma and feelings of disgust. This association, however, is likely to depend on other factors, including individual differences in the perception of facial stigma, as well as on the nature and severity of disfiguring facial features. Here, we did not measure the level of disgust or threat that individual participants might have experienced in response to our face stimuli, nor any other emotional impression formed. ⁴ Thus, while it is plausible that our findings reflect the association between facial stigma and negative emotions, it is possible that individual differences and stimulus factors may moderate this association. Using different methodology, future evaluations of emotional responses to the type of facial features used here would provide valuable information to clarify the role of emotion on attention to facial stigma.

Our findings raise some questions for future research into the effects of perceived facial stigma on attention and perception. The first concerns the generalisation of the results to other types of facial disfigurements. Our study involved facial disfigurements that could be perceptually segregated without affecting the generic structure (i.e., the configuration of mouth, nose, and eyes) of the face. While these stimuli were distinctive and realistic (as the ratings of our observers testify), other types of disfigurement may affect the structure of a face more profoundly and also involve the deformation of face parts, for example, in the case of a cleft lip or palate (Meyer-Marcotty et al., 2010). Such structural facial deformities may elicit stronger emotional responses like threat or disgust (Shanmugarajah et al., 2012) and could also result in a stronger attentional engagement.

Second, while our results demonstrate that the presence of a facial disfigurement interferes with the holistic encoding of faces and therefore promotes a more analytical, feature-based processing of facial information, the implications for the social appraisal of faces merit further study. For example, Fincher and Tetlock (2016) found a similar shift towards a feature-based processing when observers were primed with negative cognitions about a face. Given that feature-based processing typically is associated with objects rather than faces (e.g., Maurer et al., 2002), they interpreted this shift as evidence for a “dehumanizing” effect on face perception. One might speculate whether the acknowledgement of a facial feature as disfiguring might lead to an effect similar as Fincher and Tetlock (2016) describe. We think that this speculation is not warranted for this study, for we did not manipulate, measure, or prime negative cognitions about the face stimuli participants were presented with. Furthermore, it can be questioned whether perceptual disruption of holistic face processing can lead to a “dehumanization” of face cognition, a process that stretches beyond mere basic perceptual face processing.

In conclusion, our results show, on one hand, that covert orienting of attention towards or away from faces is not directly affected by the presence of facial features that may or may not signal disease. Thus, we did not find evidence of an avoidance response at the level of covert orienting of attention. On the other hand, these facial features do appear to have a distinct effect on perceptual encoding, and promote perceptual processing of the face to proceed in a feature-based rather than holistic manner.