Fear boosts the early neural coding of faces

Abstract

The rapid extraction of facial identity and emotional expressions is critical for adapted social interactions. These biologically relevant abilities have been associated with early neural responses on the face sensitive N170 component. However, whether all facial expressions uniformly modulate the N170, and whether this effect occurs only when emotion categorization is task-relevant, is still unclear. To clarify this issue, we recorded high-resolution electrophysiological signals while 22 observers perceived the six basic expressions plus neutral. We used a repetition suppression paradigm, with an adaptor followed by a target face displaying the same identity and expression (trials of interest). We also included catch trials to which participants had to react, by varying identity (identity-task), expression (expression-task) or both (dual-task) on the target face. We extracted single-trial Repetition Suppression (stRS) responses using a data-driven spatiotemporal approach with a robust hierarchical linear model to isolate adaptation effects on the trials of interest. Regardless of the task, fear was the only expression modulating the N170, eliciting the strongest stRS responses. This observation was corroborated by distinct behavioral performance during the catch trials for this facial expression. Altogether, our data reinforce the view that fear elicits distinct neural processes in the brain, enhancing attention and facilitating the early coding of faces.

Introduction

Facial expressions are important visual signals that provide information about internal emotional states and affective dispositions (Jack and Schyns, 2015). The efficient and rapid categorization of these signals is thus critical for adaptive social interactions. Despite numerous studies in the field of neuroscience and cognitive psychology, it is still debated when, where and how the brain achieves these biologically relevant tasks. The most prominent theoretical (Bruce and Young, 1986) and neuroanatomical (Haxby et al., 2000) models of face processing have posited a spatiotemporal dissociation between the processing of facial expression and identity. More precisely, at the anatomical and functional level, the decoding of facial identity takes place in a system involving the inferior occipital gyri and lateral fusiform gyrus (including the fusiform face area). Whereas facial expression categorization occurs in a separate system that includes the occipital cortex, the right posterior superior temporal sulcus (pSTS), and the amygdala (for a review see, Calder and Young, 2005; Pessoa, 2008).

However, experimental evidence remains inconclusive on whether this clear-cut anatomical and functional separation between identity and facial expression recognition is valid. On the one hand, some studies have reported a neural dissociation of these dimensions in brain-damaged patients (e.g. Shuttleworth et al., 1982; Bruyer et al., 1983; Tranel and Damasio, 1988; Sergent and Villemure, 1989; Haxby et al., 2000; Mattson et al., 2000; Richoz et al., 2015; Fiset et al., in press). Functional dissociations have also been found in electrophysiological studies with primates (e.g. Hasselmo et al., 1989) and human functional neuroimaging studies (e.g. Winston et al., 2004). Conversely, some authors have challenged this view suggesting the existence of a complex interplay between the decoding of emotional and identity information in a network of regions within the occipital and temporal cortices. For example, several studies have shown a functional involvement of the inferior occipital gyrus in both identity and expression recognition (e.g. Adolphs, 2002; Pitcher, 2014). Similarly, the lateral fusiform gyrus seems to be involved in both tasks (e.g. Dolan et al., 2001; Williams and Mattingley, 2004; Ganel et al., 2005; Fox et al., 2009). Vuilleumier et al. (2001) have demonstrated that responses in the lateral fusiform gyrus are modulated by the nature of the facial expressions, with fearful faces eliciting stronger activation than neutral faces. Using intracranial local field potential recordings, Pourtois et al. (2010) also revealed an anatomical overlapping between identity and emotional face processing in the right fusiform gyrus. Other functional Magnetic Resonance Imaging (fMRI) studies have also reported sensitivity to identity in the middle and the pSTS (Winston et al., 2004; Fox et al., 2009). Altogether, these studies suggest the existence of a more complex and comprehensive processing system with many overlapping activities for both identity and expression recognition across those brain regions (see also D’Argembeau and Van der Linden, 2007; Todorov et al., 2007), leaving the question of an anatomical and functional dissociation still open.

Event-related potential (ERP) studies have also provided mixed results regarding the time-course of identity and facial expression categorization. The face-sensitive N170 component (Bentin et al., 1996) is the most important electrophysiological signature for studying the early dynamics of face processing. The N170 is a bilateral occipito-temporal negative deflection peaking roughly 170 ms after stimulus onset, which is larger for faces compared to other non-face visual categories (Bötzel et al., 1995; George et al., 1996). Activity in this time window is associated with early accumulation of perceptual information leading to structural encoding stages (Bentin and Deouell, 2000), such as detection (Jeffreys, 1989; Rousselet et al., 2004) and visual categorization (Eimer, 1998; Schweinberger et al., 2002; Itier et al., 2007). Many studies have convincingly shown that the N170 is also sensitive to face identity (Campanella et al., 2000; Guillaume and Tiberghien, 2001; Itier and Taylor, 2002; Jemel et al., 2005; Heisz et al., 2006; Caharel et al., 2009; Jacques and Rossion, 2009; Vizioli et al., 2010a, 2010b), as well as to the detection of other important facial features, such as gender and race (Caldara et al., 2003, 2004b). However, it remains yet unclear whether the N170 is also sensitive to facial expressions. Some studies have reported such expression-sensitive N170 modulations (e.g. Batty and Taylor, 2003; Blau et al., 2007; Schyns et al., 2007; Morel et al., 2009; Smith, 2012; DaSilva et al., 2016;), while others have not (e.g. Campanella et al., 2002; Ishai et al., 2006). A recent meta-analytic review of ERP studies investigating the sensitivity of the N170 to emotional expression by Hinojosa et al. (2015) points toward a general pattern for greater N170 responses to a certain array of expressions (fear, anger, happy), with greater effect sizes for tasks involving indirect attention to the emotional expressions of the faces. The review also highlighted a point of interest of whether emotional expression may modulate the N170 individually, or whether some expressions produce greater modulation while others none at all, which requires a broad array of expressions involved in such paradigms.

Given the inconsistency of the results between studies using different tasks and emotional expressions, the timing of the processing of these expressions is yet unresolved, arising either during the structural encoding N170 stages of processing, or with facial expressions being encoded at a later stage (Eimer and Holmes, 2002; Caldara et al., 2004a; Rellecke et al., 2013).

The differences in task design and methodology, as well as the use of only a subset of facial expressions, might account for some inconsistency in these results (e.g. Pourtois et al., 2005; Caharel et al., 2007; Langeslag et al., 2009; Righi et al., 2012; Morel et al., 2014, for review see Hinojosa et al., 2015). Tasks are, in fact, cognitive contexts that modulate the encoding functions of high-level vision (Schyns, 1998; Kay et al., 2015). It is thus plausible that the categorization task itself can drive the selective search of information from the available visual inputs, directing the visual system towards the features that are the most useful for the task (Goffaux et al., 2003). Accordingly, it is appropriate to investigate how emotion and identity discrimination tasks influence the early processing of faces containing an array of varying emotions.

Some behavioral studies have also suggested that facial expression and identity are not processed independently, through the use of selective attention tasks such as the Garner interference paradigm (Garner, 1976). In this kind of paradigm, observers have to selectively attend to a relevant dimension (e.g. identity), while ignoring another randomly varying dimension (e.g. expression). Garner interference occurs when variations of the irrelevant dimension cause a decreased accuracy and longer reaction times along the relevant dimension, and supports the conclusion that the considered dimensions are processed together. Garner interference has been observed for both identity and expression (as relevant dimensions), suggesting that both these types of facial information are not processed entirely independently (e.g. Ganel and Goshen-Gottstein, 2004; Fisher et al., 2016). Importantly, a recent ERP study employing the Garner interference paradigm with a subset of facial expressions reported evidence for an interaction between identity and expression on the identity-sensitive N250r component (occurring between the 220–320 ms time range, see Fisher et al., 2016), but not on the earlier N170 range.

Since the N170 sensitivity to facial expressions appears highly heterogeneous, more evidence is needed to clarify whether this component is sensitive to a few emotional categories or all the basic facial expressions. In fact, some emotions are more important than others for survival and would require a prompt adaptation of our behavior to salient events. Particularly, expressions that are associated to both threatening and comforting situations (e.g. fear and happy) are processed more rapidly compared to others (Öhman et al., 2001; Algom et al., 2004; Leppänen et al., 2007), even without conscious awareness (e.g. Smith, 2012; for a review see also, Tamietto and de Gelder, 2010). In summary, it remains to be clarified whether all facial expressions of emotion uniformly modulate the N170 and only when the categorization of emotion is task-relevant.

Repetition suppression (RS) or adaptation procedures are particularly promising to clarify this issue. RS reflects a short-term neural decrease, elicited by the repetition of the same visual input, occurring uniquely in neural populations coding for this information. Given its high sensitivity, RS can be considered as one of the most powerful tools for testing the coding and recognition of visual inputs during early stages of information processing (Vizioli et al., 2010b) as well as later stages as for example during identity coding (Ramon et al., 2010). In fact, we have previously demonstrated that the use of conventional electrophysiological paradigms might not be sufficiently sensitive to capture subtle electrophysiological responses on the N170 when coding for the race of (upright) faces (Vizioli et al., 2010b). For identity, there is a general consensus among studies investigating the effects of repetition on the N170 component, with greater RS found for the presentation of pairs of faces with the same identity (e.g. Itier and Taylor, 2002; Harris and Nakayama, 2007; Ewbank et al., 2008; Caharel et al., 2009; Ramon et al., 2010). A recent meta-analytic review of emotional expression effects on the N170 (Hinojosa et al., 2015) notes that while there is a majority of studies indicating this effect, the strength of such effects is modulated by the choice of electrode reference(s) in the data collection/processing stage. As the choice of the reference electrode has been found to modulate the intensity and spatial location of N170/VPP (vertex positive potential) effects (Joyce and Rossion, 2005), the ability to measure a significant N170 modulation by emotional expression may relate both to the signal to noise ratio involved in the paradigm and technical setup, as well as a priori expectations of where in the scalp and when such effects might manifest. Thus, the use of a data-driven single-trial repetition suppression (stRS) approach seems suitable to clarify this issue, especially given that this approach was able to reveal what had otherwise been an inconsistent or null effect with conventional analyses (see Vizioli et al., 2010b).

In the present study, we investigated the neural coding for all six basic facial expressions of emotion (anger, disgust, fear, happy, sad, surprise) plus neutral in recorded electrophysiological signals. Specifically, we investigated whether N170-amplitude modulation occurs only when the emotion categorization is task-relevant. To this aim, we quantified the amount of neural adaptation triggered by the repetition of faces displaying identical facial expressions (trials of interest). High-temporal resolution scalp EEG signals were thus registered during the repetition of two faces (an “adaptor” and a “target”), always displaying the same identity and the same facial expression (i.e. trials of interest; in Figure 1A reported as stRS trials). In each sequence, the two faces displayed one of the six basic expressions or the neutral expression. Importantly, to control task constraints, we also included catch trials to which participants had to react (Figure 1B), by varying identity (identity-task), expression (expression-task) or both (expression- and identity-task) on the target face (dual-task). Thus, observers were required to respond only on mismatching trials, leaving the electrophysiological trials of interest uncontaminated by behavioral responses.

Fig. 1.

Experimental procedures. Left panel (A) shows the experimental design of the trials of interest (used for single trial repetition suppression analyses, stRS). Right panel (B) shows the catch trials for the identity-, expression- and dual-task. Please note that the catch trials from identity- and expression-task were also presented during the dual-task as distractors.

We hypothesized that if the early N170 component codes for facial expressions, RS responses would be modulated by the nature of the facial expressions, facial identities being kept equal. Furthermore, if the N170-facial expression coding occurs only when the categorization of emotion is task-relevant, neural differences should emerge only when attentional resources are directed toward this information (see, e.g. Pernet et al., 2007). To anticipate our findings, the single-trial data-driven spatiotemporal analysis did not reveal task-specific modulations. Instead, our data showed a general and strong neural adaptation response for the facial expression of fear regardless of the task at hand. These observations indicate an early specific coding of facial expressions on the N170, with fear eliciting the largest adaptation regardless of task constraints.

Materials and methods

Participants

Twenty-two (10 male) right-handed first-year students from the University of Fribourg took part in the experiment, with an average age of 21.5 years old (range 19–32). All participants had normal or corrected-to-normal vision. They signed a consent form describing the main goals of the study and received course credits for their participation. The Ethical Committee of the Department of Psychology of the University of Fribourg approved the study reported here.

Stimuli and procedure

Seventy grayscale face images containing 10 different identities (five females) were obtained from the Karolinska Directed Emotional Faces database (Lundqvist et al., 1998). Each identity contained seven distinctive facial expressions (anger, disgust, fear, happy, neutral, sad, surprise). After the removal of external facial features, all images were normalized for low-level visual properties using the SHINE toolbox (Willenbockel et al., 2010). Face stimuli were presented at the center of the screen at a viewing distance of 75 cm.

Participants sat in a dimly lit, sound-attenuated electrically shielded booth. They performed three different tasks (expression-, identity- and dual-task) in a counterbalanced random order. All tasks followed the same RS paradigm with two different types of trials: trials of interest and catch trials. Only the trials of interest were analyzed in the EEG data analysis using the stRS approach (Vizioli et al., 2010b). These trials were identical across all tasks, with the same face stimulus presented both as an adaptor and as target (Figure 1A). For catch trials, we changed visual features of the target with reference to the adaptor (Figure 1B). While in the identity-task, the target stimulus displayed a different face identity with the same facial expression, in the expression-task, the target stimulus displayed the same face identity but with a different facial expression. In the dual-task, both face identity and facial expression were different. In addition, to ensure that participants’ attention was directed toward a double genuine categorization of both dimensions, during this task, we presented also catch trials from the identity-task and the expression-task as distractor trials. However, participants had to respond only to catch trials where changes occurred for both identity and expression while ignoring the distractor trials with changes in only one dimension (i.e. identity or expression). Participants were requested to press the space bar whenever they saw a catch trial. No behavioral responses were expected during the trials of interest.

All three tasks consisted of the same 560 trials of interest (10 identities × 7 expressions × 8 repetitions). Participants responded to 70 catch trials in all three tasks, whereas in the dual-task there were 140 additional distractor trials, which were the same catch trials as those found in the identity and expression tasks (i.e. distractor trials). Moreover, the image sequence was pseudorandom, so that each face stimulus was presented with equal probability as adaptor and as a target.

The experiment was presented on a VIEWPixx/3D display system [resolution 1920 (H) × 1080 (V) pixels, refresh rate 120 Hz]. Each trial started with a black fixation cross ≈0.3° of visual angle, presented at the center of the screen for 300 ms. The adaptor face was then presented for 350 ms, followed by an interval of random duration with a black fixation cross (200, 250, 300 ms). The target face was then presented for 300 ms. The offset of the second face was followed by a randomized inter-trial interval between 1300 and 1500 ms.

To minimize low-level adaptation, we manipulated the stimuli size for the adaptor and target: the size of the adaptor face was 11.8 cm × 15 cm (9.636° × 12.231°), whereas the size of the target was smaller: 10.7 cm × 13.5 cm (8.74° × 11.016°). The whole experiment was programed in MATLAB, using the Psychophysics Toolbox (PTB-3 Brainard, 1997; Kleiner et al., 2007). EEG triggers were sent from the VIEWPixx system and were synchronized with the stimuli presentation using the DataPixx Toolbox.

EEG recording

EEG data were acquired with an 128-channel Biosemi Active Two EEG system (BioSemi, Amsterdam, Netherlands). The recording reference and ground consisted of two active electrodes (CMS, Common Mode Sense; DRL, Driven Right Leg). Analog signal was digitized at 1, 024 Hz. Electrode offsets were kept between a maximum of ±25 μV, with a mean offset around 15 μV. Participants were asked to minimize blinking, head movements and swallowing.

Data analysis

We applied mixed-effects logistic regression analyses for the behavioral responses (Jaeger, 2008). We analyzed the frequency of hits (accurately identified catch trials) and false alarms (incorrect responses during the trials of interest and incorrect responses during the presentation of the distractor trials in the dual-task) using GeneralizedLinearMixedModel in MATLAB R2014b with a binomial model. After model fitting, statistical hypothesis testing was performed on the model coefficients using CoefTest in MATLAB. The fixed effects were Expression (7 levels: angry, disgust, fear, happy, neutral, sad, surprise) of the face adaptor, Task (3 levels: identity, expression, dual) and their interaction. The subject-specific effects were considered as the random effect of the model. We predicted the worst performance in the dual-task, as this task makes it particularly difficult for the participants to ignore the presentations of identity and expression catch trials. To further investigate and clarify the modulation effects of facial expressions on this latter task, we conducted a separate mixed-effects logistic regression analysis on the false alarm rates of the dual-task.

We used EEGLAB (Delorme and Makeig, 2004) to perform EEG pre-processing. Only trials of interest with no behavioral response (i.e. correct rejections) were analyzed (i.e. valid trials). EEG signal was low-pass filtered at 40 Hz with a slope of 6 dB and high-pass filtered at 1 Hz. Trials were segmented across an epoch of −100 ms to +550 ms and the average 100 ms pre-stimulus activity was removed from every time-point, independently for each electrode at the single trial level.

We used the same approach as the one used in Vizioli et al. (2010b) for statistical inferences on the valid trials (stRS). As RS refers to a stimulus specific reduction of neural activity, the stRS consists in subtracting the activity elicited by the target face from the activity elicited by the adaptor. This data-driven approach was used primarily for mapping significant electrophysiological effects at all electrodes and time-points, considering the response to the target as not independent from the response to the adaptor. Importantly, this analysis makes no a priori assumption about where and when to look for effects in the ERP signals (Vizioli et al., 2010b).

The signal elicited by both adaptor and target face was rejected from the analysis if either one of the two epochs was contaminated by artifacts (e.g. blink). We used a variance-based algorithm (for details see Vizioli et al., 2010b) during the stRS analysis for excluding on a subject-by-subject basis the epochs contaminated by artifacts. The minimum number of accepted trials across all participants and tasks was 401. The trials included in the average count were limited to the trials of interest. The mean numbers of accepted trials per expression across all the tasks were 67.92 (s.d. = 3.56).

We then applied Hierarchical Linear Model on the EEG data using functions from the LIMO EEG toolbox (Pernet et al., 2011). Statistical analyses of stRS were then performed first within single subjects (level 1) and then at the group levels (level 2). In the first level, we modeled the stRS amplitudes for each categorical condition independently for each participant using a general linear model (GLM) across trials, at all-time points and all electrodes. In the second level, we modeled the stRS amplitudes in GLM as follows:

$${\mathbf{s}\mathbf{t}\mathbf{R}\mathbf{S}}_{t,e}=\mathbf{X}\mathit{\beta }+\epsilon ,\mathrm{ }for\mathrm{ }t\mathrm{\hspace{0.17em}}\sim \mathrm{\hspace{0.17em}}\left[-100,550\right]\mathrm{ }milli\hspace{0.17em}\text{sec}\hspace{0.17em}ond,\mathrm{ }e\mathrm{\hspace{0.17em}}\sim \mathrm{\hspace{0.17em}}\left[1,128\right]\mathrm{ }channel$$

where stRS was the response matrix containing the amplitudes for each time frame (t) and electrode (e) and X coded for 21 experimental conditions [all possible combinations of expressions (7) and tasks (3)]. The beta parameters (β) were estimated using ordinary least square. Hypothesis testing was performed by conducting a 7 × 3 (Expression: 7 levels: angry, disgust, fear, happy, neutral, sad, surprise, × Task: 3 levels: identity, expression, dual) ANOVA on the model coefficients (β).

Multiple comparison correction was performed using a bootstrap spatiotemporal clustering technique (Wilcox, 2005; Maris and Oostenveld, 2007; Vizioli et al., 2010b; Lao et al., 2013). We first centered the stRS so that each condition had a mean of zero. We bootstrapped the centered stRS by sample with replacement of the subjects, and then performed the same linear contrasts as statistical testing. This procedure was repeated 500 times; each time we recorded the maximal F-value sum in the significant cluster (cluster mass). By doing this, we derived an estimate of the cluster mass distribution under the null hypothesis. The 95% largest value in the bootstrapped cluster distribution was set as the cluster threshold at P < 0.01. The cluster mass in the original result was then compared with this threshold to assess the significance after multiple comparison correction.

Finally, we used a conventional RS analysis to clarify the underlying mechanism of the stRS effect (i.e. different ERP modulations triggered independently by the adaptors and the targets presentation). We applied a one-way ANOVA within each significant spatial cluster considering the Expressions (7 levels: angry, disgust, fear, happy, neutral, sad, surprise) independently for the adaptor and target conditions. Paired t tests were also performed to investigate which facial expression triggered significant N170 effects at the adaptor and target levels. Importantly, we applied these analyses on the same trials as those included in the stRS analysis.

Results

Behavioral results

We found a significant main effect of Task for both hits and false alarms. As predicted, participants performed worse in the dual-task compared to the other two tasks [hits: F(2, 441) = 114.61, P < 0.05, Figure 2A; false alarms: F(2, 441) = 355.43, P < 0.05, Figure 2B]. The main effect of Expression was also significant for hits [F(6, 441) = 5.15, P < 0.05], as well as the interaction between Task and Expression for both hits and false alarms [hits: F(12, 441) = 2.47, P < 0.05; false alarms: F(12, 441) = 1.98, P = 0.024]. As shown in Figure 2B, by performing a mixed-effects logistic regression on the dual-task false alarm rate, we found that participants made significantly more false alarms when they were adapted to fear compared with sad [t(147) = 3.53, P = 0.0005, significant after Bonferroni correction], disgust [t(147) = 3.16, P = 0.0019], happy [t(147) = 3.66, P = 0.0003] and neutral [t(147) = 3.72, P = 0.0003] expressions.

Fig. 2.

Behavioral results obtained for the catch trials during the identity-, expression- and dual-task. The bar plots show the estimated coefficients from the logistic mixed-model and error bars show their 95% confidence interval (95% CI). (A) Percentage of hits (i.e. correct responses to catch trials) for the three tasks and the six facial expressions, plus neutral. (B) Percentage of false alarms (i.e. incorrect responses to catch trials) for the three tasks and the six facial expressions, plus neutral. Asterisks indicate statistical significance.

Electrophysiological results

stRS results

The 7 × 3 (Expression × Task) ANOVA carried out on the model coefficients (β) gave only a significant main effect for the factor Expression over bilateral occipito-temporal electrodes (Figure 3A and B, P < 0.05 corrected for multiple comparisons). The stRS amplitude and beta coefficients time course are shown independently for the left and right occipito-temporal cluster in Figure 3C–F. stRS responses were time-locked to the onset of the N170 component.

Fig. 3.

N170 repetition suppression effects. (A) Significant F values across the time (ms) for all electrodes after bootstrap clustering as multiple comparison corrections. (B) Topography of the significant F value at around 130 ms (maximum F value). (C) Average stRS in μV for the left occipito-temporal cluster (Cluster 1 in B). (D) Average stRS in μV for the right occipito-temporal cluster (Cluster 2 in B). (E) Average beta coefficients for the left occipito-temporal cluster (Cluster 1 in B). (F) Average beta coefficients for the right occipito-temporal cluster (Cluster 2 in B). All six facial expressions plus neutral are depicted (anger, blue line; disgust, red line; fear, yellow line; happy, purple line; neutral, green line; sad, light blue line; surprise, dark red line). (G) Bar plots of the mean stRS beta values within the significant whole cluster (i.e. left and right occipito-temporal electrodes) for the identity-, expression- and dual-task. Error bars show standard errors from the means.

The F-value of both clusters reached its maximum at ∼134 ms [left cluster maximum at electrode P9: F(6, 441) = 20.35; right cluster maximum at electrode P8: F(6, 441) = 30.96; P < 0.05 clustered corrected]. Posthoc paired-sample t tests on the model coefficients independently within each cluster (i.e. the left and right occipito-temporal electrodes) revealed that the repetitions of pairs of faces expressing fear elicited the largest stRS responses [β_left = –2.11, 95% CI (2.438, –1.775), β_right = –1.92, 95% CI (—2.253, –1.587); Figure 3G] across all tasks.

Post hoc ERP peak analysis within cluster

To pinpoint potential significant contributions of the different facial expressions in the adaptor and target conditions in modulating the stRS responses during each task, we performed a series of analyses on the ERP mean amplitude within each significant spatiotemporal cluster and in the time window ranging from 110 to 145 ms (as revealed by the data-driven stRS analysis—see Figure 4).

Fig. 4.

ERPs for the adaptor and target faces, and stRS elicited at the left (A, electrodes POz, P003h, O1, Oz, Ol2h, Ol1h) and the right occipito-temporal clusters (B, electrodes PO4, PO6, PO8, POO10h, PO10, PPO10h) for the seven Expressions (anger, blue line; disgust, red line; fear, yellow line; happy, purple line; neutral, green line; sad, light blue line; surprise, dark red line) in each Task (identity-, expression- and dual-task). Asterisks indicate statistical significance.

We found a significant main effect of the Expression for both the adaptor [F(6, 453) = 18.48, P < 0.05] and the target [F(6, 453) = 2.23, P = 0.04], within the left occipito-temporal cluster. Post hoc paired t tests revealed that fearful faces elicited significantly different responses compared with the other facial expressions in all tasks (see Figure 4A). At the adaptor presentation level, fear elicited significantly larger (negative) ERP responses compared with all other expressions (t_max = –8.49, t_min = –3.20, P < 0.05 Bonferroni corrected). At the target presentation level, fear elicit significantly larger (positive) ERP amplitude than surprise only (t = –3.41, P < 0.05 Bonferroni corrected).

Within the right occipito-temporal cluster, the Expression effect was significant only for the adaptor [F(6, 453) = 11.05, P < 0.05] but not for the target [F(6, 453) = 1.05, P = 0.395] in all tasks (see Figure 4B). Post hoc paired t tests revealed that fear among the other facial expressions elicited larger N170 compared with all other expressions (t_max = –6.77, t_min = –3.09, P < 0.05 Bonferroni corrected).

Discussion

We investigated the electrophysiological coding of facial expressions of emotion by using a RS paradigm with three tasks directing attention towards either identity, expression or on both types of information (i.e. dual-task). We measured the neural RS responses following the repetition of pairs of faces displaying the same identity and expression, for all the basic facial expressions, plus neutral. Importantly, by manipulating task demands while keeping the visual inputs identical, we aimed to assess and quantify whether facial expressions could modulate information processing, especially on the early N170 face-sensitive ERP. We applied a logistic mixed model on the behavioral data acquired on catch trials to isolate task-effects, and a spatiotemporal data-driven analysis on stRS ERP data on the trials of interest to quantify task-specific neural modulations (Vizioli et al., 2010b; Lao et al., 2013).

Our data revealed a distinctive signature for the facial expression of fear, both at the behavioral and electrophysiological levels. The presentation of fear as an adaptor induced significantly higher false alarm rates in the dual-task (i.e. incorrect responses to distractor trials), as well as stronger electrophysiological stRS signals over a bilateral cluster—the left and right typical face-sensitive N170 occipito-temporal electrodes, regardless of the task (i.e. identity, expression or both). To clarify further the nature of the mechanisms underlying such adaptation responses, we also performed conventional analyses on the average ERP on these two clusters, independently for the adaptor and the target amplitudes. Our data revealed that ERP modulations appeared over left and right occipito-temporal electrodes already for the adaptor faces, while a significant effect for both adaptor and target faces on the left occipito-temporal electrodes. Even if the behavioral performance and stRS electrophysiological signals arise from two dissociable sources of information (i.e. catch trials and trials of interest) and cannot be straightforwardly compared, both results corroborate the hypothesis of a distinct neural processing for fear compared to other facial expressions (see, e.g. Vuilleumier et al., 2001; Batty and Taylor, 2003; Öhman, 2005; Blau et al., 2007).

The central and novel finding of our study is the strong electrophysiological stRS response for fear. In line with many studies showing an early coding and sensitivity to facial expressions on the N170 component (e.g. Schyns et al., 2007), our study feeds the N170-facial expression sensitivity debate by contradicting further the view that it is not modulated by the emotional content (Münte et al., 1998; Krolak-Salmon et al., 2001; Eimer and Holmes, 2007; Fisher et al., 2016). In particular, we found a specific RS modulation for fear that allowed us to reveal a discrete nature of this emotional information coding at the N170 level. Indeed, RS represents a sharpening mechanism within the neural population that is engaged in the processing of the repeated stimulus (Wiggs and Martin, 1998; Grill-Spector et al., 2006). Thus, the amount of suppression could be interpreted as an indication of the ability of the face-sensitive neural populations to discriminate between different visual information. Our spatiotemporal data-driven approach indicates that the expression of fear boosts the early coding of individual faces regardless of the attentional constraints required for the effective categorization of identity, expression or both. Our findings are also consistent with an event-related fMRI study by Vuilleumier et al. (2001) investigating the role of spatial attention in modulating neural responses to fearful and neutral faces. Their results revealed that fear elicited stronger activations in the fusiform gyrus, independently of the effect of attention. From a sociobiological perspective, the expression and perception of fear is highly advantageous for human survival (LoBue, 2010) and can thus serve to trigger an enhanced processing of perceptual events (e.g. Phelps et al., 2006). Indeed, fearful faces automatically modulate attention (Fox et al., 2001; Pourtois et al., 2005; Carlson and Mujica-Parodi, 2015), producing a greater bias compared with happy faces (e.g. de Haan et al., 2003, Leppänen et al., 2007) and interfering with behavioral performance, by generating specific attentional narrowing (Eastwood et al., 2003). In our experiment, we observed stronger occipito-temporal responses for fear on the N170 time window, not only in the stRS responses, but also already during the presentation of the adaptor and target faces. It is worth noting that these electrophysiological modulations were observed with comparable attentional task demands across trials of interest. In line with these previous reports, these observations suggest that fear might indeed automatically and uniquely enhance attention.

As reported above, the novelty of the current study relies on the stRS paradigm and analysis, which takes into account the combined electrophysiological effects triggered by both adaptor and target presentations. It is worth noting that this methodological approach does not ignore the responses to adaptor or target faces, but rather incorporates both. The stRS approach represents, in fact, a new way to tap precise neural computations of RS (Vizioli et al., 2010b). A feature of the stRS approach was that while it allowed us to identify that the N170 RS response was sensitive to the encoding of a fearful facial expression, we were also able to analyze the ERP responses to the adaptor and target faces through conventional ERP analyses, which also confirmed an effect of emotional expression on the N170 responses. Although the precise neural computations of RS are still not completely understood and are highly debated in the literature, the adaptation elicited by two stimuli presented in rapid succession is currently interpreted as the engagement of the same neural population in the processing of both adaptor and target stimuli (Vizioli et al., 2010b; Grill-Spector et al., 2006). In our study, we found as expected the stRS for fearful faces triggered in both clusters by the adaptor presentation (for a similar result, see Williams et al., 2006). This implies that since fearful faces led to greater RS in both scalp hemispheres, the processing of fearful expressions appears to be bilateral in nature (e.g. Alves et al., 2008).

The increased sensitivity we observed uniquely for fear, might be related to a rapid perception of possible threatening situations and sustains coping strategies such as fighting, freezing or rapid escape (e.g. Armony and LeDoux, 2000; Calder et al., 2001). Thus, despite the fact that all facial expressions of threat are evolutionarily significant and more likely to capture attention (the threat-superiority effect, see, e.g. Öhman et al., 2001; Smith et al., 2003; Blanchette, 2006), when it comes to a task requiring a very rapid response, only fear enhances neural modulations at the early stage of visual processing. Therefore, it is possible that identity and emotional processing occurs in parallel through overlapping structures/cognitive modules, but only in response to certain evolutionarily important emotions (fear), rather than there being either a clear separation of identity and emotion processing or indeed a general overlap.

While we observed an early distinct electrophysiological response to fear, other researchers have also shown greater N170 modulations for other facial expressions, such as anger and happy (e.g. Williams et al., 2006; Almeida et al., 2016, see Hinojosa et al., 2015 for review). Theoretically, earlier anger and happy effects could be expected given their behavioral significance (see, e.g. Calvo and Beltrán, 2013). When the results of the present study are compared with those reported in the meta-analysis by Hinojosa et al. (2015), there are points of difference regarding the anger and happy expression, which may mostly be accounted for by differences in the experimental designs. The current study used an array of all facial expressions of emotion, involved a repetition paradigm and differed in the EEG analysis approach. It is likely that this combination of differing factors may lead to observed differences between the current and the previous studies. For example, the sensitivity to anger has been reported during orthogonal or passive viewing tasks. Similarly, the advantage for processing happy faces has been mostly demonstrated in long-term memory tasks. Interestingly, a very recent ERP study found this advantage (compared with neutral and pride-positive expressions), but only at later stages (starting from 800 ms) of face processing (DaSilva et al., 2016). However, here we used a facial adaptation paradigm, which might have increased the sensitivity to capture early neural responses, particularly during the adaptor presentations. Adaptors, in fact, could drive a general category-adaptation mechanism acting at a very early stage of visual processing (Eimer et al., 2010); please note that, in our study, fearful face adaptors triggered N170 modulations in both left and right hemispheres.

Contrary to previous reports (Williams et al., 2006; Smith et al., 2003), we did not find modulations (e.g. enhanced amplitudes) on the P100 (or P1) and C1, as well as later components (i.e. P2, N2). These components have been associated in particular with automatic attentional orientation (e.g. Luck et al., 2000; Taylor, 2002) for threatening stimuli (e.g. Pourtois and Vuilleumier, 2006). However, the early enhancements elicited on the P100 component by fearful expressions might be linked to difference in the low-level properties across stimuli and facial expressions, such as differences in luminance and brightness (Puce et al., 2013). In our study, all the stimuli were normalized for low-level visual properties and such a control might explain the absence of modulations on this component. The P2, N2 and C1 components have been related to both automatic task evaluation processing and controlled cognitive processing in a wide variety of tasks. For instance, modulations on those components have been reported during subliminal/masking or shifting paradigms, in which emotional faces were processed without awareness (Eimer and Kiss, 2008; Pegna et al., 2008) or when covert attention was shifted towards emotional face-cues (during a bar-probe task, e.g. Pourtois et al., 2004). Similar results have also been observed during the visual search of emotional stimuli in the occipital-temporal region (i.e. negative deflection on the N2pc; Luck and Hillyard, 1994), which have been related to an attentional shift (e.g. Eimer and Kiss, 2008). Such modulations occurred at about the same latency of a prefrontal positivity (the P2a), a neural index of attentional capture and stimulus evaluation (e.g. Kanske et al., 2011). Following this logic, a possible explanation for the lack of modulations on those components in our study might relate to a decrease in the allocation of attentional resources due to the absence of active behavioral responses during the trials we used to extract the neural adaptation signals (i.e. trials of interest). Therefore, despite these studies supporting the current findings, with fearful expression generally boosting earlier electrophysiological effects, once again the use of varying paradigms might be on the basis of the absence of an effect on those components.

Regarding the emotional state and intentions conveyed by facial expressions, an early posterior negativity (occipito-temporal EPN; 150–300 ms) have also been observed, in particular, for angry as compared to neutral faces, and happy faces (Sato et al., 2001; Liddell et al., 2004; Schupp et al., 2004; Williams et al., 2006; Holmes et al., 2008). It is important to underline that these effects have been shown to be more pronounced in socially anxious participants (Moser et al., 2008; Sewell et al., 2008) and participants undergoing socially-mediated aversive anticipation (Wieser et al., 2010; Bublatzky and Schupp, 2012). In addition, stimulus arousal level contributes highly to EPN since highly arousing pictures (mutilations and erotica) elicit larger amplitude EPNs than fewer arousing pictures for both unpleasant and pleasant categories (Schupp et al., 2004). Therefore, it is reasonable to conclude that an absence of an effect on the EPN component could be simply due to the use of both participants who were not selected according to differences in anxiety levels and only face stimuli in our study.

Contrary to the electrophysiological results, at the behavioral level participants made more false alarms when they saw catch trials displaying fearful compared with disgust, happy, sad and neutral adaptors but only during the dual-task. As expected, this task was cognitively more demanding (Pashler, 1992, 1994). In fact, to engage participants into a genuine categorization of a simultaneous violation on both identity and expression dimensions, the dual-task contained also catch trials with a unique violation of either identity or expression (i.e. the identity- and the expression-only catch trials), which participants were required to ignore.¹ At first sight, fearful adaptors seem to distract more. However, this result might also indirectly support the hypothesis of an activation of the ‘fear module’, which elicited a greater reactivity (i.e. false alarms) in our participants. On the other hand, fearful faces may have influenced participants’ performance by decreasing the ability to disengage attention from these faces (Fox et al., 2002). Indeed, it is well-established that fearful expressions play an important role in the allocation and capture of attentional resources (Fenske and Eastwood, 2003).

Further studies are necessary to clarify this observation, by either using an active task on all the trials (and not only on the catch trials) or an experimental design involving the repetition of pairs of faces with different identities but same expressions and similar task constraints. Similar studies should also clarify the role of a potential interaction occurring during the first stages of face processing between identity and expression information, as reported recently by Fisher et al. (2016). In fact, it must be noted again that the use of a different task (i.e. Garner and RS paradigms) could be at the origin of the time course differences found by the current and this earlier study. Overall, the current pattern of results posits the facial expression of fear as one of the most significant for the human adaptive functioning.

Conclusions

Our data show that the facial expression of fear influences the early neural responses of face processing, regardless of whether attention is directed towards identity, expression or both dimensions. Fear elicited stronger stRS responses compared with any other facial expressions of emotion, an electrophysiological effect that was rooted in both the adaptor and the target face over the bilateral occipito-temporal cortex. Fear also modulated categorization efficiency for the most difficult dual-task. These findings suggest that the expression of fear triggers distinct mechanisms, by enhancing attention and leading to a better coding of facial information (i.e. greater adaptation). Such a unique role of fear echoes with the modern theory of threat detection, which highlights the important evolutionary significance of the communication of this expression.

Funding

This publication was supported by the Conseil de l'Université de Fribourg.

Conflict of interest. None declared.