EMOTIONAL MODULATION OF ATTENTION ORIENTING BY GAZE VARIES WITH DYNAMIC CUE SEQUENCE

Abstract

Recent gaze cueing studies using dynamic cue sequences have reported increased attention orienting by gaze with faces expressing fear, surprise or anger. Here, we investigated whether the type of dynamic cue sequence used impacted the magnitude of this effect. When the emotion was expressed before or concurrently with gaze shift, no modulation of gaze-oriented attention by emotion was seen. In contrast, when the face cue averted gaze before expressing an emotion (as if reacting to the object after first localizing it), the gaze orienting effect was clearly increased for fearful, surprised and angry faces compared to neutral faces. Thus, the type of dynamic sequence used, and in particular the order in which the gaze shift and the facial expression are presented, modulate gaze-oriented attention, with maximal modulation seen when the expression of emotion follows gaze shift.

1. INTRODUCTION

Accurate mental state attribution is necessary for successful social interactions and requires proper monitoring of facial cues such as eye gaze direction (Baron-Cohen, 1995). The direction of others’ gaze indicates their object of focus and a large body of work has now shown that gaze also directs the observer’s attention spontaneously in the direction of perceived gaze (Friesen & Kingstone, 1998; Frischen et al., 2007 for a review). This spontaneous attention orienting by gaze has been demonstrated using gaze cuing paradigms in which a face cue with averted gaze is presented at the center of a computer screen and is followed by the onset of a lateral target presented on the gazed-at side (congruent or valid trials) or opposite to the gazed-at side (incongruent or invalid trials). Participants are typically faster to respond to congruent than to incongruent targets, an effect known as the Gaze Orienting Effect (GOE) and thought to reflect the increased attention allocation to the gazed-at side.

Facial expressions are another important clue to others’ mental states, especially when combined with gaze direction. For instance, the facial emotion expressed by a gazing face reflects the gazer’s reaction to the object and provides additional information regarding the potential nature of that object (e.g. dangerous if the face is fearful, pleasant if the face is happy, etc…). Whether this additional emotional information triggers even faster attention orienting toward the gaze direction compared to when the face is neutral has been investigated and many previous studies have reported an enhancement of the GOE for fearful compared to neutral and/or happy faces, an effect generally interpreted as reflecting increased orienting to a potential threat (Graham, Kelland-Friesen, Fichtenholtz, & LaBar, 2010; Pecchinenda, Pes, Ferlazzo, & Zoccolotti, 2008 [although only for task-relevant emotions];Tipples, 2006; Putman, Hermans, & Van Honk, 2006; Bayless et al., 2011; Neath et al., 2013; Lassalle & Itier, 2013; Lassalle & Itier, 2015). Recent studies have also reported larger GOE for surprised faces compared to happy and angry faces (Bayless et al., 2011) and compared to neutral faces (Lassalle and Itier, 2013; Neath et al., 2013). In all cases the magnitude of that effect was as large as that seen for fearful faces. A larger GOE for angry faces compared to both happy and neutral faces was also reported (Lassalle & Itier, 2013; see also Holmes et al., 2006 but only in highly anxious participants). This suggests that the perceived negative valence of the emotion (as in the case of angry and fearful expressions) and possibly the signaling of uncertainty (as in the case of surprised expressions), rather than threat per se, might be driving these effects.

Modulation of the gaze orienting effect by facial emotion is not always reported, however (e.g. Fichtenholtz, Hopfinger, Graham, Detwiler, & LaBar, 2007, 2009; Galfano, Sarlo, Sassi, Munafo, Fuentes & Umilta, 2011; Hietanen & Leppänen, 2003; Holmes, Mogg, Monje Garcia, & Bradley, 2010; Bayliss, Frishen, Fenske & Tipper, 2007), and studies have started highlighting important modulators of these emotional effects. Trait anxiety has been shown to potentiate the increased GOE reported with fearful faces (Matthews et al., 2003; Holmes et al., 2010; Fox, Mathews, Calder, & Yiend, 2007; Putman et al., 2006; see also Tipples, 2006 with trait fearfulness) although the effect was also seen in the general, non-anxious population (Neath et al., 2013; Lassalle & Itier, 2013, 2015). Stimulus Onset Asynchrony (SOA), i.e., the time between gaze shift and target onset, has proven to be another important factor with one study reporting emotional modulation of the GOE only for SOAs greater than 300ms (Graham et al., 2010; but see Putman et al., 2006 and Bayless et al., 2011 for effects seen with 200ms SOAs). Finally, one of the most important factors is the use of a dynamic rather than a static cue sequence. A static cue generally consists in a face picture displaying an emotion with an averted gaze. In contrast, a dynamic cue typically involves several frames, i.e. several pictures of the same facial identity presented rapidly back to back so as to elicit the impression of a dynamic change from a neutral face with straight gaze to an emotional face with averted gaze. None of the studies that used static displays reported emotional modulations of the GOE (e.g. Hietanen & Leppänen, 2003 [Exps.1–4]; Holmes et al., 2006 [Exp.3]; Hori, Tazumi, Umeno, Kamachi, Kobayashi, Ono & Nishijo, 2005). However, amongst the studies that used a dynamic display, the sequence chosen for the dynamic face cues varied substantially across experiments. The parameters associated with a particular sequence might influence the way in which the emotion is processed, and, in some cases, facial expressions might not have been processed well enough to modulate gaze orienting.

Some studies have used dynamic stimuli sequences in which the emotion was expressed first, before gaze shift (Hietanen & Leppänen, 2003 [Exps. 5 and 6]; Mathews et al., 2003; Galfano et al., 2011). That is, an emotional face gazing straight ahead was followed by the same emotional face gazing to the side. Of these studies, only Matthews and colleagues (2003) showed an enhancement of the GOE with fearful faces, but this effect was restricted to highly anxious participants. Such null results could be due to a lack of ecological validity as in real life, most often, people react to a stimulus after, rather than before, localizing it. In addition, given that facial expressions remain constant throughout the stimulus presentation while gaze shifts abruptly, processing of gaze might be prioritized over emotion processing (Graham et al., 2010).

Other studies have adopted a dynamic stimulus display in which gaze and emotion were changed simultaneously such that a neutral face with direct gaze was immediately followed by the same identity expressing an emotion with averted gaze (Holmes et al., 2010; Tipples, 2006; Lassalle & Itier, 2013) or by a succession of frames representing the face gradually shifting its gaze and expressing an emotion (Bayless et al., 2011; Fichtenholtz et al., 2007; 2009; Putman et al., 2006; Uono et al., 2009). Some of these studies showed a modulation of the GOE with facial expressions (Lassalle & Itier, 2013; Tipples, 2006; Putman et al., 2006; Bayless et al., 2011, Uono et al., 2009) but others did not (Fichtenholtz et al., 2007; 2009; Holmes et al., 2010). Although more ecologically valid, this stimulus sequence confounds eye size with facial expression, as wide-open eyes are seen in fearful and surprised faces whereas squinted eyes are seen in angry and happy facial expressions (Tipples, 2005; Bayless et al., 2011). As gaze is more salient in large eyes than in squinted eyes (Tipples, 2006), gaze processing could be facilitated and trigger stronger attentional shifts in fearful and surprised faces compared to angry and happy faces. Thus eye size, rather than emotion per se, might be driving the reported modulations of the GOE with facial expression in that type of dynamic sequence.

The last type of dynamic facial cue used involves a sequence in which the gaze is averted first, before the emotional expression. Using such a sequence, an increased GOE with fearful faces was reported in both an unselected sample (Graham et al., 2010) and in non-anxious samples (Neath et al., 2013; Lassalle & Itier, 2015). This stimulus sequence makes the most sense in terms of ecological validity since one would need to foveate toward an environmental stimulus before reacting to it. In addition, this sequence allows the effect of the gaze shift to be independent from the eye aperture of the emotional expression as it occurs when the face is neutral, before the expression of emotion.

The extent to which differences in these dynamic cue sequences influence the modulation of gaze-oriented attention by facial expressions is currently unknown. It is possible that some of the null findings reported in the literature are driven, in part, by the type of sequence used. In the present study, which involved only non-anxious participants (to avoid any confound due to anxiety), we investigated whether the modulation of the GOE by various facial expressions (neutral, fearful, happy, angry, surprised) varied with the dynamic cue sequence used. We used three sequences, one in which the face cue expressed an emotion before averting its gaze, one in which the cue averted its gaze before expressing an emotion, and finally one in which both emotion and gaze changed concurrently. We predicted that the emotional modulation of the GOE would be largest when the emotional expression followed gaze shifts, due to the ecological nature of this sequence.

2. METHODS

2.1. Participants

In each of the three experimental sequences, 18 students from the University of Waterloo were included (Table 1), for a total of 54 participants. They all ranged from 18 to 28 years of age. Participants had no self-reported history of psychiatric or neurological illness, were all right-handed and had a corrected-to-normal vision. Initially, 74 participants were tested but 7 were discarded due to a technical error occurring during the experiment, 2 because they didn’t complete the experiment and 11 because of their high anxiety scores.

Table 1.

(a) Demographics of the participants for the three Sequences (F= female, M= male, SD into brackets) and (b) Statistics relative to demographic information

Measures	Sequence Emo-first	Sequence Emo-second	Sequence Emo-concurrent
N	18 [10F, 8M]	18 [9F, 9M]	18 [9F, 9M]
STICSA scores	30.83 (5.66)	30.39 (6.14)	30.50 (4.93)
Age (years)	21.6 (1.82)	21.7 (2.68)	19.80 (1.44)

Comparisons	Emo-first/Emo-second	Emo-second/Emo-concurrent	Emo-concurrent/Emo-first
STICSA	t= .44, p=1.00	t= .11, p=1.00	t= .33, p=1.00
Age	t= .11, p=1.00	t= 1.89, p=.02	t= 1.78, p=0.4

Indeed, only participants scoring below 42 on the trait anxiety scale (State-Trait Inventory for Cognitive and Somatic Anxiety [STICSA]: Gros, Antony, Simms, & McCabe, 2007) were included in the final analysis. This was in accordance with Van Dam and colleagues (2013) who suggested that a cut-off of 43 should be used in research settings to indicate probable cases of clinical anxiety (sensitivity=.73, specificity=.74, classification accuracy=.74). Mean anxiety scores and age did not differ between the 3 groups as measured by independent t-tests (Table 1). Participants were either paid $10 or received a course credit for their participation in the study. The experimental procedure was approved by the Ethics Research Board of the University of Waterloo and all participants gave written informed consent.

2.2. Stimuli and procedure

The stimuli used for this study consisted of 40 face pictures of eight individuals (4 females) expressing surprised, fearful, happy, angry and neutral expressions, taken from the MacBrain Face Stimulus Set (Tottenham, Hare, Millner, Gihooly, Zevin & Casey, 2009: models #02, 03, 06, 09, 20, 22, 24, 27)¹. Gaze was manipulated for each image to produce leftward and rightward gaze in addition to the original straight gaze (with an eye displacement of 20 pixels). An elliptical mask was applied to each picture so hair, ear and shoulders were not visible. The set of images was equated for contrast and pixel intensity (pixel intensity = 0.8021; RMS contrast = 0.3901 for each picture), using the SHINE toolbox (Willenbockel, Sadr, Fiset, Horne, Gosselin & Tanaka, 2010). Faces were centrally presented on a white background at a 67cm distance in a quiet, medium-lit and electrically shielded room. A chin rest and a head restraint kept viewing distance constant and minimized participants’ movements.

The study was programmed using Presentation software (Neurobehavioral Systems) and consisted of 5 blocks of 240 trials each. Within a block, each of the 8 identities was presented 30 times, twice in each of the 15 possible conditions (one emotion [fear, surprise, neutral, anger and happiness] and one congruency [congruent, incongruent, non-congruent]). The trials were randomized within a block. Across the 5 blocks there were thus 80 trials per condition (8×2×5 blocks). Each trial started with a fixation cross (1.28° by 1.28° of visual angle) presented randomly for 800, 900, 1000, 1100 or 1200ms at the center of the screen, subsequently replaced by a dynamic face sequence (8.02° by 12.35 ° of visual angle). The trial ended by the presentation of a black asterisk target (.85° by .85° of visual angle) on either side of the monitor, 7.68° from the center, which remained on the screen until the participant’s response or for a maximum of 500ms. In congruent trials, targets were presented at the gazed-at location while in incongruent trials, targets were presented on the side opposite to gaze direction (the same number of left and right targets were included in each congruent and incongruent condition). In non-congruent trials, the face gaze remained straight throughout the sequence.

The face sequence differed between the three experiments (Figure 1). In the “Emo-first” Sequence (emotion followed by gaze shift), a neutral face with direct gaze was presented for 200ms followed by the same face displaying one of the five possible expressions for 300ms. The emotional face then averted gaze to the left or right for 500ms. In the “Emo-second” Sequence (gaze shift followed by emotion expression), the initial neutral face with direct gaze was presented for 500ms before the onset of the gaze shift. The same neutral face with averted gaze was then presented for 200ms followed by the same gazing face now expressing an emotion for 300ms. In the “Emo-concurrent” Sequence (gaze shift and emotion occurring concurrently), the initial neutral face with direct gaze was also presented for 500ms, followed by the emotional face with averted gaze for 500ms. Thus, in all three conditions the stimulus onset asynchrony (SOA) between the onset of gaze shift and the appearance of the target was always 500ms (an SOA sufficient to reveal emotion modulations of the GOE, Graham et al., 2010) and the entire dynamic facial sequence (before target onset) was always 1000ms (Figure 1). Participants were instructed to respond to the appearance of the target as fast and as accurately as possible by pressing a right button for right targets and a left button for left targets (using both hands).

Figure 1.

Trial Sequence for a) Emo-First Sequence: facial expression followed by Gaze shift, b) Emo-Second Sequence: Gaze shit followed by facial expression, c) Emo-Concurrent Sequence: Gaze and facial expression concurrent. In all three sequences the SOA (between gaze shift onset and target onset) was 500ms.

2.3. Data analyses

Responses were recorded as correct if the response key matched the side on which the target appeared and if reaction times (RTs) were within 100–1200ms. The remaining responses (<100ms or >1200ms RT) were marked as incorrect (see Table 2 for the percentage of trials lost due to errors and timeouts). For each experiment, mean response times for correct answers were calculated according to facial emotions (happy, angry, neutral, fearful and surprised) and congruency (congruent, incongruent), with left and right targets averaged together. For each participant, only RTs within 2.5 standard deviations from the mean of each condition were retained for that participant’s mean RT calculation (Van Selst & Jolicoeur, 1994).

Table 2.

Mean percent of errors obtained in the Emo-first Sequence (a), Emo-second Sequence (b), and Emo-concurrent Sequence (c)

a) Sequence Emo-first	Happy	Surprised	Neutral	Angry	Fearful
Congruent	5.44	7.06	6.32	7.21	6.18
Incongruent	6.91	6.62	9.71	7.94	8.68
b) Sequence Emo-second	Happy	Surprised	Neutral	Angry	Fearful
Congruent	4.44	7.06	6.67	4.72	4.17
Incongruent	7.78	9.86	9.31	7.50	9.72
c) Sequence Emo-concurrent	Happy	Surprised	Neutral	Angry	Fearful
Congruent	3.75	2.64	5.97	4.86	3.33
Incongruent	5.42	4.31	5.56	5.42	7.08

Statistical analyses all employed repeated measures ANOVA (SPSS 22) that are detailed in the result section. In all analyses the Greenhouse-Geisser degrees of freedom correction was used when the sphericity assumption was violated. Paired comparisons were Bonferroni corrected.

3. RESULTS

Faces with direct gaze are known to capture attention more than faces with averted gaze (e.g., Senju & Jonhson, 2005), especially when the face expresses fear (Fox et al., 2007; Georgiou et al., 2005; Mathews et al. 2003). In addition, direct gaze activates different brain regions than averted gaze (George, Driver, & Dolan, 2001). Those findings have been taken to suggest that the analysis of direct and averted gaze is governed by different mechanisms (see George & Conty, 2008 for a review). In light of this possibility and given the focus of the present study on the GOE, direct gaze trials will not be discussed further. Nonetheless, ANOVA of direct gaze are included in the Appendix for researchers interested in responses to direct gaze.

We first ran an omnibus mixed model ANOVA with Congruency (2: congruent or incongruent) and Emotion (5) as within-subject factors and Sequence (3) as between-subject factors. The main effect of Congruency (F (1.77, 90.37) =135.74, MSE=253.31, p<.01, η_p²=.73) was due to faster RTs for congruent than incongruent trials in all conditions, reflecting the classic gaze orienting effect (Figure 2). Importantly, the Sequence by Emotion by Congruency interaction was significant (F (11.94, 304.42) =3.95, MSE=64.53, p<.01, η_p²=.13), which justified focusing on the GOE (mean RT for incongruent trials minus mean RT for congruent trials), using a 3(Sequence) by 5 (Emotion) mixed ANOVA.

Figure 2.

RTs (ms) to targets following happy, surprised, neutral, angry, and fearful faces in congruent and incongruent trials for the three cue sequences.

The GOE showed a main effect of Sequence (F (1, 2) =6.97, MSE=563.46, p<.01, η_p²=.22) and was larger in Sequence Emo-second than in Sequences Emo-first and Emo-concurrent (p<.01 for both comparisons, Figure 3) which did not differ. This effect of Sequence was due to the added difference of the congruent and the incongruent trials as neither congruent trials, nor incongruent trials exhibited an effect of Sequence when analyzed separately (p= .86 and p= .55 respectively).

Figure 3.

GOE (ms) for happy, surprised, neutral, angry and fearful facial expressions for the three cue sequences.

In addition the GOE showed a Sequence by Emotion interaction (F (8,204) =3.76, MSE=36.56, p< .01, η_p²= .13) and was thus analyzed for each Sequence separately. This analysis revealed a main effect of Emotion on the GOE only for Sequence Emo-second (F (4, 68) = 7.59, MSE=72.74, p< .01, η_p²=.31) but not for Sequences Emo-first or Emo-concurrent (F (4, 68) =1.61, MSE=95.32, p=.18, η_p²=.09 and F (4, 68) = 2.03, MSE=51.29, p=.10, η_p²=.11, respectively). The effect of emotion on the GOE for Sequence Emo-second was due to a larger GOE for angry, fearful and surprised expressions (which did not differ) compared to neutral expressions (p<.05 for all Bonferroni corrected comparisons). The GOE for happy faces was intermediate, but not significantly different from the GOE in any other condition (Figure 3). In that Emo-second sequence, these effects of emotion were due to the additive effects of congruent and incongruent trials as an effect of emotion was found for both trial types (congruent: F (4, 68) =32.20, MSE=66.24, p<.01, η_p²=.65; incongruent: F (4, 68) =13.34, MSE=47.13, p<.01, η_p²=.44).

4. DISCUSSION

In this experiment, we aimed at determining the extent to which differences in dynamic cue sequence influence the modulation of gaze-oriented attention by facial expressions. We compared dynamic cue sequences in which the facial expression of emotion preceded (Emo-first Sequence), followed (Emo-second Sequence) or occurred concurrently with the gaze shift (Emo-concurrent Sequence). A classic gaze orienting effect was found in all three sequences. As predicted, this effect was largest in the Emo-second sequence and reflected the combined effects of a faster orienting of attention to congruent targets and a slower disengagement of attention from incongruent target locations. In addition, the Emo-second sequence was the only cue sequence that yielded an enhancement of the GOE with angry, fearful and surprised facial expressions compared to neutral expressions. No emotion modulation of the GOE was seen in the other two sequences. Although less feature displacement occurred in neutral face stimuli (in which only the gaze direction changed) than in emotional face stimuli, this was the case for all three sequences. The diminished facial displacement of neutral faces thus cannot explain the emotional modulation of the GOE in only one sequence. The type of dynamic cue used impacts the gaze orienting effect and its modulation by emotion. The expression of emotion following gaze shift is the most ecological sequence as in real settings one tends to react to a stimulus after localizing it and we believe that this ecological validity is the primary reason for the effects we found. In accordance with those results, all the studies that used a similar dynamic sequence in which the face averted gaze before expressing an emotion yielded a modulation of the GOE with emotions (Graham et al., 2010; Neath et al., 2013; Lassalle & Itier, 2015) while those that used Emo-first or Emo-concurrent types of sequences yielded mixed results.

The eye size has also been suggested to contribute to the modulation of the GOE by facial expressions to some extent (Bayless et al., 2011; Tipples, 2006). Fearful and surprised facial expressions have characteristic widen eyes while angry or happy expressions have squinted eyes. The aperture of the eyes, rather than the facial expression of emotion per se, might thus facilitate the processing of gaze shift and yield the increases in GOE with fearful and surprised faces reported previously. If this was the case, we would expect the GOE modulations with emotions to be largest in sequences in which the gaze shift was confounded with the eye aperture associated with the emotion, i.e. in the sequence where the emotion was expressed before gaze shift (Emo-first sequence) or simultaneously with gaze shift (Emo-concurrent sequence). On the contrary, we found that the GOE was modulated by emotion only in Sequence Emo-second where the gaze shift always occurred in a neutral face and thus was not confounded by eye aperture. Our results suggest that eye aperture is not a critical factor in the modulation of gaze-oriented attention by emotional faces (see also Bayless et al., 2011).

The lack of GOE modulation by emotion when emotion was expressed first (Emo-first) is consistent with the null findings reported by most studies using similar designs (Hietanen & Leppänen, 2003[Exps. 5 and 6]; Mathews et al., 2003; Galfano et al., 2011). Of those studies only Mathews and colleagues (2003) found a GOE enhancement with fear but this effect was restricted to highly anxious participants and might thus have been driven solely by anxiety. The lack of GOE emotional modulation when the face cue expressed an emotion and shifted its gaze simultaneously (Emo-concurrent) is in line with some studies (Fichtenholtz et al., 2007; 2009; Holmes et al., 2010) but not others (Lassalle & Itier, 2013; Tipples, 2006; Putman et al., 2006; Bayless et al., 2011, Uono et al., 2009). Differences in experimental parameters other than the cue sequence might explain this discrepancy. For instance, the study by Lassalle and Itier (2013) differs from the present study in that it included only three emotions in the design (fear, surprise and neutral -Exp1- or happy, angry and neutral –Exp2) while five emotions were included in the present study (fear, surprise, neutral, happy, angry). This difference in design might have an impact, with the “affective context” in which an emotion is presented possibly influencing its ability to impact gaze-oriented attention. Fear and surprise could have been perceived as threatening in Lassalle and Itier (2013) because they were presented concurrently with neutral faces only, not with happy faces, which could have led to an increased saliency of threat-related emotions and, in turn, to a GOE enhancement for those emotions. In contrast, in the Emo-concurrent sequence of the present study the saliency of threat-related emotions might have been clouded by the presence of a positive emotion. The other studies that used Emo-concurrent type of sequences yet reported an enhancement of the GOE with fearful faces relative to neutral faces (Putman et al. 2006; Tipples et al., 2006) included highly anxious participants and showed that anxiety (and fearfulness traits in the case of Tipples, 2006) potentiates gaze-oriented attention with fearful faces (also see Mathews et al., 2003; Fox et al., 2007), making it unclear whether the sequence itself, rather than anxiety alone, contributed to these positive results.

Our findings are important for future gaze cuing experiments as they suggest that the use of the more ecologically valid cue sequence where facial expression follows gaze shift is the best to reveal modulations of the GOE by emotion (see Risko et al., 2012 for a discussion on the importance of ecological validity to study social attention). However, while we attribute our findings to the difference in order between gaze shift and emotion expression, other experimental parameters varied between the sequences and might also have played a role in the present results. For instance, despite a constant SOA of 500ms (between gaze shift and target onset) and a constant gaze shift onset of 500ms (after face onset), the expression onset varied across sequences (it occurred 200ms after face onset in the Emo-first sequence, 500ms in the Emo-concurrent sequence and 700ms after face onset in the Emo-second sequence). In addition, the emotion was expressed for various durations in the three sequences: for 800ms in Emo-first sequence, for 300ms in Emo-second sequence and for 500ms in Emo-concurrent sequence. The fact that emotion effects were seen in the sequence in which facial expression duration was the shortest suggests emotion duration was sufficient in that Emo-second sequence. Finally, the time between the last dynamic change in the stimulus sequence and the target onset also varied between sequences (500ms for Emo-first and Emo-concurrent sequences but 300ms for Emo-second sequence). This last change in the stimulus sequence could have acted as a warning signal and facilitated responses to Emo-Second sequence relative to the other two sequences. Whether each of these parameters impacts the GOE will have to be determined by future studies.

The present study confirmed that using a certain type of dynamic cue sequence (one where emotion follows gaze shift) can reveal emotional modulations of the GOE in non-anxious participants (see also Lassalle and Itier, 2015; Neath et al., 2013). Results confirmed that both fearful and surprised expressions elicited larger GOE than neutral expressions. Importantly, the present study also found that angry facial expressions increased the GOE to the same extent as fearful and surprised faces, suggesting that negative valence and possibly uncertainty, rather than threat alone, can potentiate attention orienting. Using the same sequence and design, Neath et al. (2013) reported larger GOE for fearful than angry, happy and neutral faces while Lassalle and Itier (2013) reported larger GOE with angry than neutral and happy faces in a design that included only these three emotions (Exp.2). Lassalle and Itier (2013) speculated that the GOE with angry faces could be attenuated when fearful faces were included in the design because the threat they indicate is indirect compared to fearful faces (an angry person looking to the side is not a direct threat for the observer, only for the target of the angry emotion, while a fearful face with averted gaze indicates a threat in the environment for anyone, including the observer). The results of the present study argue against this possibility. It is at present unclear why the GOE was potentiated for angry faces in the present study but not in Neath et al. (2013), although sample differences including sample size, might be at play.

Interestingly, the GOE for emotions carrying a negative valence and signaling a threat or uncertainty was not significantly greater than the GOE for happy faces in this study although in the right direction. This lack of statistical difference could be due to a lack of power in the present study with a fairly small number of participants in each experiment. This lack of statistical difference could also be due to the relatively high social abilities of participants in the present sample. Indeed, Lassalle and Itier (2015) recently found that contrary to participants with high autistic traits, participants low on autistic traits orient as rapidly with happy as with fearful faces. The authors suggested that this might be because the gaze of smiling faces could indicate a social interaction and social individuals (with low autistic traits) are strongly motivated to attend to social interactions. In the present study, the social ability of participants was not assessed but could have been relatively high (given that participants were Arts students, who tend to have high social skills [Baron-Cohen et al., 2001]), which would explain the lack of statistical difference between the negative emotions and happy facial expressions. Future gaze-cueing studies will need to include an assessment of participants’ autistic traits to clarify whether it could play a role in the observed effects and should aim at replicating the current results with larger samples.

5. CONCLUSION

The type of dynamic cue sequence impacts gaze-oriented attention and its modulation by facial expression. The GOE was largest when the emotion was expressed after gaze shift, as if the individual was reacting to what they had just seen, and that sequence also yielded clearly larger GOE for facial expressions that were negatively valenced or expressed uncertainty compared to neutral faces. Small GOEs that were not modulated by emotion were found when the facial expression was expressed before or concurrently with gaze shift. The type of dynamic cue sequence might have contributed to the lack of emotional modulation of the GOE reported in some previous studies. The present study highlights the importance of using more ecological displays to study social attention (see Risko et al., 2012).

APPENDIX

Direct gaze trials were analyzed using a mixed ANOVA with Sequence (3) as a between-subject factor and Emotion (5) as a within-subject factor. There was a main effect of Sequence (F(1, 2) =3.85, MSE=4479.44, p= .03, η_p²=.13) which was due to overall slower responses in Sequence Emo-second than in the other two sequences (Sequence Emo-second and Sequence Emo-first paired comparison significant). There was also a main effect of Emotion (F(4, 204) =117.61, MSE=80.25, p< .01, η_p²=.70) which was qualified by an Emotion by Sequence interaction (F(4.68, 119.36) =5.61, MSE=138.56, p< .01, η_p²=.18). Although each Sequence analyzed separately displayed a main effect of Emotion (Sequence 1: F(4, 68)=18.04, MSE=60.52, p<.01, η_p²=.52; Sequence 2: F(4,68)=49.41, MSE=108.06, p<.01, η_p²= .74; Sequence 3: F(4, 68)=54.03, MSE=72.16, p<.01, η_p²= .76), such that targets preceded by emotional faces were responded to faster than targets preceded by neutral faces (p<.01 for each emotion in each Sequence), this effect of Emotion was smallest in Sequence Emo-first.