Amygdala processing of social cues from faces: an intracrebral EEG study

Abstract

The amygdala is a key structure for monitoring the relevance of environmental stimuli. Yet, little is known about the dynamics of its response to primary social cues such as gaze and emotion. Here, we examined evoked amygdala responses to gaze and facial emotion changes in five epileptic patients with intracerebral electrodes. Patients first viewed a neutral face that would then convey social cues: it turned either happy or fearful with or without gaze aversion. This social cue was followed by a laterally presented target, the detection of which was faster if it appeared in a location congruent with the averted gaze direction. First, we observed pronounced evoked amygdala potentials to the initial neutral face. Second, analysis of the evoked responses to the cue showed an early effect of gaze starting at 123 ms in the right amygdala. Differential effects of fearful vs happy valence were individually present but more variable in time and therefore not observed at group-level. Our study is the first to demonstrate such an early effect of gaze in the amygdala, in line with its particular behavioral relevance in the spatial attention task.

Introduction

Gaze and emotional facial expression are social cues of particular behavioral relevance. They indicate the focus of interest and mental states of others and may signal events of particular relevance in the environment (see for a review Graham and Labar, 2012). Consequently, the failure to process these cues has been associated with various behavioral and social disorders in recent literature (Kliemann et al., 2012; Comparelli et al., 2013; Gilboa-Schechtman and Shachar-Lavie, 2013; Tye et al., 2013; Zalla and Sperduti, 2013). The amygdala has proven to be a key structure in this domain (Adolphs and Spezio, 2006; Gobbini and Haxby, 2007). Initially perceived as a ‘fear module’ to react to aversive stimuli, its implication in a variety of tasks pertaining to social cognition, reward learning and spatial attention has led to a much more general view of the amygdala as a ‘relevance detector’, which constantly evaluates and disambiguates environmental stimuli (Sander et al., 2003; Whalen, 2007; Adolphs, 2010). A wealth of studies in the social field is congruent with this view. As part of the network of face sensitive areas (Hoffman and Haxby, 2000; Ishai et al., 2005), amygdala activation to neutral faces has been shown both in intracranial (Krolak-Salmon et al., 2004) and fMRI studies (Pessoa et al., 2002; Ishai et al., 2005). Emotional expressions reliably elicit amygdala activation and the ‘fear module’ has been shown to react essentially to all types of emotion (Winston et al., 2003). Furthermore, it is activated by dynamic facial changes (Sato et al., 2011) and it is also involved in the processing of gaze direction (Young et al., 1995; Kawashima et al., 1999; George et al., 2001), particularly in conjunction with emotional expression (Adams et al., 2003; Sato et al., 2004; Hadjikhani et al., 2008; N’Diaye et al., 2009; Cristinzio et al., 2010). However, direct evidence for the time course of amygdala responses to relevant stimuli, particularly social cues, is scarce and little is known on the dynamics of amygdala responses to gaze and emotional expression.

Intracranial EEG (iEEG) recordings in the human brain have excellent time resolution and provide a unique opportunity to observe neuronal responses directly from deep structures of the brain, such as amygdala. iEEG is performed in epilepsy patients eligible for surgery to determine the epileptic focus (Lachaux et al., 2003). After the seminal work of Halgren et al. (1994), few intracranial studies to date have examined the amygdala responses in relation to emotion or gaze. Krolak-Salmon et al. (2004) studied the amygdala response to emotional faces in four epileptic patients. Two out of the four patients with electrodes implanted in the amygdala showed significant selective responses to fearful faces arising from about 200 ms post-stimulus, when these patients paid attention to emotional expression. The dependency of amygdala responses on explicit emotion processing in this study contrasts starkly with the findings of Pourtois et al. (2010), who documented differentiated amygdala responses to fearful vs neutral faces starting around 140 ms post-stimulus, even in the absence of explicit emotion processing. Context-dependent amygdala processing thus warrants further scrutiny.

Even less is known on amygdala responses to gaze. Robust data point to the fact that the eye region is of particular salience. Yet, only two studies have examined the amygdala responses to seen eyes or to gaze from intracerebral recordings. Meletti et al. (2012) showed that the amygdala is especially sensitive to the eye region of a face when processing facial signals. They observed a preferential response of the amygdala to fearful eyes between 200 and 400 ms post-stimulus onset, with later effects of eyes with positive valence. Only one amygdala depth electrode study tried to address specifically the question of gaze. Sato et al. (2011) examined the amygdala response to eyes and mosaics pointing in averted and straight directions. They found increased gamma band oscillations in the amygdala for straight and averted eyes in comparison to mosaics, with a peak of activity at 200 ms. Only a trend for a change in gamma band activity to gaze aversion was found at around 285 ms (Sato et al., 2011). While there is abundant literature on amygdala response to the face onset, the paradigm of Sato and colleagues had also the merit of introducing a dynamic change in the stimulus, which should be closer to real-world experience (Larsen and Bundesen, 2009; Ulloa et al., 2014).

Here, we intended to study the amygdala response to changes in gaze direction and emotional expression of seen faces. We purposefully separated face onset and changes in social cues in time so as to dissect the amygdala response to the face onset from the response to the dynamic social cues. We used a sequential paradigm where a neutral face with direct gaze turned happy or fearful, with or without gaze aversion. Perception of gaze and emotion preceded a target detection task which allowed measuring the behavioral impact of gaze in the form of the well-described gaze cueing effect, i.e. faster reaction time to a target which appears on the side looked at by a cue face than to a target presented on the opposite (not looked at) side (Friesen and Kingstone, 1998; Driver et al., 1999; for review see Frischen et al., 2007).

We expected a first evoked potential to the neutral face, equivalent to an N200 (Allison et al., 1999) in line with the view of amygdala as a face sensitive area, followed by a differentiated amygdala response to the social cue. We anticipated this second response to be early (∼200–400 ms, Sato et al., 2011), in agreement with the amygdala role as a ‘relevance detector’, capable to update facial information in this socially meaningful sequence. As emotion was always present and does not entail intrinsic directional information, we hypothesized that gaze would have precedence over emotional valence in this spatial attention paradigm

Materials and methods

Patients

Five patients with electrodes exploring the amygdala were enrolled in this study. These patients represented a subgroup of a larger study of 17 patients who all underwent invasive intracerebral recordings as part of their clinical pre-surgical evaluation with various brain regions explored. All patients gave informed consent to take part in the experiment. The study was approved by the local ethics committee (CPP Ile-de-France VI).

Of the 17 patients recorded, eight patients had contacts in the amygdala. Three of these were excluded because of a technical problem (one patient) or constant epileptic activity and drowsiness (two patients). Therefore, amygdala recordings of the five remaining patients were analyzed (three females; mean age = 38 years, range 25–48 years; see Table 1 and Supplementary Methods for clinical data). None of the patients had an amygdala lesion or seizures originating from the amygdala.

Table 1.

Clinical characteristics of the included patients

Patient	Sex	Age	Handedness	Epilepsy onset	Medication (N AED)	Epileptogenic zone	MRI lesion	STAI
1	M	25	L	10	3	Bi-temporal	Possible L hippocampal atrophy	36
8	F	28	R	24	2	R temporo-polar or temporo-mesial	None	40
9	F	47	L	38	2	Bi-temporal with R predom	R hippocampal sclerosis	41
14	M	48	R	22	2	Bi-temporo-lateral	None	38
16	F	44	R	19	2	R temporal	R temporo-basal lesion	35

Patients had variable profiles of epilepsy. Two patients were left handed (L) according to medical history. The age of epilepsy onset is given in years. N AED: number of antiepileptic drugs during recordings. The epileptogenic zone was determined by intracerebral recordings, none of the patients had seizure onset in the amygdala, three patients had bi-temporal seizures, mostly with right (R) predominance. STAI = State-Trait Anxiety Inventory (Y version) score

Stimuli and experimental protocol

We used the same stimuli and modified Posner attention cueing paradigm as in a prior MEG study (Lachat et al., 2012). The paradigm is specified in Figure 1 and in Supplementary Methods. In brief, the face stimuli comprised 16 different faces, each with direct and averted gaze, under happy, fearful and neutral expressions. The stimuli were prepared with maximal care to control for the eye movement under the gaze change condition across emotions (see Supplementary Methods for details and analysis of apparent motion across the stimuli).

Fig. 1.

Experimental paradigm. A trial consisted of a central fixation cross followed by a neutral face with direct gaze. After a variable time interval of 400–600 ms, the same face would turn happy or fearful, with or without gaze aversion. Then, after a variable cue-to-target interval (SOA: 300–450 ms), a black and white checkerboard target appeared either on the left or on the right side of the screen.

Each trial consisted in a central fixation cross followed by a neutral face with direct gaze, which turned happy or fearful, with or without gaze aversion, after a variable delay (except for patient 1 for whom this delay was constant). Then, after a variable cue-to-target interval, a target appeared either on the right or on the left side of the screen. The subject’s task was to maintain central fixation and to press a response key as soon as possible in response to the target. The recording session comprised a total of 768 trials intermixed with 96 catch trials (in which no target was presented), distributed into eight blocks.

The stimuli were projected by means of a classical cathode screen, placed at 60-cm distance from the patient. All patients performed a debriefing and the State-Trait Anxiety Inventory (STAI; Form Y, Self Evaluation Questionnaire—Spielberger et al., 1983) at the end of the session. This test revealed anxiety scores within the normal range (viz. 20–80, mean 36 ± 10) (Bishop et al., 2004) for the five subjects: 38 ± 2.5 (see Table 1).

Intracranial recordings

Patients were implanted with stereotactic depth electrodes composed of 4–10 contacts each (AD-Tech, electrode width: 2.4 mm, inter-electrode spacing: 10 mm in patients 1, 8, 9, 14, and 5 mm in patient 16). Data were acquired with a Nicolet 6000 (Nicolet-Viasys, Madison, WI, USA) at a sampling rate of 400 Hz (band-pass: 0.05–150 Hz) for the first patient and with a Micromed System 3 Plus (Micromed S.p.A., Mogliano Veneto, Italy) at a sampling rate of 1024 Hz (bandpass: 0.16–330 Hz) for the other four patients. The reference electrode was located between Fz and Cz and a ground electrode was placed onto the chest.

Intracranial electrode localization

Electrode positions were determined by a post-implantation MRI scan (see Supplementary Methods) that was normalized to define the MNI coordinates of each of the electrode contacts using SPM 8 (http://www.fil.ion.ucl.ac.uk/spm/software/spm8/). Electrodes of interest in the amygdala were selected based on visual inspection of the normalized post-implantation MRI scan as illustrated in Figure 2. In two patients, only the right amygdala was explored, in one patient only the left, and two patients were implanted bilaterally (see Table 2).

Fig. 2.

Evoked potentials to the presentation of the neutral face. All patients generated discernable ERPs to the onset of the neutral face, equivalent to an N200. The shaded trace represents the standard error of the mean ERP. The normalized post-implantation MRI scan of each patient is shown, illustrating the exact localization of the first and second amygdala contacts. The ERPs of the innermost contact are shown in the upper panel (solid black arrows), the ERPs of the second contact in the lower panel (dotted black arrow). Note that for the first patient recorded (upper left panel), the SOA between the neutral face and the emotional face gaze cue was fixed at 500 ms. For this patient, a clearly discernable second ERP was observed in response to the emotional face (indicated by the ‘*’). All other patients were recorded with a randomly varying SOA of 400–600 ms. In the lower left part of the figure, the inset shows the grand mean of the ERPs to the neutral face, obtained by averaging all good trials across patients, on the inner and outer contacts respectively (pooling together the right and left contacts). This allows visualizing very clearly the selective neural response to the face observed around 200 ms on the inner contacts as compared with the outer ones.

Table 2.

MNI coordinates of the contacts in left and right amygdala

Right amygdala					Left amygdala
Patient	Contact	x	y	z	Patient	Contact	x	y	z
1	1	34	3	−28	1	1	−33	−1	−28
	2	41	−3	−28		2	−43	−4	−27
8	1	21	−5	−35	9	1	−26	0	−25
	2	34	−5	−34		2	−37	1	−25
9	1	21	−1	−25	16	1	−20	−4	−32
	2	32	−1	−25		2	−25	−4	−29
14	1	30	−5	−25
	2	40	−4	−21

Patients 1 and 9 had bilateral amygdala contacts. Contact 1 is the innermost contact used to calculate monopolar evoked potentials for every patient. All coordinates were determined on post-implantation MRI scans (see ‘Methods’ section). For an axial view of electrode contacts in each patient, see Figure 2

Event-related potential averaging

First, raw data were visually analyzed for artifacts and epileptic activity using a home-made visualization software (http://wiki.cenir.org/docu.php/muse). We formulated three artifact rejection criteria. First, epochs with activity exceeding ± 200 μV between the fixation cross and button press were rejected. Second, epochs with activity <200 μV in absolute value were rejected when this activity represented specific epileptic activity (e.g. sharp waves and sharp slow waves) or when this activity was synchronized with epileptic activity on an adjacent electrode. Finally, when >50% of trials in a block was rejected, this block was excluded from analysis. This resulted in a mean of more than 100 trials averaged per condition (see Supplementary Table S1 for detailed trial numbers in every patient). Data were low-pass filtered using a 40-Hz cut off. Event-related potentials (ERPs) were averaged in two time windows. First, we averaged the iEEG data in response to the initial neutral face onset. Second, we averaged the data in response to the face change, separately for the four conditions of interest (happy/fearful face cue with gaze aversion or no gaze change) (Figure 1). In both cases, the data were averaged between 100 ms before and 800 ms after the stimulus-of-interest onset, and the same 100-ms period of the fixation cross preceding the neutral face onset was used for baseline correction.

Statistics

Behavioral analysis

After omission of responses prior to target presentation and responses during catch trials, we computed the percentage of detected targets for the five patients. On average, the patients showed 99.1% (SEM = 0.38) of correct answers, with a very low percentage of false alarms during the catch trials (n = 0–3 out of 96) and few anticipations (responses given prior to target appearance or within 150 ms after its appearance, n = 0–11 out of 768). So, all patients detected the target at ceiling performance.

To test whether we found a gaze cueing effect, only trials with gaze aversion were used for the analysis of reaction times. We defined valid and invalid trials as those trials where the target appeared on the gazed-at side of the screen or on the opposite side, respectively. We ran an individual Wilcoxon signed-rank tests contrasting valid vs invalid trials to confirm this effect.

ERP analyses

We first conducted statistical analyses on the neural response to the initial neutral face. For this, we pooled together all the single trials obtained across subjects on the inner and outer contacts, respectively (pooling together right and left contacts) and we performed two types of tests: (i) we computed a t-test against 0 of the amygdala response obtained across trials on the inner and outer contacts, respectively; (ii) we directly contrasted the EEG signals recorded on the inner and outer contacts, using an unpaired t-test across trials. In both analyses, we corrected for multiple comparisons by using a data-driven cluster-based statistical approach (see below).

We then focused on the amygdala responses to the cue stimuli on the innermost electrode contacts. We performed a group analysis by pooling together the trials from the five patients separately for the left and right amygdala. We first ran a 2-by-2 ANOVA with emotion (happy/fearful) and gaze change (aversion/no gaze change) as between-trial factors in right and left amygdala contacts, respectively, and second, a 2-by-2-by-2 ANOVA including hemisphere (right/left amygdala) as an additional between-subject factor. We applied a data-driven cluster-based statistical approach to correct for multiple comparisons. This approach is derived from Maris and Oostenveld (2007) approach, extended to between-trials t-tests and ANOVAs. It allows correcting for multiple comparisons at the expense of sensitivity, particularly in the case of multiple clusters (Maris and Oostenveld, 2007). The amplitude computed for each time point across the retained trials was used as the dependent variable for all statistical comparisons. For every main and interaction effect, the samples for which the F-values corresponded to a P-value < 0.05 were clustered based on time adjacency. We then took the F-max as the cluster statistic. This procedure was repeated on 1000 permutations where the condition labels were randomly shuffled across trials in order to determine the distribution of the F-max under the null hypothesis, separately for every effect. We used these distributions to determine the Monte-Carlo P-value of the clusters identified in the original group data.

Moreover, for each patient, additional ANOVAs were performed to evaluate the effects of gaze and emotion at the individual level; these analyses are described in Supplementary Methods.

Results

Behavioral results

Each patient showed faster reaction times to validly than invalidly cued targets (Wilcoxon signed-rank test: P < 0.05; see Table 3), with a more reliable effect following fearful gaze cues (five out of five patients showing the effect) than following the happy gaze cues (three out five patients showing the effect). Supplementary analysis was done on the group of 17 patients, confirming a consistent gaze cueing effect, notably for the fearful faces (see Supplementary Results).

Table 3.

Reaction times for target detection

	Valid (ms)	Invalid (ms)	Direct gaze (ms)	Gaze cueing effect (ms)
Fearful	442 (55)	464 (59)	460 (54)	22 (9)
Happy	447 (57)	450 (56)	452 (58)	3 (3)
Mean	444 (56)	457 (57)	456 (56)	13 (5)

Grand mean reaction time (and standard error of the mean) is given for each emotion and each cue-target validity condition (i.e. valid, invalid and direct gaze—uncued—condition), for the group of five patients. The resulting gaze cueing effect (reaction time difference between invalid and valid trials) is indicated in the right column

Evoked responses to the presentation of the initial neutral face

As a first step, we averaged the amygdala response to the onset of the initial neutral face. We observed a clear neural response to the neutral face around 200–400 ms in all selected amygdala contacts, particularly marked on inner electrode contacts (Figure 2). All patients therefore generated face-related evoked potentials in the amygdala. Statistical analyses at group level confirmed that there was a significant negative evoked response—reflecting the simple main effect evoked by the neutral faces—between 192 and 325 ms on the inner contacts (Monte-Carlo P value = 0). No such effect was observed for the outer contacts and the direct comparison of inner and outer contacts showed a significantly greater negative evoked potential on the inner contacts between 126 and 366 ms (Monte-Carlo P value = 0). This is evidence that the response on the inner contacts originated from the amygdala and did not reflect a mere propagation of potentials generated in other face sensitive areas. We therefore decided to focus further analyses on the innermost contacts. Thus, we analyzed three contacts in the left and four contacts in the right amygdala.

Group analysis of the amygdala response to the social cue

We then analyzed the amygdala response to the social cue, i.e. to the emotional expression change with or without gaze aversion. We concatenated all good trials from the inner electrode of every patient in order to perform a between-trial ANOVA, in the right and left amygdala, respectively. The between-trial ANOVA revealed two clusters that yielded a gaze effect with less negative ERP to the gaze aversion than the no gaze change condition. The first cluster was observed between 123 and 258 ms in the right amygdala (see Figure 3); it resisted the cluster-based, permutation approach with a significant Monte-Carlo P value of 0.04. The second cluster was observed between 268 and 299 ms in the left amygdala; it was too short-lived to reach significance in the cluster-based analysis (P = 0.12). No other effect reached significance.

Fig. 3.

Amygdala responses to emotional expression and gaze changes. Illustration of group amygdala responses to the presentation of the social cue, i.e. to the emotional expression change with or without gaze aversion. The four main conditions are given in red and blue, solid or dashed lines as indicated in the legend. The effect of gaze is represented by the horizontal green bars below the neural time course. Only the right amygdala cluster resisted the cluster-based permutation that allowed correcting for multiple comparisons.

In order to compare the gaze effect between the left and the right amygdala, we included hemisphere as a between-subject factor in an additional analysis. This analysis confirmed the early main effect of gaze, which was significant between 124 and 321 ms (Monte-Carlo P value = 0.006). The interaction between hemisphere and gaze did not reach significance.

Recent studies in non-human primates reported a spatial representation of amygdala responses, with the contralateral amygdala being more responsive for lateral attentional shifts (e.g. Peck et al., 2013). We therefore separated the right and left averted gaze trials, and tested the amygdala responses to contra- vs ipsilateral gaze shifts. We found no effect of the laterality of gaze aversion in either the right or the left amygdala.

Individual amygdala responses to the social cue

Individual analysis complemented the group analysis to track down individual differences in the timing of amygdala responses to gaze and emotion changes. The ANOVAs performed in each patient’s innermost amygdala contact showed a significant early effect of gaze in two out of the four right amygdala contacts and in the left amygdala of patient 16 (see Supplementary Results).

In the individual analyses, we also found emotion effects but more variable in time and restricted to the right amygdala. They were generally observed later than the gaze effect, starting between 250 and 430 ms in three patients (see Supplementary Results).

Discussion

Our study aimed at examining amygdala responses to dynamic social cues from faces. To this aim, we recorded local field potentials from the amygdala in response to neutral faces that turned happy or fearful, with or without aversion of gaze. These social cues were integrated in a spatial attention paradigm, where patients had to detect the appearance of a simple checkerboard target presented laterally. At the behavioral level, we found a robust gaze cueing effect with faster detection of targets validly rather than invalidly cued by gaze. This supported the idea that the dynamic cues were reliably processed although they were incidental to the task, with gaze bearing a particular relevance in our spatial attention paradigm. At the electrophysiological level, we found a clear evoked potential to the initial neutral face which was followed by a robust effect of gaze, starting as early as 123 ms after the cue in the right amygdala. Emotional valence seemed to promote the behavioral impact of gaze and resulted in differentiated amygdala processing but more variable in time and observed only at the individual level.

The behavioral gaze cueing effect

In this study, the presentation of gaze and emotion was integrated in a spatial orientation task that lent particular behavioral relevance to the changes in the eye region and the direction of gaze. Gaze represents a spatial cue that is powerful at orienting the beholder’s attention toward the looked-at side; it facilitates target detection on the side that is looked at by the cue face (relative to the opposite side), the so-called gaze cueing effect (Friesen and Kingstone, 1998; Driver et al., 1999; Tipples, 2006; Frischen et al., 2007). Accordingly, even if gaze was not predictive of target location in our paradigm, we found a robust behavioral effect of gaze cueing in our patients, as has been classically demonstrated in previous studies on healthy subjects. Conversely, this effect has been shown to be absent in a group of temporal lobectomy patients (Akiyama et al., 2007; Okada et al., 2008), which underlines the importance of intact medial temporal structures in the reflexive attention orienting to eye gaze. The presence of the gaze cueing effect in our patients is compatible with a relatively preserved amygdala function.

In sum, our paradigm was suited to accentuate the behavioral importance of gaze and thus its salience in this specific context. The electrophysiological results of amygdala activity to the cue face will parallel the relevance of gaze in this setting.

Intracranial amygdala responses to the neutral face

The amygdala response to the initial neutral face consisted of a negative, sometimes biphasic-evoked potential in all patients, peaking at around 200–400 ms after neutral face onset (Figure 2). This early negative-evoked response was more pronounced on the inner as compared with the outer electrode contacts. Thus, we are confident that it was not a spillover effect of distant brain structures (Lachaux et al., 2003). This N200 is in agreement with previous ERP studies on face processing (Krolak-Salmon et al., 2004; Pourtois et al., 2010), but is not necessarily face-specific. A recent intracerebral EEG study reported comparable amplitudes of this component across faces, mosaics and houses (Sato et al., 2012). This notwithstanding, the finding of a robust amygdala response to the neutral face on the inner electrode is good evidence that all patients were able to generate face-related amygdala ERPs, as would be expected in healthy subjects (Hoffman and Haxby, 2000; Ishai et al., 2005).

Amygdala responses to the dynamic facial cues: an early effect of gaze

Our paradigm was specifically suited to single out the effect of the social cue, independent of the amygdala response to the face onset: the neutral face turned happy or fearful, with or without gaze aversion, thus forming a behaviorally relevant sequence of events. This form of presentation is suitable to induce apparent motion (Larsen and Bundesen, 2009; Sato et al., 2011). Here, we chose to have global apparent motion elicited in every trial, since the face always turned happy or fearful. We took care to create similar eye movements in the condition of gaze change for the happy and the fearful faces to avoid any low-level confounding effect of emotion on the gaze effect.

The natural sequence thus focused on the appearance of a social cue on a previously neutral face and necessarily elicited more subtle neural responses than the onset of a face. Yet, under this condition, we obtained a reliable early effect of gaze, with differentiated amygdala response to gaze aversion vs direct gaze between 123 and 258 ms in the right amygdala. A trend to a gaze effect between 268 and 299 ms was also observed in the left amygdala. As we did not find an interaction of gaze with hemisphere (left or right amygdala), we assume that the effect of gaze is valid for both left and right amygdala.

Our study is the first to reveal such early responses of amygdala to gaze direction changes. Only two previous studies have focused on the sensitivity of amygdala to the eye region of faces (Sato et al., 2011; Meletti et al., 2012). Of particular relevance to the present study, Sato et al. (2011) used a sequence of gaze change similar to ours but with isolated neutral eyes; they obtained a significant gamma band activity to eyes relative to mosaics, but only a trend around 285 ms for the gaze shift between averted and straight directions (Sato et al., 2011). Taken together with our results, this suggests that the association of the gaze aversion with an emotional change in our study may have ‘boosted’ the effect of gaze, allowing us to reveal early amygdala evoked responses to gaze, as soon as 123 ms after stimulus onset. This is consistent with the fact that emotion and gaze are usually combined in natural occurrences, and that the association of gaze and emotion greatly influences the meaning of gaze (Tipples, 2006).

Could this gaze effect be attributed to apparent motion? Motion is an integral part of many social signals, including gaze, to the point where even when static social cues are presented, they may be perceived as implied motion and induce motion-related brain activations (Kourtzi and Kanwisher, 2000; Barraclough et al., 2006). As mentioned above, apparent motion was intentionally inherent in the paradigm, not only in relation with the eye gaze change, but also in relation with the emotional expressions, which affected other face parts such as the mouth region. The latter did not modulate amygdala responses at the group level, as we did not find a global effect of emotion. While this does not allow ruling out the contribution of apparent motion to our results, it suggests that in our paradigm, the amygdala was selectively sensitive to apparent motion in the eye region. Whether this boils down to a gaze specific effect or may be true for any type of eye movement (such as eye blinking or closing) will need to be investigated in future studies.

Gaze as a salient social stimulus

The early onset of a robust gaze effect suggested that gaze was a particularly salient stimulus for amygdala processing in the context of our spatial orientation paradigm. To our knowledge, our study is the first to show such early responses of the amygdala to emotional gaze, reinforcing its view as a key structure in the processing of social signals from faces. Our data are therefore in line with the view of the amygdala being involved in ‘relevance rating’ (Sander et al., 2003) in accordance with the concomitant task (Adolphs, 2010).

Note that in our study, amygdala responses were examined in response to the cue face with no direct relation to the attention orienting effect as measured through the behavioral gaze cueing effect. However, recent studies have gathered evidence for the role of amygdala in the guidance of spatial attention orientation. In intracerebral recordings of the macaque, Peck et al. (2013) were able to relate neuronal activity in the amygdala to the coding of space and value of a given visual stimulus. Using functional MRI in humans, Gamer and Büchel (2009) demonstrated a correlation between peak BOLD activation of the amygdala and participants’ gaze shifts toward the eye region of fearful faces. Amygdala activation thus predicted the spatial orientation of gaze, most notably toward the eye region of fearful faces. The amygdala thus appears as a ‘relevance detector’ for a variety of behavioral tasks, and may play a specific role in the orientation of spatial attention toward relevant stimuli.

A variable effect of emotion

Despite a robust effect of gaze, our study was not able to single out a reliable effect of the facial expression change across subjects. It is important to recall that in our design, the neutral face always turned emotional, in order to ensure a social cue even if direct gaze was maintained. Emotional valence was therefore explored as the difference in amygdala responses for happy and fearful facial expression—and not in contrast to neutral faces. This may obviously have blunted the emotional effect, as positive valence also triggers amygdala processing (Winston et al., 2003; Fitzgerald et al., 2006; Sergerie et al., 2008; Meletti et al., 2012). In addition, a variable timing of emotional effects has been found in previous studies, ranging from 50 to 800 ms after stimulus onset (for a review see Sato et al., 2011). It seems that a modulatory variable of emotional impact may be the explicit vs implicit nature of emotion processing, or focus of attention on emotion, with a complex and time-dependent interaction between emotion- and attention-related processes (Pessoa et al., 2002; Adolphs, 2010; Pessoa, 2010).

Limitations of the study

The variable modulation of amygdala by emotion raises particular questions for studies using intracerebral electrodes, which generally rely on few patients: for example one single patient examined by Pourtois et al. (2010) to six by Sato et al. (2010, 2011). Inherent variability in timing is further enhanced by variable implantation sites, even if we focused specifically on the inner electrode. Differences in timing and precise electrode location will thus necessarily blur the effect in group comparisons. Nevertheless, the robust gaze effect observed in our study emerged despite the variability inherent in intracerebral patient data.

A further limitation of our study is the complexity of the social cue, combining gaze, motion and emotional information. While enhancing its impact and mimicking real-world conditions (Sato et al., 2011), differentiation of the effect of emotion and its interaction with gaze is rendered more difficult.

In favor of intracerebral electrode studies, particularly in the case of the amygdala, is the exploration of deep structures and their state-of-the-art time resolution (Lachaux et al., 2003; Axmacher et al., 2009; Adolphs, 2010). The latter is irreplaceable for the scrutiny of hierarchical processing, which can also serve for network studies, if the implantation scheme is appropriate (Krolak-Salmon et al., 2004). Intracerebral recordings of the amygdala in combination with the spatial attention network and the dedicated structures responsible of gaze processing would certainly help to advance our understanding of its role in network function.

Conclusion

Our study using intracerebral recordings demonstrates the implication of amygdala in a dynamic, natural sequence of face processing, where the social cue is given by gaze and emotion. To our knowledge, this is the first study to show an early and robust effect of gaze. Gaze had precedence over emotion in our spatial attention paradigm, whereas the emotional effect was more variable in time and did not reach statistical significance in the group data analysis. Altogether, our study brings direct support to the vision of amygdala as a key structure of relevance appraisal and environmental stimulus evaluation in the momentary context of task.

Funding

This work was supported by the Agence Nationale de la Recherche under Grant number ANR 12 SAMA 01104 to N.G. and by the French program “Investissements d'avenir” under Grant numbers ANR-10-IAIHU-06 and ANR-11-INBS-0006.

Supplementary data

Supplementary data are available at SCAN online.

Conflict of interest. None declared.