Human cortical activity evoked by gaze shift observation: An intracranial EEG study

Abstract

While is widely accepted that the posterior temporal region is activated during the observation of faces showing gaze shifts, it is still unclear whether its activity is stronger while observing direct or averted gaze. Furthermore, despite its assessed role in social cognition, studies describing an enhanced activity of the posterior temporal region during the observation of gaze aversion interpreted this activity in terms of spatial attention toward the target direction. This spatial attention interpretation is not easily reconcilable with the role of the posterior temporal region in social cognition, and an overarching view of its global cognitive function would be much more preferable. Here we used intracranial EEG to assess the precise spatial localization of the gaze shifts coding in the posterior temporal region, to assess its selectivity for direct versus averted gaze and to distinguish between a spatial‐attentional and a social interpretations of gaze aversion. We found stronger activation during gaze aversion than direct gaze and lateral side switch observation, the latter indicating that the crucial aspect of gaze aversion is the prior presence of the eye contact and its interruption, and not the gaze direction. These results suggest a more social‐oriented interpretation based on the view that among humans, gaze aversion signals a negative relational evaluation in social interaction. Hum Brain Mapp 35:1515–1528, 2014. © 2013 Wiley Periodicals, Inc.

INTRODUCTION

Gaze detection is one of the major topics in social cognition as well as a starting point for investigating deficits in social cognition, such as autism [Baron‐Cohen, 1995]. This particular attention is grounded on the fact that others' gaze is a source of information about their emotions, intentions to interact, or on the presence of some environmental cue attracting their attention. In particular, despite gaze detection belongs to the broad field of biological motion, different types of gaze movements, such as the establishment of eye contact or a sudden shift toward a lateral target, have been considered to convey different types of social information. It is largely accepted that direct gaze signals the intention to start communicative interactions and triggers a stronger physiological arousal compared to averted gaze [Kampe et al., 2003; Nichols and Champness 1971; see Senju and Johnson, 2009]. In line with this interpretation, it has been recently shown that the electrical stimulation of a specific region of the monkey insula evokes affiliative communicative responses only when eye contact is established between the monkey and the experimenter, while the same response is suddenly interrupted by gaze aversion [Caruana et al., 2011]. Furthermore, compared with averted gaze, a face with direct gaze is detected faster [Conty et al., 2007, 2006; Senju and Hasegawa, 2006; Von Grunau and Anston, 1995] and reaction time in a gender categorization task is reduced, while access to semantic memory is enhanced [Macrae et al., 2002].

In contrast to the shared agreement on the social meaning of direct gaze, two different interpretations have been associated to gaze aversion. The prevailing view is that gaze aversion triggers a spatial attention mechanism, based on the assumption that others' gaze aversion indicates their shift of attention toward a specific direction, thus inducing in the observer a reflexive shift of attention toward the same direction [Hadjikhani et al., 2008; Straube et al., 2010; see also Puce and Perrett, 2003]. This view focuses on the orienting function of another's gaze and does not consider the social value associated with gaze aversion. An alternative view is that gaze aversion is a nonverbal cue indicating a negative relational evaluation in social interaction, that is, a form of social exclusion or “silent treatment” [Williams et al., 1998]. In this view, the salient aspect of gaze aversion is the interruption of the eye contact and not the final direction of the gaze. Although this interpretation has usually been neglected by cognitive neuroscientists, behavioral data have shown that the observation of gaze aversion is associated with reduced self‐esteem, lower feelings of belonging, greater negative mood and the tendency to infer less positive personality traits about the gaze averter, relative to those providing direct gaze [Wirth et al., 2010]. As a consequence, gaze aversion must strongly influence the social brain. Based on the endorsed interpretation, different functional roles could be ascribed to brain regions activated by gaze coding. However, according to the spatial‐attentional interpretation of gaze aversion, one would expect that the posterior temporal region, a wide region commonly activated by gaze aversion [Hadjikhani et al., 2008; Hoffman et al., 2000; McCarthy et al., 1999; Watanabe et al., 2002], is involved in spatial attention. This role is largely independent, as well as in possible contrast, from the prevailing view that the posterior temporal region is part of the social brain involved in the recognition of others' intentions [Allison et al., 2000; Keysers and Perrett, 2004]. These views of the function of the posterior temporal region are not easily reconcilable, and an “overarching view” [Cabeza et al., 2012] of the global cognitive function supported by the posterior temporal region would be preferable.

In this study we analyzed intracranial recordings from drug‐resistant epileptic patients, explored by means of stereo‐EEG [sEEG; Cossu et al., 2005], while observing different types of dynamic gaze shift (direct gaze, averted gaze, and lateral side‐switch). This approach allowed us to disambiguate between spatial and social information associated with the gaze shift. An advantage of sEEG recordings is the ability to record reliably intracranial event‐related potentials (iERP) as well as event‐related high‐frequency activity in the gamma‐band (gamma‐band response ‐ GBR) from a very limited cortical volume. High‐frequency activity in the gamma‐band is the object of a marked interest in cognitive neuroscience because of its high correlation with multi‐unit activity, thus offering a higher spatial resolution [see Lachaux et al., 2012]. We restricted our analysis to contacts located in the posterior temporal region, including the superior, middle and inferior temporal gyri (STG, MTG, and ITG, respectively), the supramarginal gyrus (SMG), the occipitotemporal cortex (OT) and the insula, according to previous studies showing the involvement of these areas in gaze coding (for review see Allison et al., 2000; Senju and Johnson, 2008; Nummenmaa and Calder, 2009). The aims of the study were (1) to assess the precise spatial localization of the gaze shifts coding in the posterior temporal region by means of both iERP and GBR analysis, (2) to investigate the selectivity of the posterior temporal region to direct versus averted gaze, and (3) to distinguish between the spatial and social interpretations of the gaze aversion.

MATERIALS AND METHODS

The experiment was performed on nine patients suffering from drug‐resistant focal epilepsy and stereotactically implanted with intracerebral electrodes as part of their presurgical evaluation (see Table 1), at the “Claudio Munari” Center for Epilepsy Surgery, Ospedale Niguarda‐Ca' Granda, Milan, Italy. Implantation sites were selected on purely clinical grounds, on the basis of seizure semiology, scalp‐EEG and neuroimaging studies, and with no reference to the present experimental protocol. Patients were fully informed of the electrode implantation and sEEG recordings, and, according to the Declaration of Helsinki (BMJ 1991; 302: 1194) gave informed consent to participate in the study. We selected patients whose posterior temporal region was not affected by epileptic activity. No seizures were recorded during the 24 h prior to the experiment. No alteration in the sleep/wake cycle was observed, and no additional pharmacological treatment was applied before the experiment. Patients did not show any motor or cognitive deficits during the examination; they fully understood the instructions and easily performed the experimental task.

Table 1.

Results from the posterior temporal region

Patient	Gend.	Hem.	Contact	N°	x	y	z	Region	Epileptic Zone	GS vs. FIX		AG vs. DG		AG vs. SS		LG vs. RG
										iERP	GBR	iERP	GBR	iERP	GBR	iERP	GBR
P1	M	R	W	7	65	−23	31	Supramarginal	Temporal Operculum	GS	–	–	–	–	–	n/a	n/a
P1	M	R	B	4	64	−19	−12	Temporal Mid	Temporal Operculum	–	–	–	–	–	–	n/a	n/a
P1	M	R	C	3	63	−46	−0,5	Temporal Mid	Temporal Operculum	–	–	AG	–	–	–	n/a	n/a
P1	M	R	U	8	67	−15	3	Temporal Sup	Temporal Operculum	–	–	–	–	–	–	n/a	n/a
P2	F	R	W	5	65	−27	9	Temporal Sup	Orbitofrontal + Temporal pole	GS	GS	AG	AG	–	AG	–	–
P2	F	R	C	7	65	−26	−7	Temporal Mid	Orbitofrontal + Temporal pole	–	–	AG	–	–	–	–	–
P3	F	R	W	5	64	−48	11	Temporal Mid	Unknown	GS	GS	–	–	–	SS	–	–
P3	F	R	F	4	54	−68	5	Temporal Mid	Unknown	GS	GS	–	–	AG	SS	LG	–
P3	F	R	D	2	64	−47	−17	Temporal Inf	Unknown	GS	GS	–	–	–	–	RG	–
P3	F	R	C	5	68	−39	−4	Temporal Mid	Unknown	GS	GS	AG	–	AG	–	–	–
P3	F	R	U	5	67	−21	5	Temporal Sup	Unknown	GS	–	–	–	AG	–	–	–
P4	F	R	F	6	63	−60	9	Temporal Mid	Left Frontal	GS	GS	–	AG	n/a	n/a	n/a	n/a
P4	F	R	E	5	62	−57	−6	Temporal Inf	Left Frontal	FX	–	–	–	n/a	n/a	n/a	n/a
P4	F	R	W	6	68	−31	17	Temporal Mid	Left Frontal	GS	GS	–	–	n/a	n/a	n/a	n/a
P4	F	R	C	2	68	−40	1	Temporal Mid	Left Frontal	GS	–	–	–	n/a	n/a	n/a	n/a
P4	F	R	D	3	65	−35	−12	Temporal Mid	Left Frontal	–	–	–	–	n/a	n/a	n/a	n/a
P4	F	R	V	4	51	−74	21	Temporal Mid	Left Frontal	FX	FX	–	–	n/a	n/a	n/a	n/a
P5	F	L	V′	2	−47	−84	12	Occipital Mid	Left Parietal	–	–	–	AG	–	AG	–	–
P5	F	L	O′	5	−51	−84	−3	Occipital Inf	Left Parietal	GS	GS	AG	AG	AG	–	–	–
P5	F	L	Q′	5	−66	−44	29	Supramarginal	Left Parietal	–	–	AG	–	–	–	–	–
P5	F	L	D′	5	−52	−76	22	Temporal Mid	Left Parietal	–	–	–	–	–	–	–	–
P5	F	L	C′	6	−70	−39	−4	Temporal Mid	Left Parietal	–	–	AG	–	–	–	–	–
P5	F	L	W′	11	−70	−28	10	Temporal Sup	Left Parietal	–	–	–	–	–	–	–	–
P6	M	L	V′	8	−54	−70	4	Temporal Mid	Left Occipital + Mesial Temporal	–	–	AG	AG	AG	–	–	LG
P6	M	L	X′	8	−60	−52	13	Temporal Mid	Left Occipital + Mesial Temporal	GS	GS	AG	AG	–	–	LG	–
P6	M	L	F′	5	−62	−53	−2	Temporal Mid	Left Occipital + Mesial Temporal	–	GS	–	–	–	AG	–	–
P6	M	L	E′	2	−64	−39	−17	Temporal Inf	Left Occipital + Mesial Temporal	–	–	AG	–	AG	–	LG	–
P6	M	L	W′	9	−65	−24	5	Temporal Sup	Left Occipital + Mesial Temporal	GS	GS	–	–	–	–	–	–
P7	M	L	Q′	4	−62	−40	33	Supramarginal	Anterior Cingulate Cortex	–	–	AG	–	–	–	n/a	n/a
P8	M	L	F′	3	−57	−50	−20	Temporal Inf	Mesial Temporal	GS	–	–	–	–	–	n/a	n/a
P8	M	L	G′	4	−60	−44	−2	Temporal Mid	Mesial Temporal	GS	–	AG	–	AG	–	n/a	n/a
P8	M	L	C′	6	−63	−36	−15	Temporal Mid	Mesial Temporal	GS	–	–	–	–	–	n/a	n/a
P8	M	L	E′	5	−64	−21	−18	Temporal Mid	Mesial Temporal	GS	–	–	–	–	–	n/a	n/a
P8	M	L	W′	11	−63	−31	11	Temporal Sup	Mesial Temporal	–	–	–	–	SS	–	n/a	n/a
P8	M	L	U′	7	−65	−15	6	Temporal Sup	Mesial Temporal	–	–	–	–	–	–	n/a	n/a
P9	M	L	E′	4	−55	−70	−6	Occipital Inf	Unknown	GS	GS	AG	AG	AG	AG	n/a	n/a
P9	M	L	F′	6	−61	−57	12	Temporal Mid	Unknown	GS	GS	AG	AG	AG	AG	n/a	n/a
P9	M	L	V′	6	−52	−78	4	Occipital Mid	Unknown	GS	GS	AG	AG	–	–	n/a	n/a
P9	M	L	W′	8	−65	−31	15	Temporal Sup	Unknown	GS	–	AG	–	–	AG	n/a	n/a

x, y, z refers to entrance point of the electrode, in MNI coordinates.

For each electrode the precise number of contacts located in the gray matter is shown. Results were significant for one electrode when at least one of its contacts showed a significant interaction between conditions (P < 0.05). For all contacts showing a statistically significant interaction, the preferred condition was assessed by a post‐hoc analysis (paired T‐Test).

Electrode Implantation

For each patient, up to fifteen depth electrodes were implanted in different regions of the brain including the posterior aspect of the STG, MTG, ITG, and insula. To reach the clinically relevant targets, the stereotactic coordinates of each electrode were calculated preoperatively based on the individual cerebral MRI. Each electrode had a diameter of 0.8 mm and was comprised of between 10 and 15 two millimeter long contacts, spaced 1.5 mm apart (DIXI®, Besancon, France). Cerebral structures explored by each electrode contact were determined by coregistration of preimplantation volumetric brain MRI with postimplantation volumetric brain CT, and visualized by a software package for visualization and image analysis (3DSlicer®).

Paradigm

Recordings were obtained in a dimly light, quiet room. The patient sat ∼100 cm away from the laptop display where the stimuli were presented. The basic stimuli consisted of static images depicting female (n = 3) and male (n = 3) human faces, differing exclusively in eye position: eye contact, leftward gaze, rightward gaze, or eye closed. The size of the stimuli was 17.2 x 13 cm.

Each trial was constituted by the rapid succession of two basic stimuli from the same subject in different types of gaze positions, producing a clear apparent motion of the eyes, in line with previous EEG studies on “gaze shift” [Puce et al., 2000]. The duration of the first image was randomized to be between 800 and 1,600 ms, with steps of 200 ms (800 ms, 1,000 ms, 1,200 ms, 1,400 ms, 1,600 ms), in order to avoid any expectation effect, and the duration of the second image was fixed at 800 ms. A black fixation cross on a white background separated consecutive trials and lasted 1,000 ms (see Fig. 1, left panel).

Figure 1.

A: Experimental stimuli. Stimuli were composed of couples of static images depicting human faces and followed by a fixation cross. B: Experimental conditions. (1) Averted gaze (AG): black solid arrow. (2) Direct gaze (DG): black dotted arrow. (3) Side‐switch (SS): gray solid arrow.

The possible combinations of the two images, constituting the experimental conditions, were the following (see Fig. 1, right panel): (1) averted gaze (AG), i.e. from eye contact to lateral gaze (rightward or leftward); (2) direct gaze (DG), i.e. from lateral gaze (rightward or leftward) to eye contact; (3) side‐switch (SS), i.e. from rightward to leftward gaze, or vice‐versa; (4) eye‐closure (CL), i.e. from eye contact or lateral gaze, to eye closed. To control for the specificity of the investigated electrode contacts to gaze coding, two additional conditions were compared: all gaze shift types pooled together (GS) and the appearance of the fixation cross (FX). Ninety trials per each condition were presented in a fully randomized order. The patients were asked to observe the images and to press the spacebar as soon as eye‐closure occurred. For this reason, CL trials were used as catch trials and discarded from following analysis. A short practice session (less than 2 min) took place before the experiment.

Data Analysis

During the experiment continuous sEEG was recorded with a 1,000 Hz sampling rate by means of a 192 channel‐EEG device (EEG‐1200 Neurofax, Nihon Kohden®). Each channel was referred to a contact in the white matter far from the recording sites, and in which low and high frequency electrical stimulations did not produce any subjective or objective manifestation (neutral reference). As the aim of this study was to characterize the response to gaze shift in the posterior temporal and perisylvian electrodes without any functional selection criteria, at the end of each experimental session the activity of each contact located in the posterior temporal region (n = 39 electrodes, 211 recording sites) and insula (n = 17 electrodes, 21 recording sites) was selected. A visual inspection was carried out by clinicians in order to ensure the absence of any pathological interictal activity. Trials showing artifacts were removed. A band‐pass filter (0.015–500 Hz) was applied to avoid any aliasing effect. Each trial was epoched with a [−500, +1,000] ms time window, with respect to the second image onset. Intracranial event‐related potentials were computed for all contacts located in the posterior temporal region and insula. In addition, GBR were analyzed in the time‐frequency (TF) domain by convolution with complex Morlet's wavelet (50–150 Hz, following Lachaux et al., 2012; Vidal et al., 2010). According to previous intracranial studies [see Vidal et al., 2010], gamma power was estimated for 10 adjacent non overlapping frequency bands, each 10 Hz wide, and baseline corrected versus the prestimulus interval for each single band [−200/0]. More specifically, a divisive baseline was employed for high frequency activity computation, while a subtractive baseline was employed for iERP. The average of all the bands was computed and employed in the following statistical analysis.

Statistical Analysis

For all of the presented results and comparisons, we applied an analysis of single trial responses in individual studies of the nine patients. Gamma power values and EEG amplitude were epoched in 50 ms contiguous time bins. Repeated measures ANOVA were performed on both iERP and GBR values, considering CONDITION (2 levels) and TIME (30 adjacent time bins [−500, +1,000]) as factors, for specific couples of conditions (GS vs. FX, AG vs. DG, AG vs. SS and RG vs. LG). Main effects of CONDITION, TIME, and the CONDITION*TIME interaction were explored. When significant, post‐hoc analysis was performed by using a paired T‐Test. Besides single contrasts between couples of conditions, a population analysis was performed for the AG, DG, and SS, over all the electrodes and patients. For each condition iERP and GBR values were computed as the activity peak within two different time windows: early (from 0 to 400 ms) and late (from 400 to 800 ms). A repeated measures ANOVA was conducted with TIME (early and late) and CONDITION (AG, DG, and SS) as within subject factors. Furthermore, to evaluate the presence of differences in latency across the conditions (AG, DG, SS) a repeated measures ANOVA was conducted on the timing of the early iERP peak. When significant, post‐hoc analysis was performed (Bonferroni correction).

RESULTS

Gaze Shift vs. Fixation

A first analysis looked for significant differences between the observation of all the presented dynamic gaze shifts pooled together (GS) and the observation of the fixation cross (FX). Its aim was to assess whether the activation was elicited by a specificity of the investigated region to gaze shift observation or by responsiveness to mere changes in low‐level visual features. In fact, GS are characterized by minimal changes in the low‐level perceptual features, including contrast, edges and general luminance. In contrast, the shift from the Image2 to the fixation cross (FX) involves a strong variation in the same low‐level features, eliciting a strong response in the low‐level visual areas (Fig. 2). A repeated measures ANOVA was performed on sEEG activity recorded from central and posterior temporal contacts, as previous studies suggested that gaze coding is particularly represented in this location. We assessed the iERP difference between the GS and FX conditions by considering CONDITION (GS, FX) and TIME (30 adjacent time bins) and as factors. The ANOVA indicated a significant interaction (P < 0.05) in 23 out of 39 electrodes (12 = right; 11 = left). In 21 out of 23 electrodes the iERP amplitude was greater for GS, while FX elicited a greater iERP amplitude only in two cases from the right hemisphere. Significant results were recorded from a wide region including the SMG, STG, MTG, ITG, and the rostral aspect of the occipitotemporal cortex (Fig. 3 and Table 1).

Figure 2.

Upper panel. TF map recorded during gaze shift observation in a representative patient. Time zero indicates the presentation of Image2. High‐frequencies up to 400 Hz are clearly modulated by gaze shift presentation. The precise anatomical localization of the contact is shown in both sagittal and coronal sections (right). Note the lack of GBR during the presentation of the fixation window, appearing at 800 ms after stimulus onset. Middle panel. Different recruitment of a contact located in the posterior temporal region (red) and one located in the calcarine fissure (blue), during gaze shift observation (time zero) and fix condition (800 ms after stimulus onset). Posterior temporal contact shows a clear iERP and GBR at 200 ms after gaze shift observation and a weaker response during the fix condition. In contrast, the contact located in the calcarine fissure is strongly recruited during the fix condition. The precise anatomical localization of the calcarine contact is shown in both sagittal and coronal sections (right). Lower panel. Result from a representative patient (P9) in the Averted gaze vs. Direct gaze comparison. The precise anatomical localization of the contact is visualized on the inflated brain of the patient (FreeSurfer®). Single trials (up) and average response (down) during the AG (blue) and DG (red) conditions are shown, for both iERP (μV) and GBR (dB).

Figure 3.

Results of the repeated measures ANOVA (P < 0.05) applied to iERP and GBR, in the four statistical comparisons. Entrance points of the electrodes for each patient are plotted on a template brain (FreeSurfer®). Gaze shift vs. Fixation. GS>FX=green; FX > GS = orange; no response = white. Averted gaze vs. Direct gaze. AG > DG = green; nonsignificant differences = orange; no response = white. Averted gaze vs. Side switch. AG > SS = green; SS > AG = yellow; nonsignificant differences = orange; no response = white. Rightward vs. Leftward. RG > LG = green; LG > RG = orange; nonsignificant = white.

A similar procedure was applied to GBR. Results showed a significant interaction (p < 0.05) in a reduced number of electrodes (15 out of 39 electrodes; 8=right; 7 = left). All significant results showed a greater response for GS; however, a single case showed a greater GBR in the FX condition (right hemisphere). Significant results were mainly recorded from the posterior MTG.

Responses to Different Gaze Shifts Observation

A first ANOVA investigated the iERP amplitudes for the AG, DG and SS conditions, at the population level. The ANOVA indicated a significant CONDITION*TIME interaction (p < 0.05), with a greater early iERP amplitude during AG with respect to DG and SS (p < 0.005), as revealed by post‐hoc analysis. No significant difference was found between DG and SS (see Fig. 4). The ANOVA on the latency of the early iERP peak revealed a non‐significant CONDITION*TIME interaction. The mean latency for each condition lied at about 250ms (273ms, 241ms and 250ss for AG, DG, SS, respectively). The same pattern was also confirmed by the ANOVA conducted on the GBR values. Gamma‐band responses obtained from the same dataset showed a significant CONDITION*TIME interaction (p<0.00001), with a greater GBR during AG with respect to DG and SS (p<0.005; see Fig. 5). No significant difference was found between DG and SS (see Fig. 4). The ANOVA on the latency of the early GBR peak revealed a non‐significant CONDITION*TIME interaction.

Figure 4.

Statistical analysis conducted on both iERP (left) and GBR (right) values recorded by all electrodes exploring the posterior temporal region. Values were computed as the maximum absolute amplitude (iERP) and power (GBR) within two different time windows (early 0–400 ms; late 400–800 ms). In both cases, the CONDITION*TIME interaction was significant. Horizontal bars indicate significant post‐hoc between early values (**P < 0.001; ***P < 0.0001). No significant differences were found between late values. All early values were significantly higher than late ones (not shown in the figure). Error bars indicate standard error.

Figure 5.

Grandaveraged TF map of AG, DG and SS conditions are reported in the upper, middle and lower panel, respectively. Time zero indicates the presentation of Image2. All frequencies ranging from 10 Hz to 400 Hz are shown. No difference in latency are remarkable. Note the different power increase across the three conditions, with AG showing the strongest and DG the weakest response. [Color figure can be viewed in the online issue, which is available at http://wileyonlinelibrary.com.]

Averted Gaze vs. Direct Gaze

The ANOVA performed on iERP indicated a significant interaction (p < 0.05) in 16 out of 39 electrodes. Although event‐related potentials showing a steep slope (significant TIME effect) were recorded from electrodes implanted in both hemispheres, statistically significant differences between conditions were mostly obtained from the left hemisphere (4 = right; 12 = left). In all significant interactions, iERP amplitude was greater during AG than DG (Fig. 2, lower panel). Significant results were recorded from a wide region including the SMG, STG, MTG, ITG, and the rostral aspect of the occipitotemporal cortex.

Gamma‐band responses obtained from the same dataset showed a significant interaction (p < 0.05) in a reduced number of electrodes, mainly on the left hemisphere (9 out of 39 electrodes; 2 = right; 7 = left). All significant results showed a greater response for AG, and were mainly obtained from the posterior MTG adjacent to the occipitotemporal cortex (Fig. 3 and Table 1).

Averted Gaze vs. Side‐Switch

The ANOVA performed on iERP indicated a significant interaction (p < 0.05) in 10 out of 33 electrodes (data were recorded only from 8 patients). Although event‐related potentials showing a steep slope (significant TIME effect) were recorded from electrodes implanted in both hemisphere, statistically significant differences between conditions were mostly obtained from the left hemisphere (3 = right; 7 = left). Intracranial ERP amplitude was greater during AG than SS in all but one case (left hemisphere, STG). Significant results were recorded from a wide region including the STG, MTG, ITG, and the rostral aspect of the occipitotemporal cortex.

Gamma‐band responses obtained from the same dataset showed a significant interaction (p < 0.05) in a reduced number of electrodes, mainly on the left hemisphere (8 out of 33 electrodes; 3 = right; 5 = left). In the left hemisphere, all significant results showed a greater response for AG; in the right hemisphere, 2 out of 3 electrodes showed a greater response for SS. Results were mainly obtained from the posterior MTG bordering with the occipitotemporal cortex (Fig. 3 and Table 1).

Gaze Direction

This analysis looked for significant differences between the observation of rightward gaze shift and leftward gaze shift. Data were collected from only four patients (P2, P3, P5, P6). The ANOVA performed on iERP indicated a significant interaction (p < 0.05) in 4 out of 18 electrodes (2 = right; 2 = left). In 3 out of 4 electrodes, iERP was greater for LG.

Gamma‐band responses obtained from the same dataset did not show any significant interactions (p < 0.05), except in one electrode located in the left posterior MTG, which showed a greater response to LG (Fig. 3 and Table 1).

Gaze Coding in the Insula

The last analysis looked for significant differences between the observation of all the presented dynamic gaze shifts pooled together (GS) and the observation of the fixation cross (FX) in the insula. The ANOVA was conducted on 17 electrodes over 7 patients (P1, P2, P3, P5, P6, P8, P9. In two patients the insula was not implanted). Contacts were located in both the anterior (short gyri, 6 contacts) and posterior insula (long gyri, 11 contacts; see Fig. 6). The ANOVA performed on iERP indicated a significant interaction (p<0.05) in 14 out of 17 electrodes. In all cases, GS elicited a larger potential peak. Among them, only 7 electrodes showed a statistical difference before 250 ms. Significant results obtained before 250 ms were recorded from the inferoposterior aspect of the insula, while significant results obtained after 250 ms were recorded from the anterior insula. Silent electrodes were located in the posterior dorsal insula, adjacent to the parietal operculum (see Table 2 and Fig. 6). The ANOVA on iERP, using AG and DG as CONDITIONS showed a significant interaction (p < 0.05) only at two sites showing an early response (1 = right; 1 = left). The ANOVA on iERP using AG and SS did not showed any significant results (see Table 2).

Figure 6.

Left panel. Insula results of the repeated measures ANOVA (P < 0.05) applied to iERP in the Gaze shift vs. Fixation comparison. Contacts' positions are plotted on a schematic illustration of the insula [modified from Stephani et al., 2011]. For the sake of comparison all of the contacts were plotted in a left hemisphere. Right panel. Representative samples of iERPs showing an early response (up), a late response (center) and no response (down) during the gaze shift (black) vs. fixation (gray) conditions.

Table 2.

Results from the insular cortex

Patient	Gend.	Hem.	Contact	N°	Sector	GS vs. FX (early)	GS vs. FX (late)	AG vs. DG	AG vs. SS
P1	M	R	R	2	anterior	–	GS	–	–
P1	M	R	S	2	posterior	–	–	–	–
P1	M	R	T	2	posterior	–	GS	–	–
P1	M	R	W	1	posterior	GS	GS	–	–
P1	M	R	X	2	anterior	–	GS	–	–
P2	F	R	R	1	posterior	–	–	–	–
P2	F	R	U	1	posterior	GS	GS	AG	–
P3	F	R	R	1	anterior	–	GS	–	–
P3	F	R	S	1	posterior	–	–	–	–
P3	F	R	T	2	anterior	–	GS	–	–
P5	F	L	W′	1	anterior	–	GS	–	–
P6	M	L	W′	1	posterior	GS	GS	–	–
P6	M	L	T′	2	anterior	–	GS	–	–
P8	M	L	T′	1	posterior	GS	–	–	–
P8	M	L	W′	1	posterior	GS	–	–	–
P9	M	L	U′	1	posterior	GS	GS	–	–
P9	M	L	W′	2	posterior	GS	GS	AG	–

For each electrode the precise number of contacts located in the gray matter is shown.

Results were significant for one electrode when at least one of its contacts showed a significant interaction between conditions (P < 0.05). For all contacts showing a statistically significant interaction, the preferred condition was assessed by a post‐hoc analysis (paired T‐Test). Results refer to iERP only. No gamma‐band activity was found in any insular site.

Gamma‐band responses obtained from the same dataset did not showed any significant interaction (p < 0.05). More interestingly, the lack of a significant TIME effect for GBR demonstrated that the observation of the gaze shift was not able to elicit any modulation of the high‐frequency activity, neither in the early nor in the late response sites.

DISCUSSION

In the present experiment we investigated the iERP and GBR elicited by the observation of different types of gaze shift, with the specific aim of localizing gaze coding in the posterior temporal region, and to distinguish between a social and a spatial‐attentional interpretation of gaze representation in this region. The results of the present study showed that (1) gaze shift observation evokes a reactivity at about 250ms after stimulus onset, possibly corresponding to the N200 scalp‐ERP [Watanabe et al., 2002]from both the posterior temporal region and the inferoposterior insula; (2) a stronger activation was recorded during the observation of gaze aversion, as compared to both direct gaze and side‐switch; (3) gaze direction (leftward vs. rightward) was poorly represented; (4) GBR appears to be more spatially selective, circumscribing the region recruited during gaze aversion in the posterior aspect of the MTG adjacent to the occipitotemporal cortex.

Gaze Aversion or Direct Gaze?

Previous EEG studies found that averted and direct gaze are dissociated at the early stages of gaze processing, evoking different event‐related potentials between 160 and 210 ms from the temporal region [Conty et al., 2007; Senju et al., 2005; Watanabe et al., 2002]. Imaging studies using similar paradigms suggested that the gaze‐evoked potential is generated within a wide posterior temporal region, including the posterior aspects of the middle temporal gyrus (MTG), the dorsal and ventral branches of the superior temporal sulcus (STS), and the superior temporal gyrus [STG; Calder et al., 2002; Ethofer et al., 2011; Hadjikhani et al., 2008; Hooker et al., 2003; Mosconi et al., 2005; Nummenmaa et al., 2010]. When the averted vs. direct gaze preference was explicitly investigated, a majority of studies found that this wide region is strongly activated during gaze aversion [Grossman et al., 2007; Hadjikhani et al., 2008; Hoffman and Haxby, 2000; McCarthy et al., 1999; Nummenmaa et al., 2010; Puce et al., 2000; Watanabe et al., 2002]. However, other studies described a preference for direct gaze [Conty et al., 2007; Pelpherey et al., 2004], or no significant difference between the two conditions [George et al., 2001; Wicker et al., 1998].

In the present experiment we showed that, in the left hemisphere, gaze aversion triggers a stronger response with respect to direct gaze or side‐switch. The stronger involvement of the left hemisphere during the observation of gaze aversion confirms previous imaging data [Hoffman and Haxby, 2000]. Compared with iERP, GBR were more selective, localizing the response in the posterior MTG. It is interesting to compare our results with a previous iEEG study by McCarthy et al. 1999. They recorded iERP from patients observing three types of static images: direct gaze, averted gaze, or closed eyes. The authors found that all conditions in which the eyes were not looking at the viewer evoked N200s larger than the ones evoked during direct gaze, but, differently from our findings, their results were not statistically significant. Since classical biological motion experiments viewed the posterior temporal region as an area primarily involved in the general processing of articulated human movements [Beauchamp et al., 2002; Grossman and Blake, 2002; Pyles et al., 2007; Vaina et al., 2001], a possible explanation of the weaker results of McCarthy et al. 1999 is the employment of static images (photographs) instead of dynamic stimuli (gaze shifts creating an apparent motion), as in our case. A similar explanation was also suggested by Straube and coworkers to account for the absence of a main effect of gaze in the STS in their fMRI study on gaze observation with static images [Straube et al., 2010]. Accordingly, scalp EEG data on gaze shift observation using apparent motion found significant larger responses during eye aversion, confirming this hypotheses [Puce et al., 2000]. Recent fMRI experiments on humans [Jastorff and Orban, 2009] and monkeys [Jastorff et al., 2012] found that this region is particularly selective to the dynamic aspects of the biological stimuli, such as kinematics, but not to static aspects such as shapes, that are coded in a more ventral region. Interestingly, it has been recently shown in the monkey that the electrical stimulation of the temporal operculum adjacent to STG elicits a lateral shift of gaze [Jezzini et al., 2012]. In line with this observation, similar gaze shifts have been observed in epileptic patients during seizures involving STG and the inferoposterior quadrant of the insula [Isnard et al., 2004]. These results are particularly interesting because, together with the present results obtained during gaze shift observation, they open the quest for a common code between observation and execution.

Spatial Attention or Social Cognition?

To account for the strong activity elicited by gaze aversion observation in the posterior temporal region, a spatial attention interpretation has been proposed based on the view that others' gaze aversion indicates their shift of attention toward a specific direction, thus inducing in the observer a reflexive shift of attention toward the same direction [Hadjikhani et al., 2008; Hoffman and Haxby, 2000; Grossman et al., 2007; Straube et al., 2010; see also Puce and Perrett, 2003]. A competing interpretation is that gaze aversion signals rejection and avoidance and, as a consequence, that the posterior temporal region may distinguish between gaze signaling approach or avoidance [Ethofer et al., 2011]. This interpretation is confirmed by behavioral studies showing that gaze aversion is associated with reduced self‐esteem, lower feelings of belonging, greater negative mood and the tendency to infer less positive personality traits about the gaze averter, relative to those providing direct gaze [Wirth et al., 2010]. In our experiment, the comparison between AG and SS supports the latter, social, interpretation. In fact, while statistical differences are elicited during the presentation of Image2, in both the AG and SS Image2 is comprised of the same set of stimuli, that is, lateral gaze. As a consequence, the lateral gaze evokes different responses depending on the gaze position in Image1. This observation leads to two conclusions. First, the present data support the view that dynamic stimuli are required. Static stimuli depicting lateral gaze, as used in previous experiments, could not highlight this selective activation. Second, they suggest that the most interesting aspect of the gaze aversion is the interruption of the eye contact and not the gaze direction per se. Our results of the analysis on the gaze direction are also in line with this interpretation: the comparison between leftward and rightward gaze shift did not produce significant results, suggesting that specific spatial directions, toward which the attention must be addressed, are not segregated in this cortical region, while it remains possible that different directions are coded by different neurons jeopardized within the same cortical area, thus making this selectivity undetectable in the average response of the overall neuronal population. Note also that the sole spatial attention interpretation is not compatible with the findings that also direct gaze elicited a clear response in the posterior MTG (Fig. 2, lower panel) and that in different cases this response was not statistically different to AG. As a consequence, the supporters of the spatial attention interpretation must at least consider that attentional processes and high level social cognition processes interact within this region, and that the latter plays a major role. Another social interpretation of our results is that the detection of a direct gaze preceding the gaze shift engages the attention of the observer and thus is more socially salient [Csibra and Gergely, 2009; Frischen et al., 2007]. Previous studies showed that the presence of eye contact modulates the processing of social stimuli that follow it [Senju and Johnson, 2009] and that a period of eye contact is required for infants to shift their attention towards the direction of another's gaze [Farroni et al., 2003; Senju and Csibra, 2008]. As a consequence it is also possible that in our experiment the direct gaze preceding the lateral gaze shift in the AG engages the attention of the observer more than an opposite lateral gaze, as it is in the SS and DG. Despite this interpretation is in line with our data, as well as with our social interpretation of them, however it does not account for the stronger activity elicited by gaze aversion even in the absence of a prior eye contact, as demonstrated by other studies employing static stimuli [McCarthy et al. 1999; Straube et al., 2010]. Finally, a third possible interpretation of our data is that this region signals direction of gaze, regardless of its salience (social and/or spatial), as indicated by the activity elicited by every gaze shift; however, the stronger activation evoked by gaze aversion should imply that higher hierarchical stages of the process, eventually elaborated in a more extended network, start within the same cortical area. This would confirm that the posterior temporal region is part of this socially‐oriented cortical network.

Taken together, our data suggest that posterior MTG is neither involved in the processing of articulated biological movements per se, nor in spatial attention, but rather in the perception of intentional actions with specific social meanings [Jarstorff and Orban, 2009]. This view offers a more parsimonious explanation of the global cognitive function supported by the posterior temporal region, bridging the gap with other studies showing that the region surrounding the posterior STS is involved in the meaningful social and intentional aspects of a specific action, and not only in its mere perceptual features. In particular, it has been demonstrated that the region surrounding STS is more strongly activated during the observation of moving geometric shapes depicting complex social interactions compared to animations depicting inanimate motion [Castelli et al., 2000]. Furthermore, identical visual stimuli depicting the eye region activated the STS only when subjects were asked to interpret their feelings of the target, and not during gender/age judgment tasks [Gunther Moor et al., 2012a], while identical visual actions, presented in different contexts, activate the STS in a different way [Saxe et al., 2004]. This result is in accord with our results that the inferred intentions of the actor, and not its actions per se, are represented in this region. Finally, and more interestingly, previous imaging studies [Gunther Moor et al., 2012b] demonstrated that STS is also activated in the dictator game, in which participants were asked to divide coins between themselves and players who previously included or excluded them. This activation is stronger while subjects are making an offer to players who previously excluded them, thus confirming a role for the posterior temporal region in social exclusion.

Gaze Coding in the Insula

Given its deep position within the sylvian fissure, it is difficult to understand to what extent insular activity contributes to scalp‐recorded ERP. In particular, potentials generated from the insular cortex could be attributed exclusively to the surrounding lateral cortex. It follows that it is difficult to evaluate the contribution of the insular cortex to the N200 generation elicited by gaze shift observation. A possible contribution of the insula to this response is suggested by fMRI studies showing its involvement in gaze processing [Ethofer et al., 2011] as well as, more generally, in social cognition [see Kurth et al., 2010; Lamm and Singer, 2010]. The present data confirm this hypothesis demonstrating that the inferoposterior quadrant of the insula, adjacent to STG, produced an iERP similar to the one from the posterior temporal region and possibly contributes to the scalp‐recorded N200. However, the absence of significant distinctions between different types of gaze shifts suggests a different role for this region of the insula in social cognition. Interestingly recent data from the monkey [Caruana et al., 2011] demonstrated that the intracortical microstimulation (ICMS) of the ventral aspect of the insula, adjacent to the temporal operculum, elicits an affiliative facial gesture (lip‐smacking) only if the stimulation is delivered during the eye‐contact between the monkey and the experimenter, while ICMS of the same site is totally ineffective when eye‐contact is interrupted. Furthermore, these results are in accord with a recent meta‐review of imaging data based on the activation likelihood estimation (ALE) method [Kurth et al., 2010], demonstrating that the inferoposterior sector of the insula appears to be functionally segregated from the rest of the insula and specifically involved in the socio‐emotional domain.

Beside the early iERP at 200ms, a late potential was recorded from the anterior insula. Evoked potentials showing a similar slow negativity were previously recorded from different face‐specific ventral temporal sites [Allison et al., 1999; Puce et al., 1999], and specifically ascribed to an attentional top‐down modulation [Engell et al., 2010]. The rostral localization of the late potential is in line with previous fMRI studies describing the activation of the anterior insula during gaze perception [Ethofer et al., 2011]. Because of technical limitations, imaging studies were not able to specify the precise timing of the insula activation. Nevertheless, the top‐down nature of the anterior insula recruitment in gaze processing has been suggested by these authors.

Spatial Differences in Gaze‐Related GBR and iERP

An interesting result from the present work concerns the different effects obtained by iERP and GBR analysis. In particular, differences in GBR were statistically significant in a reduced number of contacts compared to the number of contacts with significant changes in iERP, circumscribing the gaze region in the posterior MTG of the left hemisphere. How can these different results be interpreted? Previous intracranial recordings from human subjects demonstrated that iERP and high‐frequency activity in the gamma‐band are two different neural markers presenting little spatial overlap [Vidal et al., 2010]. More specifically, previous experiments demonstrated that gamma‐band responses and iERP show different response characteristics in terms of amplitude, latency, and recruitment as well [Engell and McCarthy, 2010; Vidal et al., 2010]. However, the specific functional meaning of each of these markers is still poorly understood. A crucial but unsolved point is whether these markers systematically reflect the same information processing or different information. A possible interpretation of the discrepancy between iERP and GBR is that iERP is constituted by several components of the mass neural activity that can be partly separated by decomposing it into different frequency bands. The high‐frequency activity in the gamma‐band is only one of the possible neural sources of the evoked‐potential; however, this band is thought to be the most informative one [Magri et al., 2012]. While gamma‐band frequency is considered a more focal neural marker, iERP recordings could integrate signals located more than a centimeter away from their origins, and they probably reflect a mixture of local potentials with “volume conducted” potentials from distant sites [Kajikawa and Schroeder, 2011]. Another possible explanation is that, compared to GBR, iERP reflects a less specific recruitment of a given region. These hypotheses can lead to two possible interpretations or our data. Following the first view, a significant activation of both iERP and GBR indicates a more reliable localization of the neural source, leading to the conclusion that only the posterior sector of MTG is involved in the gaze processing. It follows that iERP from STG and the inferoposterior insular are mainly due to volume conducted potentials and, as a consequence, that these regions are not involved in the task. However, this interpretation is in contrast with previous findings that both STG and the insula are activated by gaze shift observation in imaging studies [Puce et al., 1998; Hoffman and Haxby, 2000; Engell and Haxby, 2007; Ethofer et al., 2011; see also Puce and Perrett, 2003]. The second interpretation is that the presence of significant iERP without GBR reflects a weaker and less specific involvement of these regions in the task, possibly dependent on others factors such as selective‐attention [Engell and McCarthy, 2007]. Accordingly, posterior MTG is more strongly involved in gaze shift observation, while the insula activation during the same task is compatible with a more general coding, receiving gaze‐related information from the temporal lobe to modulate socio‐emotional responses [Caruana et al., 2011].