Skip to main content
NCBI home page
Social Cognitive and Affective Neuroscience logoLink to Social Cognitive and Affective Neuroscience
. 2019 Oct 8;14(9):977–986. doi: 10.1093/scan/nsz071

A look at actions: direct gaze modulates functional connectivity of the right TPJ with an action control network

Imme Christina Zillekens 1,2,✉, Lena Maria Schliephake 3, Marie-Luise Brandi 1, Leonhard Schilbach 1,2,4,5
PMCID: PMC6917026  PMID: 31593216

ABSTRACT

Social signals such as eye contact and motor actions are essential elements of social interactions. However, our knowledge about the interplay of gaze signals and the control of actions remains limited. In a group of 30 healthy participants, we investigated the effect of gaze (direct gaze vs averted) on behavioral and neural measures of action control as assessed by a spatial congruency task (spatially congruent vs incongruent button presses in response to gaze shifts). Behavioral results demonstrate that inter-individual differences in condition-specific incongruency costs were associated with autistic traits. While there was no interaction effect of gaze and action control on brain activation, in a context of incongruent responses to direct gaze shifts, a psychophysiological interaction analysis showed increased functional coupling between the right temporoparietal junction, a key region in gaze processing, and the inferior frontal gyri, which have been related to both social cognition and motor inhibition. Conversely, incongruency costs to averted gaze were reflected in increased connectivity with action control areas implicated in top-down attentional processes. Our findings indicate that direct gaze perception inter-individually modulates motor actions and enforces the functional integration of gaze-related social cognition and action control processes, thereby connecting functional elements of social interactions.

Keywords: direct gaze, action control network, spatial compatibility, fMRI, connectivity

Introduction

The interdependency of gaze processing and motor actions plays a key role in our everyday social interactions. Underlining their joint functioning, empirical studies have revealed a strong overlap between brain areas that process hand and gaze movements (e.g. Pierno et al., 2006). Furthermore, it has been shown that social gaze impacts goal-directed movement precision (Becchio et al., 2008) as well as reaction speed (Hietanen et al., 2006; Schilbach et al., 2011). The latter, however, could not be observed in individuals with autism spectrum disorder (Schilbach et al., 2012), who are also characterized by abnormalities in motor behaviors as well as the processing of eyes and observed movements (Marsh and Hamilton, 2011; Tanaka and Sung, 2016).

In social interactions, a specific role needs to be attributed to the perception of direct gaze, which reflexively attracts attention (Friesen and Kingstone, 1998; Bristow et al., 2007). Crucially, direct as compared to averted gaze has been described as a signal that conveys the intention to interact (Senju and Johnson, 2009). In line with this, empirical evidence suggests a facilitation effect of direct gaze on imitative behavior (Wang et al., 2011a; Prinsen et al., 2017) as well as an attentional effect of gaze cues on manual responses to target stimuli (Bristow et al., 2007; Böckler et al., 2014). Yet, besides imitation and beyond attentional guidance of gaze, social interactions might require re-actions to gaze movements that are compatible but not identical with observed actions (Schilbach et al., 2013). Still, how gaze interacts with action control processes on the behavioral and brain level and how the specific gaze context modulates functional connectivity between gaze and action control areas, particularly when tendencies towards spatial congruency need to be suppressed, remains unclear. Therefore, we systematically investigated how the perception of direct or averted gaze affects action control in the context of an fMRI-compatible and previously established spatial stimulus-response compatibility (SSRC) paradigm (Schilbach et al., 2011; Schilbach et al., 2012). Instead of using social and non-social stimuli as in previous studies, we realized a 2 × 2 factorial design by asking participants to generate button presses in a spatially congruent or incongruent manner (factor congruency: CON vs INCON) in response to gaze shifts produced by an anthropomorphic virtual character (VC), whose initial gaze position was either direct or averted (factor gaze: direct vs averted). As dependent variables, we measured task performance (accuracy) and reaction time (RT) as well as brain activity obtained via BOLD fMRI.

In line with empirical evidence, incongruent compared to congruent reactions incur increased computational load and thus, lead to prolonged RTs and a decreased percentage of correct responses (Iacoboni et al., 1996; Hietanen et al., 2006). Additionally, the incongruency effect should be reflected in an increased activation in a bilateral dorsal fronto-parietal network of frontal motor areas and superior parietal lobules, a network responsive to increased top-down attentional demands and need for increased action control (Corbetta and Shulman, 2002; Cieslik et al., 2015). For the main effect of direct compared to averted gaze, we hypothesized brain regions sensitive to eye contact and gaze-related movements (Haxby et al., 2000; Schobert et al., 2018), namely the temporoparietal junction/posterior sulcus temporalis superior (TPJ/pSTS) and the fusiform gyrus, to show increased BOLD signal in response to direct gaze stimuli (Calder et al., 2002; Pelphrey et al., 2003; Senju and Johnson, 2009).

The main focus of this study was to investigate the interaction between the perception of gaze and mechanisms of action control. While some evidence suggests a general facilitation effect of direct gaze (Schilbach et al., 2011, 2012), in other studies, an association of direct gaze and accelerated reactions has only been found for compatible stimulus-response mappings (Bristow et al., 2007; Böckler et al., 2014; Prinsen et al., 2017). On the brain level, both motor control areas such as the inferior frontal cortex as well as the gaze sensitive TPJ have been implicated in the interaction of gaze and motor control processes (Schilbach et al., 2011; Wang et al., 2011b). Building on this, the present study tested whether the same brain regions are differentially recruited as representations of gaze-dependent incongruency costs. Furthermore, in light of evidence that indicates gaze-dependent functional connectivity changes of the TPJ/pSTS with an extended gaze perception network (Nummenmaa et al., 2010) as well as multi-modal functional coupling of the right TPJ (Bzdok et al., 2013), we expected gaze and action control networks to interact at the level of right TPJ connectivity, reflecting a differential integration of gaze-related and action control processes. Thus, in order to systematically investigate the relationship of gaze-specific incongruency costs in terms of functional connectivity, we conducted a psychophysiological interaction analysis and analyzed whether the interplay of the gaze context and action control demands modulates the functional connectivity between the right TPJ and an ‘action control network’, being composed of all action-associated brain regions as defined by a Neurosynth (Yarkoni et al., 2011) search including the search term ‘action’. In a context of direct gaze and an increased demand for action inhibition due to spatial incongruence, we expected to see increased functional coupling between our seed region, which was located in a functional cluster that has been related to social cognition (Bzdok et al., 2013), and particularly the IFG, indicating an integration of gaze-related social cognition and action control (Schilbach et al., 2011; Wang and Hamilton, 2012; Callejas et al., 2014).

In light of autism-related differences observed in the original version of our SSRC task (Schilbach et al., 2012), we further obtained measures of autistic traits and hypothesized to replicate a positive relationship between autistic traits and gaze-specific incongruency costs.

Methods

Thirty-two volunteers (15 females) participated in our study. Due to neurological and psychiatric conditions (sleeping disorder, ventricumegaly), two participants were excluded from all further analyses. The remaining 30 participants (14 females) had a mean age of 24 (s.d. = 5.08, range = 19–41), normal or corrected-to-normal vision, no history of neurological or psychiatric history and were right-handed as assessed by the Edinburgh Handedness Inventory (Oldfield, 1971). The mean group autism quotient (AQ) (Baron-Cohen et al., 2001) was 14.99 (s.d. = 6.38, range = [6, 32]). All participants gave informed written consent and received a fixed monetary compensation of 30€. At the end of the experiment, participants were debriefed and thanked for their participation. The study protocol followed the guidelines of the Declaration of Helsinki and was approved by the ethics committee of the Ludwig-Maximilians-Universität München.

Experimental design and procedure

The paradigm used in this fMRI study resembled an adapted version of previously used SSRC paradigm (Schilbach et al., 2011; Schilbach et al., 2012). Instead of asking participants to respond to the gaze movement of an anthropomorphic VC or the movement of a geometric symbol as in previous studies, VCs were always present. This allowed us to keep the social stimulus constant while now systematically manipulating exposures to direct compared to averted gaze.

Before the experiment and before entering the fMRI scanner, participants received detailed instructions on the overall procedure and MRI safety. During the experiment, they were asked to respond as fast as possible to gaze shifts shown by the VC by pressing a right or left button using the right or left index finger, respectively. The experiment consisted of 24 blocks of 12 events each with 50% left- and 50% right-directional gaze shifts, realizing a 2 × 2 factorial design: congruent blocks were instructed by the initial cue ‘GLEICH’ (German for ‘same’) and required participants to respond to gaze shifts in a spatially compatible manner, i.e. pressing the ipsilateral button. The initial cue ‘GEGEN’ (German for ‘opposite’) introduced blocks of spatially incompatible responses, where participants had to press the contralateral button in response to a gaze shift, for instance the right button had to be pressed following a gaze shift to the left (Figure 1). Each cue was presented once for 1500 ms at the beginning of each block of 12 gaze shifts and each block was followed by a jittered inter-stimulus interval of 15 to 17 s. During the experiment, participants did not receive any feedback on their performance. Besides the factor ‘congruency’, our second experimental factor ‘gaze’ was expressed by the VC either looking up (averted gaze) or facing the participant (direct gaze). Pixel coordinates and the timing of gaze shifts were identical over all conditions. In each block, male participants experienced one of two male VCs while female participants were confronted with one of two female VCs. The appearance of either of the two same sex VCs was equally likely. Stimuli were presented through the software package Presentation (Neurobehavioral Systems, Inc.; Version 18.1) on an fMRI compatible computer monitor (refresh rate = 59 Hz, resolution of 1024 × 768, viewable region of 500 mm × 380 mm) and were created manually in Poser 10 (Smith Micro Software, Inc., CA, USA). As stimuli of the present study differed from stimuli of previous studies, a pre-study was conducted to control for unbalanced stimuli preferences. Twelve volunteers (employees, 8 females) from the Max Planck Institute of Psychiatry rated all four VCs on a five-level Likert scale on attractiveness, valence, arousal and other characteristics (Supplementary Table S1). A repeated measures ANOVA using stimulus type (VC 1–4) and characteristics (Supplementary Table S1) revealed no significant effect of stimulus type (F[1,11] = 0.94, P = 0.43) or interaction effect on VC ratings, F(1,11) = 1.15, P = 0.10. All volunteers correctly indicated whether the VC demonstrated direct or averted gaze and whether the gaze was directed to the left or right.

Fig. 1.

Fig. 1

Open in a new tab

Experimental task. (A) One of two female VCs demonstrating direct gaze in the congruent condition [indicated by the initial cue ‘GLEICH’ (German for ‘same’)]. The first gaze shift to the left requires a congruent left button press, the second gaze shift to the right a right button press. (B) One of two male VCs demonstrating averted gaze in the incongruent condition [indicated by the initial cue ‘GEGEN’ (German for ‘opposite’)]. The first gaze shift to the left requires an incongruent right button press, the second gaze shift to the right a left button press.

Behavioral and questionnaire data preprocessing

RTs, the dependent variable that expressed the behavioral outcome of action control, reflected the time frame between the onset of the gaze shift and the button press of the participant. We applied the following RT data preprocessing steps (e.g. Engell et al., 2010; Böckler et al., 2014): trials with no answer, multiple answers or incorrect answers were categorized as error trials. Further, trials with responses two standard deviations from the participant-specific mean RT over all conditions were interpreted as anticipation error or missed response and also labeled as error trials. In total, 9.4% of all trials were error trials. In order to exclude uninformative task blocks, e.g. blocks in which participants missed the initial instructive cue, blocks with more than 25% error trials (> = 3 error trials/block) were not considered in subsequent analyses, resulting in an average exclusion of one block per participant (Supplementary Table S2 for details). Task performance reflected the mean percentage of correctly answered trials of all correct and error trials, which was calculated for each combination of experimental conditions. The AQ of participants was assessed in order to evaluate the relationship of autistic traits and gaze-specific incongruency costs. To conserve comparability of AQ scores, missing values (four participants did not fill in one item each) were interpolated over the individual sub-scale values of the respective item filling in the missing data point.

Behavioral data analyses

Main effects and interaction effects of experimental conditions on task performance and RTs were tested by means of repeated measures 2 (gaze: direct vs averted) × 2 (congruency: congruent vs incongruent) ANOVAs. To test whether direct gaze modulates responses in the congruent or incongruent condition, we implemented post-hoc contrasts of conditions (direct_CON vs averted_CON; direct_INCON vs averted_INCON) as Bonferroni corrected paired two-sided t-tests. After calculating the RT incongruency costs, i.e. RT slowing in incongruent compared to congruent trials, we obtained the difference in RT incongruency costs between the direct and averted gaze condition (incongruency costs direct—incongruency costs averted) as a measure of direction and size of effect of gaze on RT incongruency costs. To further analyze the relationship of the difference in RT incongruency costs between the direct and the averted gaze condition, we correlated the measure with AQ scores. Here, due to non-normally distributed AQ scores (Shapiro–Wilk statistic = 0.93, P < 0.05), the non-parametric two-sided Spearman’s rank correlation statistic was used.

fMRI data analysis

Participants completed the experiment inside a 3T MR scanner (MR750, GE, Milwaukee, USA). The procedure comprised a single functional run of 290 volumes of 40 slices (32-channel head coil, AC-PC-orientation, 96 × 96 matrix, 3 × 3 mm voxel size, 3 mm slice thickness, 0.5 mm slice gap). First, structural T1-weighted images were acquired [BRAVO FSPGR pulse sequence, 1 mm isotropic voxels, repetition time (TR) of 6.2 ms, echo time (TE) of 2.3 ms]. Second, during the experiment, T2*-weighted functional images were obtained by means of gradient echo planar imaging (TR of 2000 ms, TE of 20 ms, 90° flip angle) and the first four functional volumes we removed to control for non-equilibrium effects. FMRI data preprocessing and analysis were performed in SPM12 (Statistical Parametric Mapping Software, Wellcome Department of Imaging Neuroscience, London; http://www.fil.ion.ucl.ac.uk) and included the following steps: functional images were spatially realigned to the mean functional image (rigid body transformation). Next, functional and structural images were co-registered. Both structural and functional images were spatially normalized to the Montreal Neurological Institute (MNI) template using tissue segmented T1-weighted anatomical images (BRAVO FSPGR pulse sequence, 1 mm isotropic voxels, TR of 6.2 ms, TE of 2.3 ms). Functional images were resliced to 2 × 2 × 2 mm voxel size. Finally, a 3D Gaussian Kernel with full width of half maximum of 8 mm was used for smoothing.

All valid experimental blocks (RT data preprocessing) were modeled as epochs in a general linear model (GLM) with an average duration of 54 s (range 46–64 s). Experimental factors, i.e. ‘gaze’ (direct vs averted gaze) and ‘congruency’ (congruent vs incongruent) were captured in four different regressors of interest. Error blocks were modeled by a regressor of no interest. Our GLM design matrix further contained 26 confound regressors of no interest: the first 24 contained six z-standardized rigid body motion realignment parameters, their temporal derivatives and the squared values of both realignment parameters and derivatives (Friston et al., 1996). Another two regressors captured confounding signal from white matter and cerebrospinal fluid. Here, we obtained a binarized mask from the respective segmented individual structural images using a 0.95 threshold in SPM’s image calculator (imcalc tool) and calculated the first principal component of the respective tissue type, explaining 85.42% (s.d. = 4.28%) and 79.11% (s.d. = 5.85%) of variance in the signal (Caballero-Gaudes and Reynolds, 2017). No global scaling was applied and low-frequency signal drifts were filtered out (128 s cutoff period). In order to correct for temporal autocorrelation of the data, voxel-wise maximum likelihood estimators were calculated (Kiebel and Holmes, 2004).

Studying the effect of congruency and gaze as well as their interaction, BOLD signal during main effects and interactions of conditions were analyzed in a second-level flexible factorial design. A binarized group-specific explicit grey matter (GM) mask (sum of participant specific probability of GM > 0.05; imcalc tool) contained all voxels of interest. Besides our two experimental factors, we added a ‘subject’ factor, accounting for subject-specific heteroscedasticity, and implemented SPM’s default settings of unequal variances within each factor. In order to analyze the main effects of congruency and gaze, we contrasted congruent and incongruent as well as direct and averted gaze conditions [congruency: (direct_CON + averted_CON) > (direct_INCON + averted_INCON), (direct_INCON + averted_INCON) > (direct_CON + averted_CON); gaze: (direct_CON + direct_INCON) > (averted_CON + averted_INCON), (averted_CON + averted_INCON) > (direct_CON + direct_INCON)]. Statistical interactions of conditions were modeled as contrast of incongruency costs in the direct and averted gaze conditions [IA1: (direct_INCON > direct_CON) > (averted_INCON > averted_CON), IA2: (averted_INCON > averted_CON) >(direct_INCON > direct_CON)].

Moreover, we conducted a generalized condition-specific psychophysiological interaction analysis (McLaren et al., 2012) to investigate the context-dependent functional coupling between gaze and action processing areas. Based on the available literature and a term-based meta-analysis in Neurosynth (Yarkoni et al., 2011), we identified a region typically labelled as right TPJ (Schurz et al., 2017) as the seed region for our gPPI analysis. The coordinates of our seed region [44, −52, 12] represented the peak coordinates in the brain map of the term ‘gaze’ (retrieved 2 October 2018 from www.neurosynth.org, z-score = 7.33) and were further situated in a functional sub-section of the right TPJ involved in social cognition (Bzdok et al., 2013; Neurosynth, retrieved 3 June 2019, meta-analytic association of peak coordinates with terms ‘default network’, ‘mentalizing’). After creating a sphere of 6 mm radius in marsbar (Brett et al., 2002; Supplementary Figure S1A), we extracted the first eigenvariate of our seed sphere and allowed actual ROIs to vary in size between participants, but restricted them to first level masks generated by SPM12. In order to investigate the context-dependent functional coupling of our right TPJ seed with brain areas involved in action control, we retrieved an associative ‘action’ mask from Neurosynth (Yarkoni et al., 2011: retrieved 2 October 2018 from www.neurosynth.org). After smoothing (3D Gaussian Kernel with full width of half maximum of 4 mm) and binarization (imcalc, i1 > 0.1, Supplementary Figure S1B), it was implemented as explicit mask in our second level analysis. Specifically, we were interested in the functional coupling of the right TPJ and the action network for the statistical interactions of our experimental conditions [IA1: (direct_INCON > direct_CON) > (averted_INCON > averted_CON)] and [IA2: (averted_INCON > averted_CON) > (direct_INCON > direct_CON)].

Statistical maps of the activation analysis are shown at a cluster-forming threshold of P < 0.001 (uncorrected) and a cluster threshold of P < 0.05 (FWE). In the psychophysiological interaction analysis, P-values were thresholded at P < 0.05 (FWE) at voxel level. The Anatomy Toolbox (Eickhoff et al., 2005; Version 2.2c) and the AAL atlas in MRIcron (Rorden and Brett, 2000) were used for functional localization and the Surf Ice software for brain visualizations (https://www.nitrc.org/projects/surfice/).

Results

Behavioral results

As expected, a repeated measures ANOVA focusing on the condition-specific performance revealed a significant main effect of congruency on the percentage of correct responses, F(29,1) = 32.09, P < 0.001, ηp2 = 0.53. There was no main effect of gaze [F(29,1) = 1.80, P = 0.19, ηp2 = 0.06] or an interaction effect of congruency and gaze on performance, F(29,1) = 1.27, P = 0.27, ηp2 = 0.04 (Figure 2A).

Fig. 2.

Fig. 2

Open in a new tab

Behavioral measures. (A) Mean percentage of correct responses and (B) left panel: mean RTs; right panel: mean RTs for direct and averted gaze in the congruent condition. Light blue lines mark a decrease in RT from direct to averted; dark blue lines mark an increase. The light blue solid line represents the mean decrease in RTs from the direct to the averted gaze condition. Black horizontal lines represent the mean values, boxes represent the standard error of the mean (SEM), blue vertical lines the standard deviation (s.d.).

A second repeated measures ANOVA demonstrated a significant main effect of congruency also on RTs, F(29,1) = 134.71, P < 0.001, ηp2 = 0.82. Neither did gaze impact participants’ RTs [F(29,1) = 1.30, P = 0.26, ηp2 = 0.04] nor did the interaction effect of experimental conditions reach significance, F(29,1) = 3.88, P = 0.06, ηp2 = 0.12 (Figure 2B). Post-hoc contrasts showed that congruent RTs were significantly higher in the direct gaze compared to the averted gaze condition, t(29) = 2.86, P < 0.01, R2 = 0.22. Incongruent RTs, however, did not differ between gaze conditions, t(29) = 0.17, P = 0.87. Table 1 presents the condition-specific performance and RTs.

Table 1.

Condition-specific RTs and accuracy. Brackets contain the standard deviation (s.d.).

Gaze condition Experimental condition Mean RT Mean performance (percentage of correct trials)
Direct Congruent 412.79 (44.35) 92.87 (6.53)
Incongruent 461.02 (49.37) 83.24 (9.33)
Averted Congruent 405.82 (42.15) 93.13 (6.05)
Incongruent 461.70 (52.23) 86.57 (10.36)
Open in a new tab

Condition-specific RT incongruency costs, i.e. the increase in RTs in incongruent compared to congruent trials, are displayed in Figure 3. On average, RTs of incongruent reactions increased by 48 ms (s.d. = 27 ms) in the direct gaze condition and by 56 ms (s.d. = 27 ms) in the averted gaze condition. Building on this, a two-sided Spearman’s rank correlation analysis indicated a significant negative correlation between AQ scores and the difference in RT incongruency costs for direct as compared to averted gaze, rs(28) = −0.40, P < 0.05 (Figure 4).

Fig. 3.

Fig. 3

Open in a new tab

Condition-specific RT incongruency costs. Boxes represent the SEM.

Fig. 4.

Fig. 4

Open in a new tab

Linear association of AQ scores and the difference in RT incongruency costs (ranks) between experimental conditions (direct—averted).

fMRI results

Applying a cluster-forming threshold of P < 0.001 (uncorrected) and a cluster threshold of P < 0.05 (FWE), incongruent contrasted to congruent trials [(direct_INCON + averted_INCON) > (direct_CON + averted_CON)] were associated with a differential increase in BOLD signal in the right inferior parietal lobule, left superior parietal lobule and right middle frontal gyrus (Figure 5A). For the reversed contrast [(direct_CON + averted_CON) > (direct_INCON + averted_INCON)], a large cluster of 2319 voxels emerged in the bilateral medial prefrontal cortex (MPFC), including voxels in the superior medial gyri, superior frontal gyri and the anterior cingulate cortices, spreading to the right medial cingulate cortex. Congruent compared to incongruent trials further elicited activation in the right IFG as well as the left cerebellum and posterior part of the left fusiform gyrus (Figure 5B).

Fig. 5.

Fig. 5

Open in a new tab

Main effects of conditions in the left (L) and right (R) hemisphere. (A) Incongruent vs congruent, (B) congruent vs incongruent, (C) direct vs averted gaze. The cluster forming threshold was set to P < 0.001 (uncorrected), the cluster threshold to P < 0.05 (FWE) and cluster size (A) k > 414 voxels, (B) k > 287 voxels, (C) k > 638 voxels. [(A) SPL: superior parietal lobule, mFG: medial frontal gyrus; MFG: middle frontal gyrus (B) SFC: superior frontal cortex, FFG: fusiform gyrus, ACC: anterior cingulate cortex, MPFC: medial prefrontal cortex, CRBL: cerebellum; IFG: inferior frontal gyrus (C) lPS: intra-parietal sulcus].

During direct gaze vs averted gaze [(direct_CON + direct_INCON) > (averted_CON + averted_INCON)], increased signal was found in the right intraparietal sulcus (Figure 5C). The contrast of averted gaze vs direct gaze [(averted_CON + averted_INCON) > (direct_CON + direct_INCON)] did not show any suprathreshold activation. Similarly, significant clusters emerged in neither of the interactions of congruency and gaze [IA1: (direct_INCON > direct_CON) > (averted_INCON > averted_CON)] and [IA2: (averted_INCON > averted_CON) > (direct_INCON > direct_CON)] (Supplementary Table S3 for coordinates, T-values and cluster sizes).

In our psychophysiological interaction analysis, we analyzed how the right TPJ was coupled with the action network for the interactions of the experimental factors, i.e. IA1 and IA2 (Supplementary Figure S2 and Table S4 for coordinates, T-values and cluster sizes of all PPI contrasts). Statistical maps were thresholded at P < 0.05 (FWE) at voxel level. Results demonstrated that for IA1, which represented increased BOLD incongruency costs for direct compared to averted gaze, the right TPJ showed context-dependent connectivity with the IFG and the right middle temporal gyrus (Figure 6, brown color map). For IA2, reflecting increased BOLD incongruency costs for averted compared to direct gaze, activation in the seed region was coupled to activation in a dorsal network of superior and inferior parietal lobules, pre- and postcentral gyri, temporal gyri, occipital gyri, left superior, posterior medial, middle and IFG, right paracentral gyrus, left putamen and right cerebellum (Figure 6, blue/green color map).

Fig. 6.

Fig. 6

Open in a new tab

Interaction effects in a psychophysiological interaction analysis in the left (L) and right (R) hemisphere. IA1 (brown): (direct_INCON > direct_CON) > (averted_INCON > averted_CON), IA2 (blue/green): (averted_INCON > averted_CON) > (direct_INCON > direct_CON). The results were FWE corrected at P < 0.05 voxel level. [IFG: inferior frontal gyrus, MTG: middle temporal gyrus MFG: middle frontal gyrus, PreCG: precentral gyrus, PostCG: postcentral gyrus, SPL: superior parietal lobule, OccG: occipital gyrus, CRBL: cerebellum, ITG: inferior temporal gyrus; pmFG: posterior medial frontal gyrus; ParaCG: paracentral gyrus].

Discussion

The present study investigated the effect of gaze perception on behavioral and neural correlates of action control of non-imitative re-actions. Our results demonstrate context-dependent functional integration of gaze and action control processes and our behavioral findings are in line with theories suggesting a relationship between gaze effects and autistic traits.

As hypothesized, we found a significantly lower percentage of correct responses and longer RTs when participants had to respond in a spatially incompatible manner to the VCs’ gaze shifts (Iacoboni et al., 1996; Hietanen et al., 2006; Schilbach et al., 2012). Moreover, in line with a priori expectations, key regions of the so-called dorsal fronto-parietal attention network showed increased activation in incongruent as compared to congruent experimental blocks, possibly reflecting the increased need for top-down control (Corbetta and Shulman, 2002; Cieslik et al., 2015).

The opposite contrast, namely congruent vs incongruent, depicted increased brain activation in the left posterior fusiform gyrus and the MPFC. Given the lack of significant results in previous studies, we did not have specific hypotheses about the present contrast. A possible explanation for the brain activation found might be that similar to the sensitivity of the left fusiform gyrus towards faces and shapes (Kourtzi and Kanwisher, 2001), MPFC activation has previously been found in response to spatially congruent gaze shifts, potentially representing occurrences of joint attention (Bristow et al., 2007). Hence, in a situation of low task difficulty, participants might have used free cognitive capacities to thoroughly process the social encounter with a VC (Kanwisher et al., 1997). Alternatively, representing a central hub of the default mode network, which is known as the task-negative network (e.g. Fox et al., 2005), MPFC activation might indicate the occurrence of stimulus-independent thoughts that have been referred to as ‘day dreaming’ or mind wandering (Mason et al., 2007, Neurosynth, retrieved 3 June 2019: association of peak coordinates with terms ‘default mode’, ‘mentalizing’). Despite of the richness of literature on the decisive role of the IFG in response inhibition processes (e.g. Zhang et al., 2017), in the present contrast, the right IFG was activated during congruent blocks not requiring to withdraw from or cancel motor actions. Instead, the IFG might have come into play through holding representations of the CV’s gaze movements and hence, might have supported action understanding (Rizzolatti and Craighero, 2004; Casartelli and Molteni, 2014; Neurosynth, retrieved 3 June 2019: association of peak coordinates with terms ‘decision task, ‘comprehension’, ‘reappraisal’). Further, in light of its implication in gaze-grasping mappings (Bowman et al., 2009; Ambrosini et al., 2011), the IFG might have promoted a congruent button pressing by translating the gaze movement into a finger movement that corresponded to the direction of the gaze. Here, future research needs to clarify the specific role of the IFG during congruent task conditions.

Direct compared to averted gaze was followed by increased brain activation in the right intraparietal sulcus, a region known to be involved in visuo-spatial aspects of action planning, the understanding of complex or irrational actions and the integration of visual and motor computations (Rizzolatti and Matelli, 2003; Fogassi and Luppino, 2005; Marsh and Hamilton, 2011). However, contrary to our hypotheses, direct gaze was not accompanied by increased activation in the right TPJ and fusiform gyrus—a result that might be caused by block design-induced habituation effects (Bruno et al., 2014).

Incongruency costs describe the behavioral or neural cost of performing a spatially incompatible motor response. In the present study, we were interested in the differences in incongruency costs between the direct and averted gaze conditions. Contrary to our hypothesis, incongruent RTs did not differ between the direct and the averted gaze condition. As has been shown previously (Wang et al., 2011a), in a more difficult task situation, gaze did not have an impact on behavior. However, contrary to the reported facilitation of motor imitation with direct gaze, in our study, the translation of a gaze shift into a left- or right-handed button press was less time-consuming for averted gaze movements. Thus, our results indicate that the facilitation effect of direct gaze might not apply to non-imitative behaviors. Consistent with behavioral results, there was no interaction effect of experimental conditions at the brain level.

The difference in RT incongruency costs between the direct and averted gaze condition represented a measure of the gaze-depended incongruency effect on reaction speed. A correlation analysis showed that high AQ values were associated with higher incongruency costs in the averted gaze condition, whereas the difference in incongruency costs between conditions diminished and even changed towards higher incongruency costs in the direct gaze condition with decreasing AQ scores. This result points towards inter-individual differences in the sensitivity towards social gaze, as a function of autistic traits. In this sense, individuals with low AQ scores might be more susceptible to the influence of direct gaze than individuals with higher AQ values.

How the communication between the right TPJ and the action network changes depending upon the interplay of the experimental factors was addressed by means of a psychophysiological interaction analysis. Importantly, studies have indicated a functional partitioning of the right TPJ into an anterior and a posterior cluster: while the global functional integration of the anterior cluster suggests a mediating role in shifting from one functional brain state to another (Kernbach et al., 2018), our ‘gaze’-associated seed region overlaps with the posterior TPJ cluster, implicated in social cognition, imagination and episodic memory retrieval (Bzdok et al., 2013). As hypothesized, the context-dependent connectivity between our seed and the IFG, known to be involved in the integration of action inhibitory tendencies and motivational, emotional or social input (e.g. Schulz et al., 2009; Chen et al., 2018; Neurosynth, retrieved 3 June 2019: association of peak coordinates with term ‘theory of mind’), was increased for incongruency costs in the context of direct gaze. As a consequence, the connection between the right TPJ and the IFG might reflect an upregulated exchange of gaze information and inhibitory control processes in the context of direct gaze. Moreover, in parallel to the association of our TPJ region to object or scenic imagination (Bzdok et al., 2013), the IFG has been discussed not only to contribute to reactive but also proactive motor control (Aron, 2011; Di Russo et al., 2016). Accordingly, it would be possible that the IFG has been involved in preparing or anticipating a reorientation response that might have been supported by gaze-related input from the TPJ. In line with this post-hoc hypothesis, the right middle temporal gyrus has been indicated in mapping hypothetical motor actions to perceptual input (Hashimoto and Sakai, 2003).

Conversely, costs for reacting incongruently in the context of averted as compared to direct gaze movements were represented in increased functional connectivity between our seed region and major parts of the action network, predominantly in the left hemisphere and including the parietal lobules, the primary motor and sensory cortex, the frontal and temporal gyri. Besides belonging to the action network, the superior parietal and frontal regions are also relevant in top down attentional control processes (Corbetta and Shulman, 2002; Vossel et al., 2012) and have been shown activated in working memory tasks, during spatial attention towards or the planning of actions (Ptak et al., 2017). In summary, incongruency costs for averted gaze appear to manifest in more wide-spread connectivity that encompasses somatosensory motor areas. Incongruency costs for direct gaze, however, are reflected in increased connectivity with brain regions that are involved in both action control and social cognition.

In conclusion, the results of the present study shed new light onto the neurobiology that underlies the specific role of direct gaze in social encounters: by increasing the connectivity of multimodal brain regions, the processing of direct gaze results in an integration of brains regions implicated in action control and social cognition. In this way, direct gaze could be seen as contributing to a comprehensive processing of the social situation that goes beyond a strongly stimulus-driven orientation.

Funding

L.S. was supported by a grant for an Independent Max Planck Research Group by the Max Planck Society.

Conflict of interest. The authors declare no competing financial interests.

References

  1. Ambrosini E., Costantini M., Sinigaglia C. (2011). Grasping with the eyes. Journal of Neurophysiology, 106, 1437–42. [DOI] [PubMed] [Google Scholar]
  2. Aron A.R. (2011). From reactive to proactive and selective control: developing a richer model for stopping inappropriate responses. Biological Psychiatry, 69, e55–68. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baron-Cohen S., Wheelwright S., Skinner R., et al. (2001). The autism-spectrum quotient (AQ): evidence from Asperger syndrome/high-functioning autism, males and females, scientists and mathematicians. Journal of Autism and Developmental Disorders, 31, 5–17. [DOI] [PubMed] [Google Scholar]
  4. Becchio C., Bertone C., Castiello U. (2008). How the gaze of others influences object processing. Trends in Cognitive Sciences, 12, 254–8. [DOI] [PubMed] [Google Scholar]
  5. Böckler A., van der Wel R.P.R.D., Welsh T.N. (2014). Catching eyes. Psychological Science, 25, 720–7. [DOI] [PubMed] [Google Scholar]
  6. Bowman M.C., Johannson R.S., Flanagan J.R., et al. (2009). Eye–hand coordination in a sequential target contact task. Experimental Brain Research, 195, 273–83. [DOI] [PubMed] [Google Scholar]
  7. Brett M., Anton J., Valabregue R., et al. (2002). Region of interest analysis using an SPM toolbox [abstract]. Presented at the 8th International Conference on Functional Mapping of the Human Brain, June 2-6, 2002, Sendai, Japan. Available on CD-ROM in NeuroImage, Vol 16, No 2. [Google Scholar]
  8. Bristow D., Rees G., Frith C.D. (2007). Social interaction modifies neural response to gaze shifts. Social Cognitive and Affective Neuroscience, 2, 52–61. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bruno J.L., Garrett A.S., Quintin E.-M., et al. (2014). Aberrant face and gaze habituation in fragile X syndrome. American Journal of Psychiatry, 171, 1099–106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bzdok D., Langner R., Schilbach L., et al. (2013). Characterization of the temporo-parietal junction by combining data-driven parcellation, complementary connectivity analyses, and functional decoding. NeuroImage, 81, 381–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Caballero-Gaudes C., Reynolds R.C. (2017). Methods for cleaning the BOLD fMRI signal. NeuroImage, 154, 128–49. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Calder A.J., Lawrence A.D., Keane J., et al. (2002). Reading the mind from eye gaze. Neuropsychologia, 40, 1129–38. [DOI] [PubMed] [Google Scholar]
  13. Callejas A., Shulman G.L., Corbetta M. (2014). Dorsal and ventral attention systems underlie social and symbolic cueing. Journal of Cognitive Neuroscience, 26, 63–80. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Casartelli L., Molteni M. (2014). Where there is a goal, there is a way: what, why and how the parieto-frontal mirror network can mediate imitative behaviours. Neuroscience and Biobehavioral Reviews, 47, 177–93. [DOI] [PubMed] [Google Scholar]
  15. Chen T., Becker B., Camilleri J., et al. (2018). A domain-general brain network underlying emotional and cognitive interference processing: evidence from coordinate-based and functional connectivity meta-analyses. Brain Structure and Function, 223, 3813–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Cieslik E.C., Mueller V.I., Eickhoff C.R., et al. (2015). Three key regions for supervisory attentional control: evidence from neuroimaging meta-analyses. Neuroscience and Biobehavioral Reviews, 48, 22–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Corbetta M., Shulman G.L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience, 3, 215–29. [DOI] [PubMed] [Google Scholar]
  18. Di Russo F., Lucci G., Sulpizio V., et al. (2016). Spatiotemporal brain mapping during preparation, perception, and action. NeuroImage, 126, 1–14. [DOI] [PubMed] [Google Scholar]
  19. Eickhoff S.B., Stephan K.E., Mohlberg H., et al. (2005). A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data. NeuroImage, 25, 1325–35. [DOI] [PubMed] [Google Scholar]
  20. Engell A.D., Nummenmaa L., Oosterhof N.N., et al. (2010). Differential activation of frontoparietal attention networks by social and symbolic spatial cues. Social Cognitive and Affective Neuroscience, 5, 432–40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Fogassi L., Luppino G. (2005). Motor functions of the parietal lobe. Current Opinion in Neurobiology, 15, 626–31. [DOI] [PubMed] [Google Scholar]
  22. Fox M.D., Snyder A.Z., Vincent J.L., et al. (2005). From the cover: the human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proceedings of the National Academy of Sciences, 102, 9673–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Friesen C.K., Kingstone A. (1998). The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin and Review, 5, 490–5. [Google Scholar]
  24. Friston K.J., Williams S., Howard R., et al. (1996). Movement-related effects in fMRI time-series. Magnetic Resonance in Medicine, 35, 346–55. [DOI] [PubMed] [Google Scholar]
  25. Hashimoto Y., Sakai K.L. (2003). Brain activations during conscious self-monitoring of speech production with delayed auditory feedback: an fMRI study. Human Brain Mapping, 20, 22–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Haxby J.V., Hoffman E.A., Gobbini M.I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4, 223–33. [DOI] [PubMed] [Google Scholar]
  27. Hietanen J.K., Nummenmaa L., Nyman M.J., et al. (2006). Automatic attention orienting by social and symbolic cues activates different neural networks: an fMRI study. NeuroImage, 33, 406–13. [DOI] [PubMed] [Google Scholar]
  28. Iacoboni M., Woods R.P., Mazziotta J.C. (1996). Brain-behavior relationships: evidence from practice effects in spatial stimulus-response compatibility. Journal of Neurophysiology, 76, 321–31. [DOI] [PubMed] [Google Scholar]
  29. Kanwisher N., McDermott J., Chun M.M. (1997). The fusiform face area: a module in human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17, 4302–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kernbach J.M., Yeo B.T.T., Smallwood J., et al. (2018). Subspecialization within default mode nodes characterized in 10 000 UK Biobank participants. Proceedings of the National Academy of Sciences of the United States of America, 115, 12295–300. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kiebel S., Holmes A. (2004). In: Frackowiak K., Friston K., Frith C., et al., editors. The general linear model, Human Brain Function 2, London: Elsevier, pp. 725–60. [Google Scholar]
  32. Kourtzi Z., Kanwisher N. (2001). Representation of perceived object shape by the human lateral occipital complex. Science, 293, 1506–9. [DOI] [PubMed] [Google Scholar]
  33. Marsh L.E., Hamilton A.F.d.C. (2011). Dissociation of mirroring and mentalising systems in autism. NeuroImage, 56, 1511–9. [DOI] [PubMed] [Google Scholar]
  34. Mason M.F., Norton M.I., Van Horn J.D., et al. (2007). Wandering minds: the default network and stimulus-independent thought. Science, 315, 393–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. McLaren D.G., Ries M.L., Xu G., et al. (2012). A generalized form of context-dependent psychophysiological interactions (gPPI): a comparison to standard approaches. NeuroImage, 61, 1277–86. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Nummenmaa L., Passamonti L., Rowe J., et al. (2010). Connectivity analysis reveals a cortical network for eye gaze perception. Cerebral Cortex (New York, N.Y.: 1991), 20, 1780–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Oldfield R.C. (1971). The assessment and analysis of handedness: the Edinburgh inventory. 9, 97–113. [DOI] [PubMed] [Google Scholar]
  38. Pelphrey K.A., Singerman J.D., Allison T., et al. (2003). Brain activation evoked by perception of gaze shifts: the influence of context. Neuropsychologia, 41, 156–70. [DOI] [PubMed] [Google Scholar]
  39. Pierno A.C., Becchio C., Wall M.B., et al. (2006). When gaze turns into grasp. Journal of Cognitive Neuroscience, 18, 2130–7. [DOI] [PubMed] [Google Scholar]
  40. Prinsen J., Bernaerts S., Wang Y., et al. (2017). Direct eye contact enhances mirroring of others’ movements: a transcranial magnetic stimulation study. Neuropsychologia, 95, 111–8. [DOI] [PubMed] [Google Scholar]
  41. Ptak R., Schnider A., Fellrath J. (2017). The dorsal frontoparietal network: a core system for emulated action. Trends in Cognitive Sciences, 21, 589–99. [DOI] [PubMed] [Google Scholar]
  42. Rizzolatti G., Craighero L. (2004). The mirror-neuron system. Annual Review of Neuroscience, 27, 169–92. [DOI] [PubMed] [Google Scholar]
  43. Rizzolatti G., Matelli M. (2003). Two different streams form the dorsal visual system: anatomy and functions. Experimental Brain Research, 153, 146–57. [DOI] [PubMed] [Google Scholar]
  44. Rorden C., Brett M. (2000). Stereotaxic display of brain lesions. Behavioural Neurology, 12, 191–200. [DOI] [PubMed] [Google Scholar]
  45. Schilbach L., Eickhoff S.B., Cieslik E., et al. (2011). Eyes on me: an fMRI study of the effects of social gaze on action control. Social Cognitive and Affective Neuroscience, 6, 393–403. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Schilbach L., Eickhoff S.B., Cieslik E.C., et al. (2012). Shall we do this together? Social gaze influences action control in a comparison group, but not in individuals with high-functioning autism. Autism, 16, 151–62. [DOI] [PubMed] [Google Scholar]
  47. Schilbach L., Timmermans B., Reddy V., et al. (2013). Toward a second-person neuroscience. Behavioral and Brain Sciences, 36, 393–414. [DOI] [PubMed] [Google Scholar]
  48. Schobert A.-K., Corradi-Dell’Acqua C., Frühholz S., et al. (2018). Functional organization of face processing in the human superior temporal sulcus: a 7T high-resolution fMRI study. Social Cognitive and Affective Neuroscience, 13, 102–13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Schulz K.P., Clerkin S.M., Halperin J.M., et al. (2009). Dissociable neural effects of stimulus valence and preceding context during the inhibition of responses to emotional faces. Human Brain Mapping, 30, 2821–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Schurz M., Tholen M.G., Perner J., et al. (2017). Specifying the brain anatomy underlying temporo-parietal junction activations for theory of mind: a review using probabilistic atlases from different imaging modalities. Human Brain Mapping, 38, 4788–805. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Senju A., Johnson M.H. (2009). The eye contact effect: mechanisms and development. Trends in Cognitive Sciences, 13, 127–34. [DOI] [PubMed] [Google Scholar]
  52. Tanaka J.W., Sung A. (2016). The ‘eye avoidance’ hypothesis of autism face processing. Journal of Autism and Developmental Disorders, 46, 1538–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Vossel S., Weidner R., Driver J., et al. (2012). Deconstructing the architecture of dorsal and ventral attention systems with dynamic causal modeling. The Journal of Neuroscience: the Official Journal of the Society for Neuroscience, 32, 10637–48. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Wang Y., Hamilton A.F.d.C. (2012). Social top-down response modulation (STORM): a model of the control of mimicry in social interaction. Frontiers in Human Neuroscience, 6, 153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Wang Y., Newport R., Hamilton A.F.d.C. (2011a). Eye contact enhances mimicry of intransitive hand movements. Biology Letters, 7, 7–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Wang Y., Ramsey R., de C. Hamilton A.F. (2011b). The control of mimicry by eye contact is mediated by medial prefrontal cortex. Journal of Neuroscience, 31, 12001–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Yarkoni T., Poldrack R.A., Nichols T.E., et al. (2011). Large-scale automated synthesis of human functional neuroimaging data. Nature Methods, 8, 665–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Zhang R., Geng X., Lee T.M.C. (2017). Large-scale functional neural network correlates of response inhibition: an fMRI meta-analysis. Brain Structure and Function, 222, 3973–90. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Social Cognitive and Affective Neuroscience are provided here courtesy of Oxford University Press

RESOURCES