Brain Mechanisms for Processing Direct and Averted Gaze in Individuals with Autism

Abstract

Prior studies have indicated brain abnormalities underlying social processing in autism, but no fMRI study has specifically addressed the differential processing of direct and averted gaze, a critical social cue. Fifteen adolescents and adults with autism and 14 typically developing comparison participants viewed dynamic virtual-reality videos depicting a simple but realistic social scenario, in which an approaching male figure maintained either direct or averted gaze. Significant group by condition interactions reflecting differential responses to direct versus averted gaze in people with autism relative to typically developing individuals were identified in the right temporoparietal junction, right anterior insula, left lateral occipital cortex, and left dorsolateral prefrontal cortex. Our results provide initial evidence regarding brain mechanisms underlying the processing of gaze direction during simple social encounters, providing new insight into the social deficits in individuals with autism.

Autism is a behaviorally defined, pervasive neurodevelopmental disorder characterized by a triad of deficits: (a) impairments in social interactions; (b) delays in or the absence of communicative skills; and (c) restricted, repetitive, and stereotyped patterns of behavior, interests, and activities (APA, 2000). While a vast amount of heterogeneity is common within the symptom domains, the unifying diagnostic feature of the disorder comprises social deficits (Kanner, 1943; Pelphrey & Carter, 2008; Wing & Gould, 1979).

Eye gaze is an important social cue (Frischen, Bayliss, & Tipper, 2007), serving several important functions in complex social interactions, including the provision of information related to a person’s physical attributes and mental states, the facilitation of communication and regulation of the flow of conversation, and the expression of intimacy and social dominance (Kleinke, 1986). It has also been suggested that gaze processing is pivotal to the appropriate development of social cognition (Baron-Cohen, 1995). A number of studies have examined the behavioral and neural correlates of the typical processing of gaze direction in infants, demonstrating preferential attention to and enhanced neural processing of direct versus averted gaze (Farroni, Csibra, Simion, & Johnson, 2002; Grossman, Johnson, Farroni, & Csibra, 2007). Thus, given the early emergence of gaze differentiation, it is likely that sensitivity to gaze direction is subserved by innate mechanisms, supporting the hypothesized importance of its role in early social development. Typical adults are similarly able to accurately determine the direction of another person’s gaze (Gamer & Hecht, 2007). Moreover, direct gaze confers task-related perceptual advantages relative to averted gaze. For example, direct gaze is detected faster than averted gaze (Conty, Tijus, Hugueville, Coelho, & George, 2006; Senju & Hasegawa, 2005; Senju, Kikuchi, Hasegawa, Tojo, & Osanai, 2008; Senju, Yaguchi, Tojo, & Hasegawa, 2003; Wallace, Coleman, Pascalis, & Bailey, 2006) and also facilitates the categorization and recognition of faces as well as memory for faces and recognition of emotional expressions (Adams & Kleck, 2003; Adams & Kleck, 2005; Macrae, Hood, Milne, Rowe, & Mason, 2002; Sander, Grandjean, Kaiser, Wehrle, & Scherer, 2007; Vuilleumier, George, Lister, Armony, & Driver, 2005).

Functional neuroimaging studies in children and adults have made progress in elucidating the neural correlates of the distinct processing of direct and averted gaze. Electroencephalographic (EEG) and event-related potential (ERP) evidence has indicated differential neural activity for direct versus averted gaze (Conty, N’Diaye, Tijus, & George, 2007; Gale, Spratt, Chapman, & Smallbone, 1975; Hietanen, Leppanen, Peltola, Linna-aho, & Ruuhiala, 2008; McCarthy, Puce, Belger, & Allison, 1999; Puce, Smith, & Allison, 2000; Senju, Tojo, Yaguchi, & Hasegawa, 2005). Functional magnetic resonance imaging (fMRI) studies have further explored the specific brain regions responsible for differential encoding of gaze direction, implicating the anterior superior temporal sulcus (STS) (Calder, Beaver, Winston, Dolan, Jenkins, Eger, & Henson, 2007), posterior STS (Hoffman & Haxby, 2000; Pelphrey, Viola, & McCarthy, 2004; Puce, Allison, Bentin, Gore, & McCarthy, 1998; Sato, Kochiyama, Uono, & Yoshikawa, 2008), intraparietal sulcus (IPS) (Hoffman & Haxby, 2000), inferior parietal cortex (Calder et al., 2007), fusiform gyrus (FFG), amygdala (George, Driver, & Dolan, 2001) and the dorsal medial prefrontal cortex (dmPFC) (Calder, Lawrence, Keane, Scott, Owen, Christoffels, & Young, 2002; Kampe, Frith, & Frith, 2003; Schilbach, Wohlschlaeger, Kraemer, Newen, Shah, Fink, & Vogeley, 2006).

In contrast to typically developing individuals, children and adults with autism display abnormalities in the processing of eye gaze. A series of elegant behavioral studies have demonstrated that direct gaze does not elicit the same task-related perceptual advantages in individuals with autism as it does in typically developing individuals (Akechi, Senju, Kikuchi, Tojo, Osanai, & Hasegawa, 2009; Dalton, Nacewicz, Johnstone, Schaefer, Gernsbacher, Goldsmith, Alexander, & Davidson, 2005; Pellicano & Macrae, 2009; Senju & Hasegawa, 2005; Senju et al., 2003; Vlamings, Stauder, van Son, & Mottron, 2005; Wallace et al., 2006; but see Senju et al., 2008). Furthermore, although inverting the polarity of the eyes impairs performance on the perception of gaze direction in typically developing individuals, individuals with autism do not exhibit the same degree of impairment (Ashwin, Ricciardelli, & Baron-Cohen, 2009). ERP studies have found abnormal neural responses to direct gaze in children with autism (Grice, Halit, Farroni, Baron-Cohen, Bolton, & Johnson, 2005; Senju et al., 2005) and in infant siblings with the broad autism phenotype (BAP) (Elsabbagh, Volein, Csibra, Holmboe, Garwood, Tucker, Krljes, Baron-Cohen, Bolton, Charman, Baird, & Johnson, 2009). A prior fMRI study from our laboratory demonstrated a lack of context-dependent activity in the STS in individuals with autism when viewing congruent and incongruent gaze shifts (Pelphrey, Morris, & McCarthy, 2005). However, no fMRI study of autism has examined the differential processing of direct versus averted gaze in the context of a realistic social situation. To address this question, we compared brain activity in adolescents and adults with and without high-functioning autism using an event-related functional magnetic resonance imaging (fMRI) design. Using virtual reality character animation, we developed a simple social scenario in which participants viewed an approaching male figure through a virtual doorway. The approaching man, with a neutral facial expression, either made continuous direct eye contact with the participant throughout the encounter, or he maintained an averted eye gaze. By taking advantage of the extremely precise level of control afforded by the virtual reality environment, all other aspects of the scenarios were held constant, allowing us to evaluate the extent to which the difference in social context modulated brain activity. We thereby sought to characterize the neural circuitry associated with processing direct relative to averted gaze in adolescents and adults with and without high-functioning autism.

Methods

Participants

We studied a group of 15 adolescents and adults with high-functioning autism (15 males, ages 14.8 – 37.8, mean = 23.4 ± 6.9 years) and 14 typically developing adolescents and adults (13 males, ages 16.1 – 42.4, mean = 24.2 ± 7.4 years). Three additional participants – one typically developing and two with autism – were excluded for excess motion during scanning. Written informed consent was obtained from each adult participant and informed parental consent was obtained for the adolescents according to a protocol approved by the local Institutional Review Board. As demonstrated in Table 1, the two groups were matched on age as well as Performance and Full Scale IQ scores. All individuals with autism met DSM-IV criteria for autistic disorder (exclusive of Asperger syndrome and pervasive developmental disorder – not otherwise specified) as based on a history of clinical diagnosis of autism, expert clinical evaluation, parental interview (Autism Diagnostic Inverview-Revised) (Lord, Rutter, & Le Couteur, 1994), and observational assessment of the affected individual (Autism Diagnostic Observation Schedule) (Lord, Risi, Lambrecht, Cook, Leventhal, DiLavore, Pickles, & Rutter, 2000).

Table 1.

Demographic information and behavioral data.

	TD (n = 14)	Autism (n = 15)
	Mean (SD)	Mean (SD)
Demographic information
Male	13	15
Age	24.2 (7.4)	23.4 (6.9)
Right-handed	13	13
Behavioral data
IQ:
-Verbal*	110.7 (11.3)	101.9 (11.1)
-Performance	109.3 (9.6)	110.5 (14.9)
-Fullscale	111.5 (11.0)	106.7 (11.0)
ADI-R:
-Social Domain		21.8 (4.7)
-Communication Domain (Verbal)		17.2 (4.7)
-Communication Domain (Nonverbal)		9.9 (3.4)
-Stereotypy Domain		6.5 (2.5)
ADOS:
-Social Domain		9.0 (2.7)
-Communication Domain		5.4 (1.3)
-Combined Social and Communication		14.4 (3.9)
-Stereotypy Domain		2.1 (1.3)

IQ data are as measured by the Wechsler Abbreviated Scale of Intelligence (WASI). All autism assessment measures met the minimum cutoff for autism. Abbreviations: Typically Developing (TD), Autism Diagnostic Interview-Revised (ADI-R), Autism Diagnostic Observation Schedule (ADOS).

^*TD > Autism, p = 0.044.

Experimental Design

Two experimental conditions were generated using Poser 7.0^® (Curious Labs Inc., Santa Cruz, California). In each, participants viewed a virtual doorway from which the same animated male figure entered; the figure walked toward the participant (Figure 1A), and passed them with equal frequency on either the right or left side. In half of the trials (the Direct gaze condition; Figure 1B), the figure looked directly at the center of the screen throughout the trial, simulating direct eye contact with the participant. In the other half of the trials (the Averted gaze condition; Figure 1C), the man looked away from the participant at an angle of approximately 20°. In both conditions, the man’s facial expression remained neutral. Each trial lasted 6 seconds. Trials were separated by jittered intertrial intervals of 12, 14, or 16 seconds consisting of a white fixation cross centered on a black background; subjects were instructed to maintain fixation on the fixation cross. The experiment consisted of one run lasting 7.03 minutes (422 seconds). The run included 10 trials from each of the two conditions, presented in pseudorandom order subject to the constraint that the same trial type could not appear more than two times in succession. Participants were instructed to attend to the displays and to remain alert and awake.

Figure 1.

A) At the beginning of the trial, an animated man entered and walked toward the participant. He then passed on the right or left. Each trial lasted 6 seconds. B) On half of the trials, the figure looked directly at the center of the screen throughout the trial. C) On the other half of the trials, the man averted his gaze from the participant throughout the trial.

Imaging Protocol

Scanning was performed on a Siemens 3 Tesla Allegra head-only scanner (Siemens, Erlangen, Germany). High-resolution, T1-weighted anatomical images were acquired using an MPRAGE sequence (TR = 1630 ms; TE = 2.48 ms; FOV = 20.4 cm; α = 8°; image matrix = 256²; voxel size = 0.8 × 0.8 × 0.8 mm; 224 slices). Whole-brain functional images were acquired using a single-shot, gradient-recalled echo planar pulse sequence (TR = 2000 ms; TE = 30 ms; α = 73°; FOV = 20.4 cm; image matrix = 64²; voxel size = 3.2 × 3.2 × 3.2 mm; 35 slices) sensitive to blood oxygenation level-dependent (BOLD) contrast. We acquired one run of 211 successive brain volumes.

Data Analysis

Data were preprocessed and analyzed using the BrainVoyager QX 1.9 software package (Brain Innovation, Maastricht, The Netherlands). Preprocessing of the functional data included slice time correction (using cubic spline interpolation), alignment of slices (using cubic spline interpolation to the first nondiscarded scan time within a scan run), 3-dimensional motion correction (using trilinear interpolation), spatial smoothing with a 4-mm Gaussian kernel, linear-trend removal, and temporal high-pass filtering (fast-Fourier transform based with a cutoff of 3 cycles/time course). The functional data sets were coregistered to the Talairach-transformed (Talairach & Tournoux, 1988), within-session, T1-weighted anatomical image series to create 4-dimensional data sets. Estimated motion plots and cine loops were examined for each participant in order to identify movements and eliminate runs in which the participant displayed a deviation in the estimated center of mass (in any dimension) or a rotation that was greater than 3 mm.

To test the hypothesis that the two gaze conditions would lead to differential activation of brain regions involved in social processes in both participant groups, multiple-participant statistical analyses were performed for each group by multiple linear regression of the time course of the BOLD response in each voxel. We modeled Direct and Averted gaze conditions to compare direct gaze activity to averted gaze. Model predictors for each gaze condition were defined by convolving an ideal boxcar response with a double gamma function model of the hemodynamic response (Friston, Holmes, Worsley, Poline, Frith, & Frackowiak, 1995). Boxcar values were equal to 1.0 during the 6-second time period when the male figure was enacting direct gaze or averted gaze, and were otherwise 0. A multi-participant random effects analysis was performed using a whole-brain mask. For the two multi-participant statistical maps (one for participants with autism and one for typically developing participants), we assessed results at an uncorrected statistical threshold of p < .01. As a protection against false positives, only clusters of 6 or more contiguous functional voxels were included in the analysis (Xiong, Gao, Lancaster, & Fox, 1995). Because of the range of both age and IQ scores within participants in this study, potential effects of age and Full Scale IQ were explored in each region found differentially activated by gaze condition. Mean β values were extracted from each region and correlated with age and Full Scale IQ using Pearson correlations.

Secondary analyses were performed to explore possible interactions between stimulus condition and group membership on brain activation. We used a 2 (Condition: Direct versus Averted) × 2 (Group: Autism versus Typically Developing) whole brain GLM analysis to identify regions exhibiting a significant Condition × Group interaction. We were particularly interested in these regions because a significant interaction would indicate that the response to the two stimulus conditions varied as a function of group membership. The threshold for significance was set at a voxel-wise uncorrected p < .01 (two-tailed), with a cluster threshold of 6 contiguous functional voxels. Individual β values were extracted from these ROIs and the average values plotted by Condition and Group to visualize the response patterns giving rise to the observed interactions.

Results

We identified a network of brain regions (Table 2) active during direct gaze compared to averted gaze in each participant group. The typically developing group exhibited greater activation to direct gaze in the right anterior insula (AI), bilateral caudate, left thalamus, left cerebellum, and left inferior frontal gyrus. No regions showed greater activation to averted gaze. In contrast, the autism group exhibited greater activation to direct gaze in left cuneus, and greater activation to averted gaze in bilateral cerebellum and left inferior occipital gyrus. In the typically developing group, activation in the left cerebellum (r = −.60, p = .02) and left thalamus (r = −.55, p = .04) correlated with age, though neither of these correlations survived correction for multiple comparisons. No regions differentially activated by gaze condition correlated with Full Scale IQ. In the autism group, no correlations between activation in regions modulated by gaze condition correlated significantly with age or Full Scale IQ.

Table 2.

Overall brain activations and group differences.

	Side	Coordinates (mm)			Cluster Size	Statistic	p value
		x	y	z
Foci of activation within each ROI*
TD group
Direct > Averted
Anterior Insula	Right	36	26	7	405	4.25	0.000944
Caudate	Right	12	−1	10	213	4.26	0.000926
	Left	−12	2	7	502	5.59	0.000087
Cerebellum	Left	−39	−55	−29	186	4.04	0.001402
IFG	Left	−54	29	23	382	4.17	0.001098
Thalamus	Left	−12	−16	−5	233	5.45	0.000112
Autism group
Direct > Averted
Cuneus	Left	−9	−82	28	228	5.20	0.000133
Averted > Direct
Cerebellum	Bilateral	3	−49	−32	638	−5.42	0.00009
IOG	Left	−33	−91	−5	545	−4.51	0.000486
Group x Condition interactions in neural activation†
Supramarginal Gyrus / TPJ	Right	48	−49	46	635	15.02	0.000614
Anterior Insula	Right	33	23	4	181	12.85	0.001314
dlPFC	Left	−33	26	43	255	12.40	0.001545
LOC	Left	−45	−70	−14	269	12.50	0.001489

Talairach coordinates and statistics refer to the voxel with the maximum signal change in each ROI. Abbreviations: Typically Developing (TD), Dorsolateral Prefrontal Cortex (dlPFC), Inferior Frontal Gyrus (IFG), Inferior Occipital Gyrus (IOG), Lateral Occipital Cortex (LOC), Temporoparietal Junction (TPJ),

^*Statistics are t scores.

^†Statistics are F scores.

Four brain regions, the right temporoparietal junction (TPJ; also supramarginal gyrus), right AI, left dorsolateral prefrontal cortex (dlPFC), and left lateral occipital cortex, exhibited significant Group × Condition interactions, indicating that the effect of stimulus condition varied as a function of group membership (Figure 2). Right TPJ and left dlPFC exhibited significant differences in activation as a function of condition in both participant groups. Right AI only showed significant differences between gaze in the typically developing group, while left LOC was only significantly modulated by gaze in the autism group.

Figure 2.

Top: Regions of significant group by condition interaction (p < .01, k = 6 contiguous voxels). The activations are displayed on a Talairach-transformed template brain. Bottom: Difference in average responses in each region to direct minus averted gaze, as a function of group membership (y-axis = difference in mean β values for each condition).

Discussion

This study is the first to specifically examine the differential processing of direct and averted gaze in individuals with autism using an fMRI paradigm. Gaze direction serves as a salient social cue and thus, elucidating the neural bases of abnormal gaze processing in autism will further our understanding of the disorder. We report three key sets of findings. First, we identified a network of brain regions sensitive to direct versus averted gaze in typically developing participants. Second, we demonstrated that this same network is not preferentially active to direct gaze in participants with autism. Third, in an analysis of the interaction between group and gaze condition, we found several regions that are sensitive to gaze direction in both participant groups, but differ in terms of the kind of gaze to which they preferentially activate. This third finding supports the conclusion that both participant groups were sensitive to the experimental manipulation, yet the gaze condition that elicits preferential neural activation differs as a function of group status.

Our first two findings, that a network of brain regions responds preferentially to direct gaze in typical individuals, and that this network is not active in individuals with autism, support previous research that demonstrates differential neural processing of gaze direction in autism (Grice et al., 2005; Pelphrey et al., 2005; Senju et al., 2005) relative to typically developing individuals. Interestingly, no regions showed increased activation to averted gaze in the control group, supporting the hypothesis that direct gaze is an especially salient social cue which should recruit increased neural activation in typically developing participants. Direct gaze may not hold the same salience in participants with autism, demonstrated by the finding of preferential activation to direct gaze in only a small region of left precuneus.

The finding that participants with autism did not show a network of neural activation preferential to direct gaze raised a concern that these participants may not have sufficiently attended to the eye gaze stimuli. However, this concern is mitigated by the finding that in our within group analysis, participants with autism showed regions which were preferentially active to averted gaze, demonstrating that these participants were sensitive to our experimental manipulation.

Our investigation of regions that demonstrated a group by gaze direction interaction identified regions that were sensitive to gaze direction in only one group, as well as regions that were modulated by gaze direction in both groups, but varied as to which gaze they were sensitive by group membership. With only BOLD response measures on responses to each gaze condition in the absence of eye-tracking and behavioral data, we can only speculate on the behavioral and psychological correlates of these functional brain differences.

Regions that were modulated by gaze direction in only one group included the right anterior insula (AI) and left lateral occipital cortex (LOC). The right AI showed increased activation to direct gaze in typical individuals, but was not modulated by gaze in individuals with autism. The AI has previously been implicated as a relay station between action representation networks and limbic areas involved in the processing of emotion (Carr, Iacoboni, Dubeau, Mazziotta, & Lenzi, 2003), suggesting an important role for the insula in reflecting upon another person’s mental state. Children with autism, however, display reduced activity in this region during the imitation of emotional facial expressions (Dapretto, Davies, Pfeifer, Scott, Sigman, Bookheimer & Iacoboni, 2006), a finding consistent with a recent meta-analysis that identified the right AI as a region of hypoactivation in autism in the context of social paradigms (Di Martino, Ross, Uddin, Sklar, Castellanos, & Milham, 2009). Furthermore, the AI has been implicated in the initiation of brain responses to salient stimuli (Uddin & Mennon, 2009). In comparison, left LOC was active to averted gaze in participants with autism, but was not modulated by gaze in the control group, suggesting that while participants with autism lack increased activation to direct gaze in right AI, this group recruited distinct regions for processing gaze (averted) that typical participants did not.

Our interaction analysis also identified regions that were modulated by gaze in both groups, including right temporoparietal junction (TPJ) and left dorsolateral prefrontal cortex (dlPFC). The right TPJ was active to direct gaze in typically developing participants, and active to averted gaze in participants with autism. The TPJ has been implicated in a host of social and attentional tasks, including judgments of others’ mental states (Aichhorn, Perner, Kronbichler, Staffen, & Ladurner, 2006; Gallagher, Happé, Brunswick, Fletcher, Frith, & Frith, 2000; Krach, Hegel, Wrede, Sagerer, Binkofsji, & Kircher, 2008; Saxe & Kanwisher, 2003; Saxe, Moran, Scholz & Garieli, 2006; Saxe & Powell, 2006; Saxe & Wexler, 2005), and visual target perception (Corbetta, Kincade, Ollinger, McAvoy & Shulman, 2000; Grosbras, Laird & Paus, 2005; Mitchell, 2008; Shulman, McAvoy, Cowan, Astafiev, Tansy, d’Avossa, & Corbetta, 2003). The finding that typical participants showed significant activation in this region during direct gaze supports the idea of direct gaze as an important social cue prompting the consideration of others’ mental states (Kleinke, 1986). Differences in the gaze condition that elicits TPJ activation might be caused by a group-driven divergence in the type of gaze that holds the most social and attentional salience.

The opposite pattern existed in dlPFC, with significant activation to direct gaze in autism participants, and to averted gaze in typically developing participants. Sensitivity to gaze in dlPFC demonstrates that direct gaze does elicit a specific neural response in participants with autism, and that this response may be similar to processing of averted gaze in typically developing participants. While our findings establish the neural correlates of differences in gaze processing between participants with and without autism, future studies exploring visual attention and arousal in each gaze condition will be important in elucidating the specific nature of the identified differences in regards to social and attentional salience of gaze direction suggested by our results.

There are some limitations to the present study that bear mentioning. First, while our participant groups were matched on Full Scale and Performance IQ, our participants with autism were characterized by a slightly lower mean Verbal IQ (p = 0.04; Table 1). However, the instructions for the task were very simple, and the task itself did not involve any verbal or language component. Thus it is unlikely that differences in Verbal IQ impacted our findings. Second, our participants span a wide range of age and IQ, and this variability had the potential to impact our results. However, the results of our correlation analyses suggest that our findings were not driven by differences in age or IQ. Finally, we did not control for nor monitor eye movements. While we cannot rule out any differences in eye movements, we can rule out the possibility that participants with autism failed to attend to the stimuli altogether, as our analyses revealed differential activation of brain regions to gaze conditions in both groups (Table 2). Nevertheless, further studies utilizing eye-tracking in conjunction with fMRI are necessary to confirm and better understand the present findings.

In sum, our results provide initial evidence regarding brain mechanisms at the neural systems level underlying the processing of gaze direction during a simple, yet realistic social encounter. Gaze processing is an early emerging social phenomenon in typically developing individuals, and this study reveals abnormalities in autism within key brain regions of social processing. Taken together, differential activation to gaze direction might serve as a potential neural correlate of abnormal processing of social interactions, beginning with perception of interpersonal approach. Thus, our findings provide new insight into the social deficits in individuals with autism by means of a novel paradigm utilizing dynamic social stimuli designed to assess the differential processing of direct and averted gaze. This paradigm lends itself to the use of concurrent eye-tracking methods, future studies of which will be important in clarifying and strengthening these conclusions.