Attentional effects of gaze shifts are influenced by emotion and spatial frequency, but not in autism

Abstract

Objective

Impaired gaze following is an important hallmark of autism spectrum disorders (ASD) in clinical settings. Yet, ASD subjects perform normally on laboratory tasks involving gaze shifts. We investigated this contradiction, hypothesizing that impaired gaze following in ASD is not related to basic impairments in attentional orienting but to impaired emotion perception and abnormal processing of spatial frequencies (i.e. local and global information).

Method

We tested 30 high-functioning, school-aged children with ASD and 30 age- and IQ-matched controls on a task involving gaze shifts that cue the location of targets. The cueing faces differed in emotionality and were filtered for different spatial frequencies. We recorded behavioral responses (reaction times) and brain responses (event-related potentials, i.e. ERPs).

Results

ASD subjects performed normally when neutral faces were used. However, emotional faces elicited modified face and gaze cue processing in control subjects, but not in the ASD subjects. Furthermore, the control group biased toward the use of low spatial frequencies (global information) to process gaze cues, while the ASD group biased toward the use of high spatial frequencies (local information).

Conclusions

We conclude that impaired gaze following in ASD is related to impaired emotion processing. Moreover, ASD subjects show an abnormal reliance on local information to process gaze cues.

Introduction

An intriguing contradiction exists between clinical reports of impaired gaze following in autism spectrum disorders (ASD) on the one hand and studies failing to reproduce this effect in the laboratory on the other hand. Gaze following is a social skill by which individuals direct their attention to the same location by observing each other’s gaze direction. In clinical settings, impaired gaze following is recognized as an important hallmark of ASD, both in young children and adolescents.^1,2 In fact, it is one of the earliest detectable symptoms of these disorders and is probably directly related to problems in social interactions typical for ASD (Baird et al., 2000).^3,4 Nonetheless, several recent studies showed irreproducibility of this attribute of ASD in the laboratory.^5–10 The present study aims at investigating this conundrum, hypothesizing that the negative results obtained in laboratory settings are an effect of the low ecological validity of the experimental stimuli used.¹¹ To date, face motion and emotion have not been included in experimental set-ups. Instead all previous studies relied on static neutral faces. Therefore, we designed an experimental task involving dynamic emotional faces.

Laboratory tasks used to study gaze following typically comprise a centrally presented, non-emotional face that cues the location of a peripheral target through a gaze shift. The rest of the face remains static. The gaze cue leads to faster detection times for correctly cued targets.^12,13 To display naturally looking emotional expressions that accompany the gaze shift, we designed a gaze cue task with dynamic faces. We hypothesized that attentional orienting in typically developing children is affected by emotion, since emotional expressions are known to influence spatial attention. Fearful expressions in particular have the capacity to modify attention orienting in a fast and involuntary manner.^14,15 For children with ASD we assumed that emotion would not affect attentional orienting, because these children have abnormalities in the processing of emotion.^16–20

Our hypothesis is supported by a deficit in the processing of facial information in ASD,¹⁶ such as impaired recognition of emotional expressions.¹⁷ In addition, brain regions involved in the perception of facial emotions are hypoactive in individuals with ASD.^18–20 These brain regions include the fusiform gyrus, often referred to as the ‘fusiform face area’, the amygdala, known for its role in the appraisal of emotional stimuli, and the superior temporal sulcus, a region mediating biological motion perception in, for example, expression changes and gaze shifts.^18–20 It therefore is likely that not impairments in gaze cue processing per se, but rather abnormalities in emotion processing lead to the impaired gaze following seen in ASD in clinical settings and daily life.

Some authors have proposed that the deficits in emotion processing in ASD are the result of a decreased social motivation that corrupts the development of face processing skills.¹⁶ However, there is increasing evidence that the cause of the abnormal emotion processing in ASD is not social in nature but is related to perceptual abnormalities.^21,22 Various previous studies indicate that ASD subjects focus on local rather than global aspects of stimuli,^22,23 both for social^17,24 and non-social^25,26 information. In other words, local aspects (i.e. individual face features, such as expression-related wrinkles) are processed more thoroughly in ASD than global aspects (coarse information about the configuration of features).

The processing of local and global information may be linked to the processing of high and low spatial frequencies (SF), respectively.^27–30 Global information conveyed by low SF (see fig. 1) activates rapid threat/saliency detection systems in the brain.^30,31 In this way low SF are believed to mediate fast and unconscious social adjustments of behavior,^30,32 whereas the detailed information conveyed by high SF (see fig. 1) mediates conscious perception and memory of emotional faces, at the expense of processing speed.^29,32 An interesting finding is that emotional faces containing solely low SF activate the amygdala and the fusiform face area, whereas emotional faces containing solely high SF do not.^31,33,34 As mentioned above the amygdala and fusiform face area are both important for emotion perception and are hypoactive in ASD.^18–20

Figure 1.

Example of an unfiltered face stimulus (middle) and its high-pass (left) and low-pass (right) filtered versions.

This suggests that the focus on local information in ASD is related to hypoprocessing of low SF or a relative overuse of high SF. In line with this idea, Deruelle et al. found that children with ASD had more difficulty than controls in identifying faces when only low SF are shown.³⁵ In contrast, the controls had more difficulty than the ASD children when only high SF were shown. Atypical processing of high SF in ASD has also been found using simple grating stimuli.³⁶ It therefore seems that, for social as well as non-social information, there is a bias toward the use of local information and high SF in ASD. This bias probably affects emotion processing, given the above described differential involvement of high and low SF in emotion processing.

We included high-pass filtered (fine-grained) and low-pass filtered (blurry) faces in our experimental set-up, to investigate whether a bias toward the processing of local information and high SF indeed influences emotion processing and, possibly, gaze cue processing in ASD. For the control group we hypothesized that low SF are of particular importance in the processing of emotional gaze cues, because the present task does not emphasize conscious emotion processing. Also, the effect of gaze cues is believed to depend on the contrast between iris and sclera,³⁷ which is a low SF feature in normal viewing conditions as well as in our experimental set-up. In contrast, we expected the ASD group to show a relative over-reliance on high SF information.

While subjects performed the task we recorded reaction times and brain activity (event related potentials, i.e. ERPs) related to face processing (N170 peak) and attentional orienting (P100 and N200 peaks). It is well documented that the N170 peak in response to faces reflects face-specific brain activity originating most likely from the fusiform gyrus or superior temporal sulcus.^38,39 Previous studies indicated that emotion and SF filtering affect the N170^40,41 (but see a study done by Holmes et al.)⁴². The occipito-temporal P100 and N200 peaks in response to the peripheral targets are typically earlier and larger for correctly cued targets, which probably reflects increased neural activity in the extrastriate visual cortex to facilitate the processing of attended stimuli.⁴³ Kemner et al. found the N170, P100, and N200 to be normal in ASD patients during a gaze cue task with neutral expressions.⁵ In our emotional gaze cue task we expect abnormal brain activity in the ASD group because of the effects of emotion and SF filtering. A group of school-aged children with a diagnosis of either Autistic Disorder or Asperger Syndrome and an evenly numbered group of age- and IQ-matched typically developing controls participated in this study.

Methods and Materials

Subjects

The ASD and control groups both consisted of 30 high-functioning (intelligence quotient (IQ) >80) children, matched for sex, age (school-aged, 7-13 years old), and IQ (Table 1). IQ was obtained using the Wechsler Intelligence Scale for Children, revised Dutch edition (WISC-RN). All subjects had normal or corrected to normal vision and no neurological history. Control children were all screened for psychopathology by means of the Child Behavior Checklist (CBCL)⁴⁴ and excluded if scores were within the clinical range. 67% of subjects was also screened by means of the Teacher’s Report Form (TRF).⁴⁵ The ASD subjects were diagnosed with either Autistic Disorder or Asperger Syndrome by a child psychiatrist using DSM-IV criteria.⁴ Additionally, the Autism Diagnostic Interview Revised (ADI-R)⁴⁶ was administered to the parents by a trained rater (Table 1). Of all patients, 24 met full ADI-R criteria for autism (see Table 1 for cut-offs) and 6 met criteria for autism spectrum disorder (defined as scoring 1 point below cut-off on only one of the three ADI-R domains, which was the stereotypy domain in 5 of the 6 cases). None of the patients had a comorbid psychiatric or neurological disorder. Seven patients used psychoactive medication (3 methylphenidate, 1 typical neuroleptic, 2 atypical neuroleptics and 1 SSRI) and patients on methylphenidate were instructed not to take this medication on the day of testing. The study was approved of by the Medical Ethics Committee of the University Medical Center and all parents gave written informed consent prior to participation.

Table 1. Subject Characteristics (Mean ± Standard Error).

	ASD	Control
No. of subjects	30 (24 M, 6 F)	30 (24 M, 6 F)
Age, y	10.7 ± 1.8	10.6 ± 1.6
Total IQ	108.4 ± 2.6	111.5 ± 2.2
Verbal IQ	113.3 ± 2.7	116.3 ± 2.5
Performance IQ	101.4 ± 3.1	100.6 ± 2.5
ADI-R social domaina	20 ± 1.1
ADI-R communication domainb	16 ± 0.8
ADI-R stereotype domainc	6 ± 0.5

Note: ASD = autism spectrum disorders; M = male; F = female; ADI-R = Autism Diagnostic Interview Revised.

^aCutoff for autism: 10;

^bCutoff for autism: 8;

^cCutoff for autism: 3.

Stimuli

Pictures of 10 different actors, displaying fearful and neutral expressions, were taken from the MacBrain Face Stimulus Set.⁴⁷ Using Adobe® Photoshop® 7.0.1 software, straight and averted eyes were created and the faces were matched for size (6.46cm horizontally, which corresponds to 3.7°, because viewing distance was 1m), shape, luminance (18 cd/m²) and contrast. To generate dynamic stimuli, fearful and neutral pictures of the same actor were ‘morphed’ using Meesoft Smartmorph© 1.55 software (resulting in 13-frame movie clips). High-pass and low-pass filtering was done using a two-dimensional Fourier transformation with cut-off values of >22.2 cycles/image (equivalent to >6 cycles/degree) and <7.4 cycles/image (equivalent to <2 cycles/degree) respectively (fig. 1).

Design

Two different emotional conditions were compared. In the ‘neutral-to-fearful’ condition (N-to-F), a neutral face changed to fearful while making a gaze shift from straight to averted. In the ‘fearful-to-neutral’ condition (F-to-N), a fearful face changed to neutral while changing gaze direction. For N-to-F we expected enhanced cueing effects, because the averted fearful gaze was expected to enhance the shift of attention to the periphery. For F-to-N we expected decreased cueing effects, because the straight fearful gaze was expected to attract attention to its location.¹⁵ SF filtering of these conditions yielded 6 conditions (2x unfiltered, 2x high SF, 2x low SF). To be able to replicate previous findings of studies testing ASD subjects on gaze cue tasks, we included a seventh condition (‘neutral-to-neutral’) involving static neutral faces (unfiltered).

Procedure

Each subject completed 700 trials (100 trials per condition), presented over 4 blocks. Additional trials (175 maximally) were presented, in the same session, to 3 ASD subjects and 2 control subjects to compensate for excessive movement artifacts in the ERP signal. The chronological sequence of events in a trial was as follows: fixation dot (1000ms), initial face with straight gaze (373ms), movie-clip showing gradual change of facial expression and gaze direction (11 frames, 40ms per frame), final face gazing randomly to the left or right (373ms), target cross (subtending 0.7°) placed randomly on the left or right (5.7° off-center, 1000ms), pseudo-random delay (1100-1700ms) (fig. 2). During target presentation, the final face remained visible on the screen to avoid offset effects. Stimuli were presented on a gray background matching the average luminance of the face stimuli. Subjects were made aware of the fact that gaze direction did not predict target location. Subjects were instructed to fixate on the central face throughout the experiment and to respond to appearance of the target by pressing a corresponding left or right button as quickly and accurately as possible.

Figure 2. Sequence of events in one trial.

Shown here is the neutral-to-fearful condition, which was compared with the fearful-to-neutral condition to reveal the effects of emotion. A neutral-to-neutral condition, involving a gaze change without an expression change, was also included. Targets were correctly cued by gaze direction in 50% of the trials. Event-related potentials were measured in response to the initial face and in response to the target.

Data Recording

Subjects were seated in an electronically and acoustically shielded room. Electroencephalography (EEG) was recorded with 34 silver/silver-chloride (Ag/AgCl) flat type active electrodes (Active Two system, Biosemi) positioned at standard locations on an elastic cap (Quickcap, Neuromedical supplies of Neurosoft inc.). EEG was sampled at 2048 Hz and stored as a continuous signal. Two electrodes in the electrode cap provided an active ground. Additionally, horizontal and vertical electro-oculograms (EOG) were measured. Data were resampled offline at 500 Hz and analyzed using Brain Vision Analyser© software (Brainproducts GmbH). A high-pass filter of 2Hz, a low-pass-filter of 20Hz, and a Notch filter of 50Hz were applied. All electrodes were referenced to the right mastoid. Eye movement artifacts (detected using the algorithm by Gratton et al.⁴⁸, which is implemented in Brain Vision Analyser©) and EEG artifacts (defined as amplitudes >±100μV, amplitude differences within epoch >137μV, amplitude differences within 200ms <3μV, voltage steps per sample point >15μV) were removed.

Data Analysis

Epochs were extracted off-line from the continuous data. The face-specific N170 peak was defined as a negative deflection 165-235ms after onset of the initial face (at electrodes P3, P4, CP5, CP6, P7, and P8). The N170 therefore reflects processing of the initial emotion (of the first face) and is not influenced by the emotion of the final face. To study attentional orienting, we analyzed the P100 (positive deflection 110-160 ms after target onset) and N200 (negative deflection 165-215 ms after target onset) at electrodes PO3, PO4, P3, P4, P7, and P8 (fig. 3). Electrode sites to be analyzed were chosen based on previous studies and data inspection.^5,38,43 For the N170 a broader window was chosen to enable scoring within one window of the high SF and low SF conditions, which differ slightly in their N170 latency. Baseline correction was done on a 100ms pre-stimulus interval for the N170 and a longer (200ms) pre-stimulus interval for the P100 and N200, the latter because there was no rest period prior to target appearance. Measured variables were baseline-to-peak amplitudes and stimulus-onset-to-peak latencies. Trials with correct responses and without EEG artifacts were included in the analyses (analyzed trials per subject: 85% and 86% on average in ASD and control groups, respectively). Reaction times ranging from 100-1500ms were included in the analyses, after exclusion of the 5% slowest reaction times per subject.

Figure 3. Positions of the analyzed electrodes; red: used for the face-specific N170, green: used for the P100 and N200 in response to target appearance.

Statistical Analysis

We performed a repeated measures ANOVA with group as between factor and emotion, laterality (electrodes in left or right hemisphere) and position (of the bilateral electrodes) as within factors. Cue-validity and filter-type (high-pass or low-pass) were added as within factors for the analysis of the P100 and N200 and the analysis of the filtered conditions, respectively. We restricted the analyses to effects relevant to our hypotheses. For the N170, we tested for effects of (and interactions with) emotion (using the unfiltered faces) and effects of filter-type (using the filtered faces). Regarding reaction times and the P100 and N200, we tested for effects of cue-validity in the static neutral condition, interactions between cue-validity and emotion in the dynamic emotional conditions, and interactions between cue-validity and filter-type in the filtered conditions. A two-tailed alpha of 0.05 was adopted and the Greenhouse-Geisser correction was applied. The filtered and unfiltered conditions were analyzed in independent analyses to avoid interference of the effect of filtering per se, which is the effect of omitting an important (middle) part of the frequency spectrum in both filtered conditions.²⁸ The static neutral condition was analyzed independently, because of the absence of face motion in this condition. Partial effects were only tested when the overall effect was significant, except for a post hoc test that was done per group on the cue-validity*emotion interaction on reaction times (based on visual inspection of the data, see fig. 7).

Figure 7. Effects of emotion on gaze cueing.

The cue-validity effect on N200 latency (left) and reaction times (right) in the unfiltered neutral-to-fearful (N-to-F) and fearful-to-neutral (F-to-N) condition (± SE). For the control group the cue-validity effect was enhanced for N-to-F and decreased for F-to-N, while the ASD group showed intermediate cue-validity effects for both emotional conditions.

Results

In none of the conditions was there a significant difference between the control group and the ASD group in absolute reaction times or number of correct responses. In the static neutral condition, the cue-validity effects on reaction times (RT), P100, and N200 latency did not differ between the subject groups. On N200 amplitude, the ASD group showed a larger cue-validity effect (fig. 4). In both groups correctly cued targets elicited earlier and larger P100 and N200 peaks and faster RTs than incorrectly cued targets (significant effects and statistical values in Table 3).

Figure 4. Results of the neutral-to-neutral condition, showing no impairments in ASD.

On the left: P100 and N200 peaks in response to correctly cued (solid lines) and incorrectly cued (dashed lines) targets at electrode P4 (right hemisphere). The peaks are earlier and larger for correctly cued targets, though the P100 amplitude effect is only evident in the left hemisphere (see Table 3), which is not shown here. Inset: heads show the distribution of N200 activity to correctly cued targets (negativity in blue, positivity in yellow). On the right: The cue-validity effect (± SE) on reaction times (RT), given as the RT to incorrectly cued targets minus the RT to correctly cued targets. This effect did not differ between the subject groups.

Table 3. Overview of the Results on Gaze Cueing Effects.

Effects of interest	Overall effects found	Group effects found	Results
EMOTION
N170 latency	lat*emo F(1,58) =4.2, p<0.05  P4-P8-CP6: emo F(1,58) =7.1, p<0.01		both groups: neutral » » fearful
N170 amplitude	emoF(1,58) =7.0, p<0.05	emo*group F(1,58) =4.3, p<0.05  control: emo F(1,29) =11.7, p<0.005 ASD: emo F(1,29) =0.2, p=0.7 neutral: group F(1,58) =1.4, p=0.2 fearful: group F(1,58) =4.5, p<0.05	control group only: fearful > neutral fearful faces only: control > ASD
FILTERING
N170 latency	fil F(1,58) =45.3, p<0.001 pos*fil F(2,57) =15.0, p<0.001  P3-P4: fil F(1,58) =23.0, p<0.001  P7-P8: fil F(1,58) =155.1, p<0.001	pos*lat*fil*group F(2,57) =3.8, p<0.05  P7: fil*group F(1,58) =5.0, p<0.05  control: fil F(1,29) =128.9, p<0.001  ASD: fil F(1,29) =12.2, p<0.005	mainly in control group: low-» » high-pass
N170 amplitude	fil F(1,58) =11.2, p<0.005 pos*fil F(2,57) =29.0, p<0.001 pos*lat*fil F(2,57) =6.2, p<0.005  CP5, CP6, P3, P4: fil all F(1,58) <13.3, all p<0.005		both groups: high-pass > low-pass
EMOTION*FILTERING
N170 amplitude		pos*lat*emo*firgroup  F(2,57) =5.2, p<0.005

Note: RT = reaction time; val = cue validity. See Table 2 footnotes for further explanation.

Effects of emotion on face processing (N170 peak)

The effects of emotion on face processing concern the emotion of the initial (not yet dynamic) face. The amplitude of the N170 was affected by emotion in the control group only (fig. 5): the control group showed an enlarged N170 for fearful faces (compared with neutral faces), whereas the ASD group did not. The N170 elicited by fearful faces was significantly larger in the control group than in the ASD group, whereas the N170 elicited by neutral faces did not significantly differ between the groups (see Table 2). Both groups showed a delayed N170 in response to fearful as compared with neutral faces. We found no interactions between effects of emotion and effects of spatial frequency (SF) filtering, however, the groups did differ in the main effect of filter-type: although both groups showed an earlier and smaller N170 for low SF filtered faces compared with high SF filtered faces, this effect was significantly smaller in the ASD group than the control group on N170 latency (fig. 6, significant effects and statistical values in Table 2).

Figure 5. Effects of emotion on face processing.

The N170 peak in response to unfiltered fearful (solid lines) and neutral (dashed lines) faces for both groups (control group in blue, ASD group in red) at electrode P4. Inset: heads show the distribution of N170 activity to fearful faces (negativity in blue, positivity in red). Note that control subjects have an enlarged N170 to fearful faces, whereas ASD subjects do not.

Table 2. Overview of the Results on Face Processing.

Effects of interest	Overall effects found	Group effects found	Results
EMOTION
N170 latency	lat*emo F(1,58) =4.2, p<0.05 P4-P8-CP6: emo F(1,58) =7.1, p<0.01		both groups: neutral » » fearful
N170 amplitude	emo F(1,58) =7.0, p<0.05	emo*group F(1,58) =4.3, p<0.05  control: emo F(1,29) =11.7, p<0.005           ASD: emo F(1,29) =0.2, p=0.7  neutral: group F(1,58) =1.4, p=0.2       fearful: group F(1,58) =4.5, p<0.05	control group only: fearful > neutral fearful faces only: control > ASD
FILTERING
	fil F(1,58) =45.3, p<0.001 pos*fil F(2,57) =15.0, p<0.001  P3-P4: fil F(1,58) =23.0, p<0.001 P7-P8: fil F(1,58) =155.1, p<0.001	pos*lat*fil*group F(2,57) =3.8, p<0.05  P7: fiTgroup F(1,58) =5.0, p<0.05  control: fil F(1,29) =128.9, p<0.001        ASD: fil F(1,29) =12.2, p<0.005	mainly in control group: low-pass» »high-pass
N170 latency
N170 amplitude	fil F(1,58) =11.2, p<0.005 pos*fil F(2,57) =29.0, p<0.001 pos*lat*fil F(2,57) =6.2, p<0.005  CP5, CP6, P3, P4:   fil all F(1,58) >13.3, all p<0.005		both groups: high-pass > low-pass
EMOTION *FILTERING
N170 amplitude		pos*lat*emo*fil*group F(2,57) =5.2, p<0.005

Note: All significant results (in bold) and partial effects (indicated by a sign) are presented. Effects without significant partial effects perelectrode (in grey) are not reported in the text. emo = emotion; fil = filter type; pos = position of bilateral electrodes; lat = laterality; »» = earlier than; > = larger than.

Figure 6. Effects of filter type on face processing.

N170 latencies (± SE) for high-pass (black) and low-pass (grey) filtered faces at electrode P7. The ASD group showed a smaller effect of filter type.

Effects of emotion on gaze cueing (P100 and N200 peaks)

The RTs, P100 peak and N200 peak reflect the effects of the dynamic change in emotion (from neutral to fearful or vice versa) on attentional orienting. The control group showed a significant cue-validity effect only in the neutral-to-fearful condition and not in the fearful-to-neutral condition. The ASD group showed no influence of emotion, with the cue-validity effects in both emotional conditions being intermediate to the cue-validity effects in the control group (fig. 7). This pattern of results was found for brain activity (N200 latency) as well as RT (the latter in a post hoc test) and did not interact with effects of spatial frequency filtering. Furthermore, we found that, for control subjects, the cue-validity effect was significantly larger in the low SF condition than in the high SF condition, whereas this was reversed for ASD subjects, with there being a trend toward a larger cue-validity effect in the high SF condition than in the low SF condition (fig. 8). This difference between the groups was most significant in the high SF condition (significant effects and statistical values in Table 3).

Figure 8. Effects of filter type on gaze cueing.

The cue-validity effect (± SE) on N200 amplitude at electrodes P3 and P7 in the high-pass (black) and low-pass (gray) filtered conditions. The cue-validity effect was largest in the low-pass filtered condition for the control group and largest in the high-pass filtered condition for the ASD group.

Discussion

The aim of the present study was to investigate the apparent discrepancy between clinical findings of impaired gaze following in ASD and laboratory findings of normal attentional orienting in response to (neutral) gaze cues in these patients. Since previous laboratory studies relied on static neutral faces, we hypothesized that this discrepancy could be the result of low ecological validity of the experimental stimuli used thus far. For this reason, we tested high-functioning children with ASD and age- and IQ-matched controls using dynamic emotional gaze cues. We demonstrated that attentional orienting in response to gaze shifts was influenced by emotion in the control group, but not in the ASD group. In addition, the ASD subjects showed a relative overuse of high spatial frequencies (SF) when processing gaze cues, in line with previous reports.^35,36 This supports the abnormal dependence on local information found previously in ASD.^23–25

Effects of emotion on face processing

The processing of static faces was studied by analyzing brain activity, i.e. the N170 peak, in response to the initial (not yet dynamic) face presented during the task. In the control group the amplitude of the N170 increased in response to fearful faces in comparison with neutral faces (fig. 5), which probably reflects the recruitment of additional neural populations for the processing of emotion (in line with a study by Batty and Taylor)⁴⁰. Contrarily, the ASD group did not show an increased amplitude for fearful faces. This could suggest that in ASD subjects the perception of fear taps the same neural processes as the perception of neutral expressions. Perhaps this reflects an inability of ASD subjects to engage additional resources as the control subjects do when processing emotion. Studies have shown that an enlarged N170 for fearful expressions gradually emerges during typical development and is most evident in adults.⁴⁹ Therefore, the present results suggest that there is a developmental lag in the specialized processing of emotion in ASD.

Rapid processing of emotion is believed to rely on low SF, which carry global and configurational stimulus information; however, patients with ASD show an overuse of local information, mediated by high SF.^22,23 We hypothesized that this hinders emotion processing in ASD. Although the present results do not provide direct support for this hypothesis, there were indications that the ASD group has a diminished specialization in the differential processing of high and low SF (fig. 6). Supporting a diminished specialization in the processing high and low SF via different neural pathways, Boeschoten et al. found that the amplitude of the P1 peak in response to grating stimuli differed for different SF in control subjects, but not in ASD subjects.³⁶ In addition, when processing faces, control subjects were found to activate different sources in the brain for high-pass and low-pass filtered faces, whereas ASD subjects did not.⁵⁰ The relative over-reliance on high SF information seen in ASD may be a consequence of a disrupted specialization process during visual development. This could well be a basic neural mechanism of ASD worthy of clinical and scientific interest.

Effects of emotion on gaze cueing

To study attentional orienting in response to gaze shifts, we analyzed reaction times as well as brain activity (the P100 and N200 peaks) in response to the appearance of targets. We replicated previous findings of normal attentional orienting in response to static neutral gaze cues in ASD (fig. 4), confirming that the processing of unemotional gaze cues is not impaired in patients with ASD. We even found an increased, rather than decreased, effect of the gaze cue on the N200 amplitude in the ASD group, which could reflect an increased effect of (expected) target onset or a superior attention to the task in ASD subjects.⁵¹ A similar result was observed in a study by Kemner et al.⁵, although it was not statistically significant in this study, perhaps due to a smaller number of subjects.

Although subjects with ASD performed normally when neutral gaze cues are used, we expected abnormalities to arise when dynamic emotional expressions co-occur with the gaze shifts. We compared two dynamic emotional gaze cues and found that the cue-validity effect, which reflects attentional orienting in response to the gaze shift, was influenced by emotion in the control group only. In the ASD group effects of emotion were absent. This pattern of results was found for brain activity (ERPs) as well as reaction times (fig. 7), though the latter did not reach significance in the overall analysis, perhaps due to the large variability. These findings reveal that subjects with ASD are specifically impaired in a naturalistic set-up, in which dynamic emotional gaze cues require the integration of emotional information and gaze information.

Attentional orienting in response to gaze shifts is an involuntary behavior that persists even when the gaze cue is counter-predictive (e.g. when subjects know that 75% of the targets are incorrectly cued).¹³ Our results, showing that facial expression influences gaze cueing in healthy control subjects, therefore emphasize the intimate link between emotion perception and subconscious behavioral adjustments. To our knowledge, this finding has not been reported before (but see Holmes et al., 2006, for a study on high-state anxious participants)¹⁴. A previous study, using static instead of dynamic emotional gaze cues, did not find emotion to modulate gaze cueing.⁵² However, static displays are less ecologically valid and miss the change in expression that was found to influence gaze cueing in the present study (i.e., static faces associate the emotion with both straight and averted gaze, instead of either of the two). In our set-up, face motion mediated a naturally looking emotional change that modified attentional orienting in healthy controls (fig. 7), but left face and gaze cue processing in ASD largely unaffected.

Some authors have proposed that abnormal emotion processing in ASD is the result of a decreased social motivation that corrupts the development of face processing skills.¹⁶ However, the perception of social cues depends on fast brain systems that involve magnocellular pathways in subcortical structures and regions of the dorsal visual stream. These systems are sensitive to low SF and global information^28–32 and may be disrupted in ASD, which is reflected in a relative overuse of high SF, i.e. local information.^29,32,35,36 The present results support this. In the control group the cue-validity effect was larger in the low SF condition, whereas in the ASD group it was more evident in the high SF condition (fig. 8). While this effect was found for brain activity (ERPs) only in this study, it may also affect behavior in more delicate social situations. An abnormal reliance on high SF could be a characteristic of ASD that is directly related to the neural mechanisms underlying these disorders. In the light of early intervention, it might serve as a useful marker for identifying young children at risk of ASD.

Analogous to the processing of gaze cues, we expected control subjects to rely on low SF for the processing of emotion, but the present results do not provide evidence for this. It is possible that subtle interactions between effects of emotion and effects of SF filter type were disrupted by filtering out the middle part of the frequency spectrum in both filtered conditions. This part of the frequency spectrum is known to be important for face perception.²⁸ Another factor that may be the relatively long cue-duration that we used.^12,13 The difference in processing speed for high and low SF may no longer have been present within the time frame of the trials. For further studies we suggest the use of variable cue-durations.

Our findings provide interesting insight into the link between emotion processing and gaze cueing in typically developing children and children with ASD. However, our experimental set-up, although more ecologically valid than previous set-ups, is still simplified compared with every day life. In daily social interactions not only impairments in emotion processing, but also other factors might contribute to impairments in gaze following behavior, for example an increased tendency to be distracted by socially irrelevant stimuli. In addition, we cannot rule out the possibility that some effects reflect a type I error. To tackle the problem of multiple testing, we restricted the number of analyses to only those that specifically addressed the hypotheses and, therefore, were clinically relevant. Nevertheless, future studies are needed to test the robustness of the present results. These studies should involve analysis of behavioral as well as brain activity measures, because, in developmental disorders as ASD, neural abnormalities may be compensated by adaptational strategies, resulting in masking of the abnormalities at the behavioral level. Indeed, with regard to gaze cueing most differences between the subject groups were found for the N200 brain activity peak.

Based on several previous studies that indicate that subjects with ASD spend less time looking at the eye region when viewing faces,⁵³ it could be suggested that abnormal gaze fixation patterns have influenced our findings. However, several other studies have found normal scanning patterns in ASD subjects, in response to both neutral and emotional faces.^54–56 Also, the fact that there was no overall group difference in the magnitude of the effect of the gaze cue contradicts a possible lack of gaze at the eye region in the ASD group. Eye-tracking equipment may be useful in future studies to support these arguments. Additionally, the diagnostic procedure could be refined by using the ADOS⁵⁷ as well as the ADI⁴⁶.

To conclude, we showed for the first time that ASD patients process gaze cues normally when static neutral faces are presented, but deviate from normal when dynamic emotional faces are presented. More specifically, the effect of the gaze cue was modified by emotion in the control group but not in the ASD group. We suggest that the impaired gaze following evident in ASD in clinical situations is not the result of a basic deficit in attentional orienting but is caused by impaired processing of social information, i.e. emotional expressions. We also found that subjects with ASD rely on high SF information when processing gaze cues, whereas the control subjects rely on low SF information. The present data thus directly relate atypical low-level visual processing to abnormalities in the processing of social information in subjects with ASD.