Patterns of fixation during face recognition: Differences in autism across age

Abstract

Difficulties with face recognition increase from adolescence to adulthood in autism, reflecting a lack of typical late development. We examined whether this reflects differences in the development of patterns of fixation to eyes and mouths during face recognition. Children, adolescents, and adults (aged 7–30) with and without autism completed the Cambridge Face Memory Test while gaze was recorded. Average duration and number of fixations were calculated for eyes and mouth regions of interest, defined individually for each face image in the task. All groups and age groups made more and longer fixations to eyes than mouths. However, during face memorization, typically developing children and adults, but not adolescents, made more fixations to eyes than did their peers with autism. During face recognition, typically developing children and adults made shorter fixations on mouths than did their peers with autism; this pattern was reversed in adolescence, with adolescents with autism making more fixations to mouths than typically developing adolescents. Results suggest that group differences in patterns of fixations to faces change with age. Furthermore, different relationships between patterns of fixations and face recognition performance in typical development and autism suggest that these differences contribute, at least in part, to difficulties in autism.

Introduction

Face recognition is a challenging skill for those with autism. Numerous studies have shown that individuals with autism exhibit difficulties with the recognition of face identity relative to typically developing (TD) individuals (Weigelt et al., 2012). Moreover, face recognition difficulties in autism become more robust during the transition from adolescence to adulthood due to a lack of typical late development, resulting in the greatest disparity in skill by adulthood (O’Hearn et al., 2010, 2014). This reflects, in part, that face recognition is a long-developing skill in TD individuals that continues to improve from adolescence to adulthood, peaking around age 30 (Germine et al., 2011). In addition, during face recognition, patterns of fixation to facial features differ in autism (Boraston et al., 2008; Chawarska and Shic, 2009; Sterling et al., 2008; Yi et al., 2013). Patterns of fixation to the face contribute to recognition skill (Peterson and Eckstein, 2012), as well as the difficulties in autism (Kirchner et al., 2011); therefore, using eye-tracking to study patterns of fixation to face stimuli provides a method to elucidate the mechanisms underlying increasing difficulties with face recognition over development in autism.

Although there are likely to be many possible explanations for the face recognition difficulties in autism, these challenges appear to be strongly related to differences in how the face is examined. Differences in patterns of looking at the face have long been described in autism, with striking qualitative illustrations of how the looking patterns differ from those of TD individuals (Pelphrey et al., 2002). However, despite the fact that atypical patterns of fixations to the face are proposed to underlie face recognition difficulties in autism (Kirchner et al., 2011; Snow et al., 2011), the literature on the nature of these abnormalities is inconsistent. These inconsistencies are likely to reflect, at least in part, individual differences in the samples across studies and individual variability in fixation behavior among participants within a given study. One obvious source of individual variability is age-related change in patterns of fixations to faces. Such changes have long been suggested in typical development (Carey et al., 1980); however, to our knowledge, no work has attempted to characterize how fixation patterns in autism change with age, from childhood through adolescence and into adulthood. By examining the developmental course of these differences, we hope to better understand the etiology of the face recognition difficulties characteristic of autism, as well as the inconsistencies in the literature.

Literature on the atypical patterns of fixations to the face reported in individuals with autism when compared to TD individuals is summarized in Table 1. A widely reported finding, both experimentally and anecdotally, is decreased attention to the eyes in children (Jones et al., 2008; Van Der Geest et al., 2002; Yi et al., 2013), adolescents (Klin et al., 2002; Norbury et al., 2009), and adults (Boraston et al., 2008; Hernandez et al., 2009; Pelphrey et al., 2002; Sterling et al., 2008) with autism as compared to age- and intelligence quotient (IQ)-matched TD individuals. Along with decreased fixation to the eyes, increased fixation to the mouth has also been reported in children (Jones et al., 2008) and adolescents (Klin et al., 2002) with autism, relative to their TD peers. However, this finding is less consistent, with the majority of studies finding no significant group differences in the number or duration of fixations to the mouth. It is important to note that the eyes are fixated more than the mouth in individuals both with and without autism, even when group differences in the amount of attention allocated within a single feature are present (Neumann et al., 2006; Rutherford and Towns, 2008; Speer et al., 2007). For example, in a study of adolescents with and without autism, Speer et al. (2007) found that both groups fixated longer on eyes than on mouths, but the group with autism made relatively shorter fixations to eyes as compared to the TD group. This finding illustrates that individuals with and without autism look longer at the eyes than at mouths; however, despite this typical preference for the eyes over other features, individuals with autism may look less at the eyes than do TD individuals.

Table 1.

Results from studies of patterns of fixations to faces in individuals with autism.

Age group	Stimuli	Task	Eye fixations	Mouth fixations	Citation
Children and adults	Dynamic scene	Passive viewing	Children: ASD = TD Adults: ASD < TD	Children: ASD < TD Adults: ASD = TD	Nakano et al. (2010)
Children	Dynamic scene	Passive viewing	ASD < TD	ASD > TD	Jones et al. (2008)
	Static faces	Recognition	ASD = TD	ASD < TD	Chawarska and Shic (2009)
	Static faces	Recognition	ASD < TD	ASD = TD	Yi et al. (2013)
	Static faces	Passive viewing	ASD < TD	ASD = TD	Van Der Geest et al. (2002)
Adolescents	Dynamic scene	Passive viewing	ASD < TD	ASD > TD	Klin et al. (2002)
	Dynamic scene	Passive viewing	ASD < TD	ASD = TD	Norbury et al. (2009)
	Dynamic scene	Passive viewing	ASD < TD	ASD = TD
	Static scene	Passive viewing	ASD = TD	ASD = TD	Speer et al. (2007)
	Static faces	Recognition	ASD = TD	ASD = TD	Snow et al. (2011)
Adults	Static faces	Emotion classification	ASD = TD	ASD = TD	Rutherford and Towns (2008)
	Static faces	Passive viewing	ASD < TD	ASD = TD	Hernandez et al. (2009)
	Static faces	Passive viewing	ASD < TD	ASD = TD	Pelphrey et al. (2002)
	Static faces	Emotion classification	ASD = TD	ASD = TD	Neumann et al. (2006)
	Static faces	Recognition	ASD < TD	ASD = TD	Boraston et al. (2008)
	Static faces	Recognition	ASD < TD	ASD < TD	Sterling et al. (2008)
	Static faces	Recognition	ASD = TD	ASD = TD	Kirchner et al. (2011)

ASD: autism spectrum disorder; TD: typically developing.

A review of the literature by Falck-Ytter and Von Hofsten (2011) suggests that these differences in patterns of fixation behavior are related to age, finding some support for the theory of diminished eye gaze and excess mouth gaze in autism, but only in adolescents and adults, and not in children. However, only one study to date has explicitly examined age-related differences in atypical patterns of fixations to faces in autism. In a study using dynamic video clips, Nakano et al. (2010) found that the number of fixations to the eyes did not differ between children with and without autism, but in adulthood, the group with autism made fewer fixations to the eyes than did the TD group. TD children made more fixations to the mouth than did children with autism, but in the adult group, there were no differences between groups. Although these results suggest age-related changes in patterns of fixation to the face, typically and in autism, there are limitations to this study. This study included only children and adults, thus excluding adolescence; moreover, this study used a passive viewing task, so it is unclear how these findings may relate to face recognition ability, which we know improves across this stage of development (O’Hearn et al., 2010).

There is evidence that patterns of fixation to the face have a functional role in face recognition ability in TD adults, though the development of this association is not well understood. Visual information from the eye region has been found to be the main determining factor used in decisions about face identity (Schyns et al., 2002). Moreover, computational modeling has confirmed that fixation just below the eyes maximizes information gain and allows for optimal performance on face identification; deviations from this optimal strategy are associated with deficits in performance (Peterson and Eckstein, 2012). Individuals with better memory for faces fixate more on, and have a longer total fixation time for, the eyes than do individuals with poorer memory for faces, although all individuals attend to the eyes relatively more than the other facial features (Sekiguchi, 2011). This suggests that more attention to the eyes, or just below the eyes, is associated with better face recognition, at least in adulthood.

Patterns of fixation to the face have also been linked to performance on face recognition tasks in individuals with autism, but few studies have examined this relationship. Snow et al. (2011) studied adolescents with and without autism as they completed recognition tests for faces and electric fans. Relative to TD adolescents, the adolescents with autism were less accurate at recognizing faces, but not fans. The number of fixations made to the face during encoding was significantly correlated with face recognition accuracy in the group with autism, but there was no such relationship in the TD group. Furthermore, TD adolescents made more fixations to faces than to fans, whereas adolescents with autism demonstrated no preference, suggesting that poorer face memory in autism may be related to reduced scanning of the face. Kirchner et al. (2011) found a significant negative correlation between the fixation time on the mouth and performance on the Cambridge Face Memory Test (CFMT) in adults with autism, but not in TD adults. For those with autism, looking at the mouth might be a learned strategy to deal with both language deficits and social discomfort, undermining face recognition but presenting a way to cope with other difficulties. Additionally, there was a trend-level relationship between fixation time on the eyes and social functioning in the adults with autism. Together, the evidence that face recognition difficulties become more robust into adulthood in autism, and that the manner in which visual attention is allocated has a functional role in face recognition, suggests that it is important to also study how patterns of fixation to the face change over late development typically and in autism, and how these changes might relate to face recognition ability.

This study examined the development of fixations to the face using a well-established face recognition task, the CFMT (Duchaine and Nakayama, 2006).¹ The CFMT is sensitive to the changes in face recognition ability during the transition from adolescence to adulthood (O’Hearn et al., 2010), as well as the link between patterns of attention and face recognition performance (Kirchner et al., 2011). The initial finding that there is a robust improvement in performance on the CFMT over adolescent development typically (O’Hearn et al., 2010) has been replicated in a large sample of TD individuals (Germine et al., 2011). Additionally, recent work using a face recognition task with low memory demands indicated that performance improves from childhood to adolescence in autism, on par with typical development, but not from adolescence to adulthood (O’Hearn et al., 2014). Our principle aims were to examine how patterns of fixation to facial features, namely, the eyes and mouth, change with age typically and in autism; whether this differential development might contribute to the lack of behavioral improvement in adulthood in individuals with autism; and whether patterns of fixation during face recognition are related to symptom severity in autism.

We hypothesized that both TD individuals and individuals with autism would fixate relatively more on the eyes than on the mouth. However, we predicted that individuals with autism would make fewer and shorter fixations on the eyes, as well as more and longer fixations on the mouth, as compared to TD individuals; we expected that these differences would become more robust by adulthood. We also predicted that these differences in fixation behavior would be relatively more pronounced in the test phases of the CFMT, as compared to the memorization phases, consistent with Weigelt et al.’s (2012) suggestion that, while face perception may be relatively typical in autism, deficits in performance emerge on tasks that impose even a minimal demand on memory. We hypothesized that fixations to the eyes would be positively correlated with performance on the CFMT and that fixations to the mouth would be negatively correlated with CFMT performance. Finally, we hypothesized that the number and durations of fixations on the eyes would be negatively correlated with measures of symptom severity in individuals with autism.

Methods

Participants

In total, 25 children (aged 7–12), 25 adolescents (aged 13–17), and 22 adults (aged 18–30) with autism, and 29 children, 25 adolescents, and 33 adults with typical development participated in the study. IQ was measured using the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999). Participants with full-scale IQ (FSIQ) scores less than 80 were excluded. IQ did not significantly differ between the groups with autism and the TD groups (allps > 0.35), although, within the child group, there was a trend toward higher scores on performance IQ (p = 0.07) and FSIQ (p = 0.14) in autism as compared to TD. Both groups consisted mostly of males due to the preponderance of males diagnosed with autism. No participant had a history of head injury, birth injury, or seizure disorder, and all participants had normal or corrected to normal visual acuity. One TD adult was dropped because he provided inaccurate personal information, and two adults with autism were dropped due to incomplete diagnostic information. Additionally, one child, two adolescents, and one adult with autism, and two TD adolescents and four TD adults were dropped due to technical errors during data collection resulting in inadequate data. Groups did not significantly differ in age or IQ after these individuals were dropped, all ps > 0.56. To increase our ability to delineate the typical developmental trajectory, a crucial step in examining atypicalities in autism, we used groups of unequal Ns, rather than substantially reduce the number of participants in the TD adult group to match the adult group with autism. See Table 2 for final participant demographic information.

Table 2.

Participant demographic information.

Measure	Children				p	Adolescents				p	Adults				p
	Autism		TD			Autism		TD			Autism		TD
	M	SD	M	SD		M	SD	M	SD		M	SD	M	SD
N	24		29			23		23			19		28
Gender	21 M, 3 F		23 M, 6 F			19 M, 4 F		19 M, 4 F			18 M, 1 F		23 M, 5 F
Age	11.2	1.7	11.4	1.4	0.68	15.3	1.5	15.4	1.5	0.87	24.2	4.9	24.0	5.3	0.94
Verbal IQ	108.7	13.0	105.6	10.9	0.35	106.3	13.2	108.4	12.0	0.58	109.8	13.0	109.4	11.4	0.92
Performance IQ	114.4	13.3	107.5	13.3	0.07a	107.1	13.1	107.1	8.9	0.99	110.3	15.9	112.0	12.2	0.67
Full-scale IQ	112.8	12.9	107.5	12.6	0.l4a	107.7	13.3	108.6	8.5	0.79	111.3	14.6	112.1	11.7	0.85
ADOS
Communication	3.4	1.4				3.9	1.2				4.4	1.5
Social	7.6	1.8				8.0	2.3				8.6	2.5
Full-scale	11.0	2.9				11.9	3.2				13.0	3.4
Calibrated severity score	6.6	1.6				6.7	1.6				7.6	1.6
ADI-R
Social	19.0	5.9				20.0	5.3				22.1	3.7
Communication	16.3	3.9				15.4	3.8				16.8	4.1
Behavior	6.4	2.1				6.1	2.4				6.4	2.6
Abnormality	3.3	1.3				2.8	1.7				3.0	l.l

IQ: intelligence quotient; TD: typically developing; ADOS: Autism Diagnostic Observation Schedule; ADI-R: Autism Diagnostic Interview-Revised; SD: standard deviation.

^aUsing a cut-off of p > 0.20 for accepting the null hypothesis, only two trends emerged, both in the child group. These trends hinted at higher performance IQ and full-scale IQ scores in the children with autism as compared to the TD children.

Participants with autism were recruited through the University’s Autism Center of Excellence (ACE) subject core (HD #055748). Autism diagnoses were determined based on current Diagnostic and Statistical Manual of Mental Disorders (DSM) criteria at the time (American Psychiatric Association, 2000) using the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 1989) and the Autism Diagnostic Interview-Revised (ADI-R; Lord et al., 1994) and were confirmed by expert clinical assessment.² Individuals in the autism group met cut-offs for autism on the ADI-R (except one individual on section D, Abnormality of Development Evident at or Before 36 Months) and for autism or autism spectrum disorder (ASD) on the ADOS. ADOS full-scale scores were transformed into a calibrated severity score (CSS), a continuous variable more appropriate for most statistics, using algorithms from Gotham et al. (2009) for participants receiving Module 3 and Hus and Lord (2014) for participants receiving Module 4. The CSS is intended to be an indicator of autism symptom severity relative to age and language level. ADOS, ADI-R, and CSS did not significantly differ between age groups in the group with autism, although there was a trend toward higher ADOS communication sub-scores in adults relative to children, p = 0.06. Three children, one adolescent, and two adults with autism also had a current, comorbid diagnosis of attention-deficit hyperactivity disorder (ADHD); how-ever, we found no significant interaction between ADHD diagnosis and age in the group with autism (p = 0.62).

TD participants were recruited through the ACE subject core and through other studies being conducted at our laboratory. Recruitment methods included web postings and flyers posted throughout the area. TD participants had no personal or family history of any psychiatric or neurologic disorder, developmental delay, or learning disability.

Approval for the study was obtained from the university’s Institutional Review Board. Written informed consent was given by all participants, or by the parents of minor participants, prior to participation.

Procedure

Participants completed the CFMT as a part of a larger battery of behavioral activities. Stimuli were gray-scale photographs of young adult, Caucasian male faces with neutral expressions (Figure 1). Images were cropped to remove hair and other identifying non-face features. The stimuli were displayed on a gray background. Six faces were designated as target faces. The task consists of three conditions: same images (condition 1), novel images (condition 2), and novel images with noise (condition 3). Each condition consists of a memory phase, during which participants learn the target face(s), and a test phase, during which recognition is tested. In each test trial, participants selected the target face from two distractors. A face was never used as both a target and a distractor, although distractors were repeated. Chance performance on this task is 33.3%. More details on this method can be found in the work by Duchaine and Nakayama (2006). Completing the task took between 10 and 15 min.

Figure 1.

(a) Trial from condition 1. Participants studied the target face from three different angles and then identified the target from two distractors. (b) Study stimuli for conditions 2 and 3. Participants reviewed the six target faces for 20 s. (c) Test stimuli for condition 2, with target faces in a novel pose/lighting. (d) Test stimuli for condition 3, with target faces in a novel pose/lighting and added Gaussian noise.

Apparatus

The task was presented on a Hanns-G 19-in screen (41.4 cm × 26 cm) with a resolution of 1440 × 900 pixels. The viewing distance to the screen was set at 60 cm, and a chin and forehead rest was used to constrain head movement. At this viewing distance, 1.05 cm (17.78 pixels) on the display was equivalent to 1° of visual angle. Eye position throughout the task was recorded using an Applied Science Laboratories (ASL) EYE-TRAC 6 system with remote pan/tilt optics, with a sampling rate of 60 Hz. The accuracy of the eye-tracker was 0.5° of visual angle. The eye-tracking camera was centered and its height was adjusted so that it was positioned just below the monitor. Overhead lights in the testing room were dimmed to reduce glare and maximize pupil size. Before administering the experiment, the eye-tracker was calibrated for each participant using a standard, evenly distributed 9-point stimulus grid. Calibration verification was performed by asking the participant to re-scan the 9-point grid and confirmed by a trained research assistant. The ASL software determined gaze position by computing distances between the corneal reflection and pupil center.

Regions of interest

Using Adobe Photoshop CS4, we defined four regions of interest (ROIs) for each image. These ROIs included both eyes, the nose, the mouth, and the remainder of the face with the other three features subtracted (Figure 2). Each ROI was defined starting with the upper left-hand coordinates and included an additional 8 pixel (0.5° of visual angle) border around each feature to account for variance in the recording and drift. The ROIs did not overlap. ROIs were defined individually for each face to accommodate for slight variability in the size of features across different images. Since we had explicit hypotheses for the eyes and mouth, we concentrate on these ROIs in this article. However, all ROIs were analyzed, and results for the nose and remainder of face ROIs are presented in the Supplementary Material.

Figure 2.

Example of the ROIs defined for each face image. The ROIs included both of the eyes, the nose, the mouth, and the remainder of the face (which did not include the other features).

Eye data analysis

Raw eye data were first examined to ensure that data quality was consistent across groups. The percentage of data collected while subjects were looking off-screen did not differ between groups (p = 0.831). The percentage of valid, on-screen data was greater in the TD group, with 91.0%, as compared to 85.7% in the group with autism (p = 0.005). However, this difference was due to a significantly higher percentage of blinks in the group with autism (11.9%) than in the TD group (6.5%; p < 0.001), consistent with evidence that blinking may differ in individuals with ASDs (Shultz et al., 2011). These blink points were filtered out prior to data analysis.

A fixation was defined as a sustained look, occurring within 1° of visual angle and lasting at least 100 ms. The ILab package (adapted for in-house analysis; Gitelman, 2002) for MATLAB (Mathworks, Natick, MA) was used to identify fixations using these parameters and to subsequently loop through each fixation to determine whether it occurred within one of the ROIs during that trial. Using these data, we then calculated two fixation measures for each ROI: (1) average fixation duration and (2) fixation count. Average fixation duration was defined as the average duration of the fixations occurring within each ROI. Fixation duration was calculated as the number of samples of at least 100 ms, within 1° of visual angle of one another, multiplied by the sampling rate of the eye tracker (60 Hz). Fixation count was defined as the total number of fixations made within each ROI. It was calculated as the number of times a set of samples within 1° of visual angle of one another, lasting at least 100 ms, occurred within a given ROI.

Statistical analysis

Behavioral performance.

An initial repeated measures analysis of variance (ANOVA) examined CFMT performance with condition (1, 2, 3) as the within-subjects variable and group (TD and autism) and age (children, adolescents, and adults) as between-subjects factors. The Greenhouse-Geisser correction was applied to account for violations of sphericity. The ANOVA revealed a significant main effect of condition (F(1.734, 242.747)=417.663, p < 0.001, ${\eta }_p^2$ = 0.749), as well as an interaction of condition and age (F(3.468, 242.747) = 2.751, p = 0.036, ${\eta }_p^2$ = 0.038). Follow-up Helmert contrasts revealed that the interaction of condition and age was significant when comparing condition 1 to conditions 2 and 3 (F(2, 140) = 4.074, p = 0.019, ${\eta }_p^2$ = 0.055), but not when comparing condition 2 to condition 3 (F(2, 140) = 0.003, p = 0.997, ${\eta }_p^2$ = 0–001). Thus, because this difference was not significant, and because these conditions have the same format, we collapsed across conditions 2 and 3 for all further analyses. Condition 1 is henceforth referred to as immediate recognition, while conditions 2 and 3 are collectively referred to as delayed recognition.

The mean, standard deviation, and range of scores on the CFMT for each group and age group are reported in Table 3. Results for analyses of CFMT performance are presented in the Supplementary Material. Importantly, we replicated previous work (O’Hearn et al., 2010), finding significant improvement in performance on the CFMT during the transition from adolescence to adulthood in typical development, but not in autism.

Table 3.

Descriptive statistics for behavioral performance on the CFMT.

CFMT condition	Group	Age group	M	SD	Range
Immediate recognition	TD	Children	0.90	0.11	0.39
		Adolescents	0.94	0.07	0.28
		Adults	0.99	0.02	0.11
	Autism	Children	0.81	0.20	0.67
		Adolescents	0.87	0.17	0.50
		Adults	0.80	0.24	0.89
Delayed recognition	TD	Children	0.52	0.16	0.66
		Adolescents	0.57	0.17	0.68
		Adults	0.71	0.17	0.56
	Autism	Children	0.42	0.08	0.31
		Adolescents	0.51	0.13	0.44
		Adults	0.50	0.16	0.56

SD: standard deviation; CFMT: Cambridge Face Memory Test; TD: typically developing.

Fixation measures.

For the fixation measures, all memorization and test trials (i.e. correct and incorrect) were analyzed. We first analyzed total fixation time within all ROIs to ensure that one group or age group did not simply attend more or less to the faces in the display. Next, to test our hypothesis that both TD individuals and individuals with autism would fixate relatively more on the eyes than on the mouth, average fixation duration and fixation count were analyzed with repeated measures ANOVAs with condition (immediate recognition and delayed recognition), trial type (memory and test), and ROI (eyes and mouth) as within- subjects repeated factors, and group (TD and autism) and age group (children, adolescents, and adults) as between-subjects factors. The Greenhouse-Geisser correction was applied to account for violations of sphericity. In order to examine group- and age-related differences within each ROI, 2 (group) × 3 (age group) univariate ANOVAs were performed separately for each ROI. If effects of age were present, post hoc analyses included one-way ANOVAs for age performed separately in each group. Additionally, to examine a priori questions about whether there were group differences at each age, independent-samples t-tests comparing individuals with and without autism were performed separately in each age group.

Finally, exploratory correlations were performed in order to examine the relationships between patterns of fixations and performance and, in the group with autism, between patterns of fixations and social functioning. We used partial correlations, controlling for age and FSIQ, to minimize the possibility of a third variable driving a significant effect. Since these analyses were underpowered and exploratory, and are likely to address distinct questions, we did not correct for multiple comparisons; results should therefore be interpreted with caution.

Results

Total fixation time

We first examined overall total fixation time within all ROIs combined (eyes, nose, mouth, and remainder of the face) for the entire task, collapsed across condition and trial type, to ensure that one group or age group was not simply looking more or less at the faces in the display. A univariate ANOVA with total fixation time (s) as the dependent variable and group (TD and autism) and age (children, adolescents, and adults) as fixed factors did not reveal any significant effects or interaction of group or age (ps > 0.22). Thus, across age and group, all participants showed similar amounts of looking to the faces throughout the task.

Patterns of fixations to eyes versus mouths

To test our hypothesis that both TD individuals and individuals with autism would fixate relatively more on the eyes than on the mouth, we next examined whether patterns of fixations to these ROIs differed across group and age with two initial repeated measures ANOVAs, with either average fixation duration or fixation count as the dependent variable. Condition (immediate recognition and delayed recognition), trial type (memory and test), and ROI (eyes and mouth) were within-subjects factors, and group (TD and autism) and age (children, adolescents, and adults) were between-subjects factors.

Significant results are reported in Table 4 (all other ps > 0.131). Importantly, we found that there were main effects of ROI for both average fixation duration (F(1, 24) = 12.012, p = 0.002, ${\eta }_p^2$ = 0.334) and fixation count (F(1, 140) = 96.881, p < 0.001, ${\eta }_p^2$ = 0.409), with significantly longer average fixation duration³ and greater numbers of fixations for the eye ROIs than for the mouth ROIs, respectively. ROI did not interact with group and/or age (ps > 0.131). There were also no significant main effects of group or age (ps > 0.299), nor an interaction of group × age for either fixation measure (ps > 0.875). Thus, consistent with our predictions, all groups and age groups made more and longer fixations to the eye ROIs than to the mouth ROIs during the task.

Table 4.

Significant main effects and interactions of condition, trial type, ROI, group, and age for each fixation measure.

Fixation measure	Main effect or interaction	Statistics
Average fixation duration	Condition	F(1, 24) =16.312, p < 0.001, ${\eta }_p^2$ = 0.405
	Trial type	F(1, 24) = 24.902, p < 0.001, ${\eta }_p^2$ = 0.593
	ROI	F(1, 24) =12.012, p = 0.002, ${\eta }_p^2$ = 0.334
	Condition × Trial type	F(1, 24) = 48.665, p < 0.001, ${\eta }_p^2$ = 0.670
	Condition × ROI	F(1, 24) = 5.361, p = 0.029, ${\eta }_p^2$ = 0.183
	Trial type × ROI	F(1, 24) =15.445, p = 0.001, ${\eta }_p^2$ = 0.392
	Condition × T rial type × ROI	F(1, 24) = 9.588, p = 0.005, ${\eta }_p^2$ = 0.285
Fixation count	Condition	F(1, 140) = 26.322, p < 0.001, ${\eta }_p^2$ = 0.158
	Trial type	F(1, 140) = 242.202, p < 0.001, ${\eta }_p^2$ = 0.634
	ROI	F( 1, 140) = 96.881, p < 0.002, ${\eta }_p^2$ = 0.409
	Condition × Age	F(1, 140) = 3.629, p = 0.029, ${\eta }_p^2$ = 0.049
	Condition × Trial type	F(1, 140) =154.774, p < 0.001, ${\eta }_p^2$ = 0.525
	Condition × T rial type × Age	F(2, 140) = 3.143, p = 0.046, ${\eta }_p^2$ = 0.043
	Condition × ROI	F(1, 140) =13.551, p < 0.001, ${\eta }_p^2$ = 0.088
	Trial type × ROI	F( 1, 140) = 58.480, p < 0.001, ${\eta }_p^2$ = 0.295
	Condition × T rial type × ROI	F(1, 140) = 58.520, p < 0.001, ${\eta }_p^2$ = 0.295

ROI: region of interest.

We next examined patterns of fixations to the eye and mouth regions separately due to our explicit hypotheses about the effects of group and age within each ROI. Because ROI interacted with condition and trial type, we performed these analyses separately for each trial type within each condition. We predicted that the groups with autism would make shorter and fewer fixations on the eyes, as well as longer and more fixations on the mouth, as compared to the TD group; we expected these differences to become more robust by adulthood. We also expected to find that these effects would be more robust in test trials than in memory trials.

Patterns of fixations to eyes

Immediate recognition: Memory trials

Average fixation duration.

A univariate ANOVA with average fixation duration for the eyes during immediate recognition memory trials as the dependent variable and group (TD and autism) and age (children, adolescents, and adults) as fixed factors revealed a significant main effect of group (F(1, 125) = 4.105, p = 0.045, ${\eta }_p^2$ = 0.032), with the TD group making longer fixations to the eyes than the group with autism. Although there was no main effect of age (p = 0.965), a group × age interaction was present at trend level (F(2, 125) = 2.397, p = 0.095, ${\eta }_p^2$ = 0.037), suggesting that group differences may change with age. Follow-up independent-samples t-tests revealed that average fixation duration for the eyes was significantly longer in TD children than in children with autism (t(48) = 3.338, p = 0.002, d = 0.981); however, groups did not differ in adolescence or adulthood (ps > 0.320; Figure 3(a)). One-way ANOVAs examining age-related changes in average fixation duration for the eyes separately in each group did not reveal significant effects of age in the TD group or in the group with autism (ps > 0.349).

Figure 3.

(a) During immediate recognition memory trials, average fixation duration for the eyes was significantly longer in typically developing (TD, dark gray bars) children than in children with autism (ASD, light gray bars). (b) Fixation count for the eyes did not differ between groups.

Fixation count.

A similar univariate ANOVA with fixation count for the eyes during immediate recognition memory trials as the dependent variable did not reveal any significant effects or interaction of group or age (ps > 0.191; Figure 3(b)).

Immediate recognition: Test trials.

Univariate ANOVAs did not reveal any significant effects or interaction of group or age on either average fixation duration or fixation count for the eyes during immediate recognition test trials (ps > 0.151).

Delayed recognition: Memory trial

Average fixation duration.

A univariate ANOVA with average fixation duration for the eyes during the delayed recognition memory trial as the dependent variable did not reveal any effects or interaction of group or age (ps > 0.139; Figure 4(a)).

Figure 4.

(a) During the delayed recognition memory trial, average fixation duration for the eyes was similar across groups and age groups. (b) Fixation count for the eyes during the delayed recognition memory trial was higher in typically developing (TD, dark gray bars) children and adults as compared to their peers with autism (ASD, light gray bars). However, groups did not differ in adolescence. Fixation count for the eyes increased from childhood to adolescence in autism, but did not change with age typically.

Fixation count.

A univariate ANOVA with fixation count for the eyes during the delayed recognition memory trial as the dependent variable revealed a significant main effect of group (F(1, 140) = 7.501, p = 0.007, ${\eta }_p^2$ = 0.051), with the TD group making more fixations to the eyes than the group with autism (Figure 4(b)). There was no main effect of age or interaction of group and age (ps > 0.122). Follow-up independent-samples t-tests indicated a significant between-group difference in childhood (t(51) = 3.290, p = 0.002, d = 0.919) and a trend toward a significant group difference in adulthood (t(45) = 1.850, p = 0.071, d = 0.573), with TD individuals making more fixations to the eyes relative to their peers with autism. Groups did not differ in adolescence (p = 0.979). Additionally, follow-up one-way ANOVAs examining age-related changes in fixation count for the eyes separately in each group revealed that the number of fixations made to the eyes did not change with age typically (p = 0.604). However, there was a marginally significant main effect of age in the group with autism (F(2, 65) = 3.044, p = 0.055, f= 0.311), driven by a significant increase in the number of fixations made to the eyes from childhood to adolescence (p = 0.043). There were no significant changes from adolescence to adulthood (p = 0.411) or from childhood to adulthood (p = 0.549) in the group with autism.

Delayed recognition: Test trials.

Univariate ANOVAs did not reveal any significant effects or interaction of group or age on either average fixation duration or fixation count for the eyes during delayed recognition test trials (ps > 0.148).

Patterns of fixations to mouths

Immediate recognition: Memory trials

Average fixation duration.

A univariate ANOVA with average fixation duration for the mouth during immediate recognition memory trials as the dependent variable and group (TD and autism) and age (children, adolescents, and adults) as fixed factors revealed a trend-level main effect of age (F(2, 99) = 2.492, p = 0.088, ${\eta }_p^2$ = 0.048). Post hoc Tukey’s tests revealed that, across both groups, there was a trend toward a decrease in average fixation duration for the mouth from adolescence to adulthood (p = 0.096), but there was no change from childhood to adolescence or from childhood to adulthood (ps > 0.135). Follow-up oneway ANOVAs examining age-related changes in average fixation duration for the mouth separately in each group did not reveal any effects of age (ps > 0.148). The initial ANOVA did not reveal a main effect of group or interaction of group and age (ps > 0.482).

Fixation count.

A similar univariate ANOVA with fixation count for the mouth during immediate recognition memory trials as the dependent variable did not reveal any effects or interaction of group or age (ps > 0.173).

Immediate recognition: Test trials

Average fixation duration.

A univariate ANOVA with average fixation duration for the mouth during immediate recognition test trials as the dependent variable revealed a significant group × age interaction (F(2, 106) = 3.984, p = 0.021, ${\eta }_p^2$ = 0.070; Figure 5(a)), although main effects of group and age did not reach significance (ps > 0.287). Follow-up one-way ANOVAs examining age-related changes separately in each group revealed a significant main effect of age in the TD group (F(2, 55) = 3.312, p = 0.044, f= 0.354), but not in the group with autism (p = 0.473). The age effect in the TD group was driven by a trend-level increase in average fixation duration for the mouth from childhood to adolescence (p = 0.089) and a trend-level decrease from adolescence to adulthood (p = 0.071). Follow-up independent-samples t-tests revealed trend-level group differences in children (t(38) = −1.698, p = 0.098, d= 0.537) and adults (t(19.517) = −1.984, p = 0.062, d = 0.739), with shorter average fixation duration for the mouth in the TD groups as compared to their peers with autism; groups did not differ in adolescence (p = 0.126).

Figure 5.

(a) During immediate recognition test trials, average fixation duration for the mouth increased from childhood to adolescence and decreased from adolescence to adulthood typically (TD, dark gray bars), but did not change with age in autism (ASD, light gray bars). TD children and adults made shorter fixations to the mouth than did their peers with autism, although groups did not differ in adolescence. (b) Fixation count for the mouth increased from childhood to adolescence typically, but did not change with age in autism. Children with autism made more fixations to the mouth than did TD children; however, this pattern reversed by adolescence, with the TD group making more fixations on the mouth relative to the group with autism. Groups did not differ in adulthood.

Fixation count.

A similar univariate ANOVA with fixation count for the mouth during immediate recognition test trials as the dependent variable revealed a significant group × age interaction (F(2, 140) = 3.206, p = 0.044, ${\eta }_p^2$ = 0.044; Figure 5(b)), although main effects of group and age did not reach significance (ps > 0.435). Follow-up one-way ANOVAs examining age-related changes separately in each group revealed a trend-level main effect of age in the TD group (F(2, 79) = 2.775, p = 0.069, f= 0.269), but not in the group with autism (p = 0.336). Post hoc Tukey’s test for age indicated that the age effect in the TD group was driven by an increase in fixation count for the mouth from childhood to adolescence (p = 0.076). Additionally, follow-up independent-samples t-tests indicated trend-level group differences in childhood and adolescence. Children with autism made more fixations to the mouth than did TD children (t(31.748) = −1.723, p = 0.095, d = 0.489). However, TD adolescents made more fixations to the mouth than did adolescents with autism (t(28.572) = 1.841, p = 0.076, d = 0.543). Groups did not differ in adulthood (p = 0.963).

Delayed recognition: Memory trial.

Univariate ANOVAs did not reveal any significant effects or interaction of group or age on either average fixation duration or fixation count for the mouth during the delayed recognition memory trial (ps > 0.243).

Delayed recognition: Test trials

Average fixation duration.

A univariate ANOVA with average fixation duration for the mouth during delayed recognition test trials as the dependent variable revealed a significant group × age interaction (F(2, 98) = 3.271, p = 0.042, ${\eta }_p^2$ = 0.063; Figure 6(a)), although main effects of group and age did not reach significance (ps > 0.353). Follow-up one-way ANOVAs examining age-related changes separately in each group revealed a trend-level main effect of age in the TD group (F(2, 54)=2.933, p = 0.062,f= 0.336), but not in the group with autism (p = 0.500). Post hoc Tukey’s tests indicated a trend toward increased average fixation duration for the mouth during the transition from childhood to adolescence typically (p = 0.059), but not from childhood to adulthood (p = 0.204) or from adolescence to adulthood (p = 0.858). Follow-up independent-samples t-tests performed separately in each age group revealed a significant group difference in childhood, with shorter average fixation duration for the mouth in TD children relative to children with autism (t(35)=−2.619, p = 0.013, d = 0.854). Groups did not differ in adolescence (p = 0.207) or adulthood (p = 0.232).

Figure 6.

(a) During delayed recognition test trials, average fixation duration for mouths changed with age typically (TD, dark gray bars), increasing from childhood to adolescence, but not in autism (ASD, light gray bars). TD children made shorter fixations to the mouth than did children with autism, but groups did not differ in adolescence or adulthood. (b) Fixation count for the mouth did not significantly differ between groups and age groups during delayed recognition test trials.

Fixation count.

A similar univariate ANOVA with fixation count for the mouth during the delayed recognition test trials as the dependent variable did not reveal any effects or interaction of group or age (ps > 0.103; Figure 6(b)).

An overall summary of between-group differences at each age for both the eye and mouth ROIs is presented in Table 5.

Table 5.

Summary of group differences for eyes and mouth in each condition and trial type.

Condition	Trial type	Eyes				Mouth
		Average fixation duration	p	Fixation count	p	Average fixation duration	p	Fixation count	p
Immediate recognition	Memory	Child: TD > ASD	0.002
	Test					Child: ASD > TD	0.10	Child: ASD > TD	0.10
						Adult: ASD > TD	0.06	Adol: TD > ASD	0.08
Delayed recognition	Memory			Child: TD > ASD Adult: TD > ASD	0.002 0.07
	Test					Child: ASD > TD	0.01

ASD: autism spectrum disorder; TD: typically developing.

Correlations

Performance.

We expected to find that more and longer fixations to the eyes would be positively correlated with performance and that more and longer fixations to the mouth would be negatively correlated with performance, but only in the TD group. We performed two-tailed, partial correlations in each group separately based on this assumption that the relationships between patterns of fixation to the face and performance on the CFMT might be different in typical development and in autism. In spite of clear predictions for the TD group, we chose to use two-tailed analyses because we thought the direction of the effects might differ in the group with autism, even though it decreased our power to see effects typically; these results should therefore be interpreted with caution. Although FSIQ was only occasionally related to the fixation measures in the TD group, we controlled for FSIQ and age (in years and months) for both groups in our analyses. This was the conservative approach to minimize the possibility that these variables or a related third variable were driving the correlations. We did not correct for multiple comparisons as we considered these analyses exploratory.

In the TD group, we found that performance on the immediate recognition condition was positively associated with the number of fixations made to the eyes during immediate recognition memory trials (r(35) = 0.501, p = 0.002; all other ps > 0.226). Performance on the delayed recognition condition was negatively related to the number of fixations made to the mouth during delayed recognition test trials (r(23) = −0.401, p = 0.047; all other ps > 0.148).

In the group with autism, performance on the immediate recognition condition was not correlated with any fixation measures (ps > 0.134). Performance on the delayed recognition condition was positively associated with the number of fixations made to the mouth during delayed recognition test trials (r(15) = 0.516, p = 0.034; all other ps > 0.128).

Autism symptom severity.

We performed similar correlations to compare fixation measures to autism symptom severity as measured by ADOS CSS. ADOS CSS was not correlated with any fixation measures during immediate recognition (ps > 0.234). However, during delayed recognition test trials, average fixation duration for the eyes was negatively associated with symptom severity as measured by the ADOS CSS at trend level (r(58) = −0.188, p = 0.075), consistent with our original predictions.

We also analyzed whether autism symptom severity was related to performance on the CFMT in the group with autism; however, ADOS CSS was not correlated with performance on any condition of the CFMT or the task overall (ps > 0.114).

Discussion

The goal of this study was to characterize the development of patterns of fixation to the face both typically and in autism, and to determine whether differences in patterns of fixations help to explain face recognition difficulties, which increase with age, in autism. Children, adolescents, and adults with and without autism completed the CFMT, while we recorded the duration and number of fixations made to the eyes and mouth regions of the face stimuli used in the task. Overall, we found that both the TD group and the group with autism consistently made longer and more fixations to the eyes than to the mouth. This result supports the idea that, rather than allocating excess attention to the mouth and diminished attention to the eyes, individuals with autism look at faces in a relatively typical pattern (Falck-Ytter and Von Hofsten, 2011), although group differences in the amount of attention within a given feature may be present (Rutherford and Towns, 2008; Speer et al., 2007). Indeed, we did find that, despite a relatively typical pattern of overall looking, patterns of fixation within the eyes and mouth ROIs sometimes differed between groups; importantly, these between-group differences were dependent upon the age group examined.

For the eyes, we predicted that the TD group would fixate more on these ROIs as compared to the group with autism and that these differences would become more robust over development. Furthermore, we expected that fixating on the eyes would be positively correlated with CFMT performance typically and negatively correlated with symptom severity in autism. In childhood, we found that the TD group made more fixations to the eyes than did the group with autism. This finding is consistent with Yi et al. (2013), who also found that children (age 5–10 years) with autism fixated less on the eye regions of static face stimuli during a face recognition task than did TD children. Although another study using static face stimuli in a face recognition task found no differences in fixations on the eyes between children with and without autism, this may be due to the considerably younger age of their sample (Chawarska and Shic, 2009). In adolescence, we found no differences in patterns of fixation to the eyes between individuals with and without autism. This result was surprising as the majority of the literature has found that TD adolescents fixate more on eyes than do their peers with autism (Klin et al., 2002; Norbury et al., 2009; Speer et al., 2007). However, these studies used dynamic scene stimuli and passive viewing tasks. The task differences are likely to contribute, at least in part, to the inconsistent results. Indeed, our findings do mirror those of the only study that utilized static faces in a recognition task to examine face processing in adolescents with autism, which found no differences across groups (Snow et al., 2011). In adulthood, we found that the TD group made more fixations to the eyes than did the group with autism, consistent with our predictions and with other studies that have used static stimuli in face recognition tasks (Boraston et al., 2008; Kirchner et al., 2011; Sterling et al., 2008).

Between-group differences in fixation patterns to the eyes evident in childhood were no longer evident in adolescence, but reemerged in adulthood, suggesting that subtle age-related changes were occurring over these periods of development. Indeed, we found that the number of fixations made to the eyes increased from childhood to adolescence in individuals with autism. However, there were no age-related changes from adolescence to adulthood or overall from childhood to adulthood in this group. Although these results could potentially be due to cohort effects, it could also be possible that individuals with autism make typical gains early on in development, “catching up” with TD individuals by adolescence, but that these gains are lost during the transition into adulthood. Unexpectedly, we found that patterns of fixations to the eyes did not change over development typically. It could be that TD individuals reach a “mature” level of fixating on the eyes early on in childhood that is stable over the course of later development. Future research examining younger age groups than our current sample may be able to elucidate how these fixation patterns develop typically.

Between-group differences and age-related changes in patterns of fixations to the eyes were only evident during memory trials of the CFMT. Additionally, the number of fixations made to the eyes during immediate recognition memory trials was positively related to performance on this condition of the CFMT typically, but not in autism. These findings suggest that information from the eye region may be most important when memorizing a face and that insufficient encoding of the eyes during memorization may contribute to face recognition difficulties in autism. Indeed, there seems to be an important relationship between attention to the eyes and activity in the fusiform gyrus, a region in the ventral stream thought to underlie face processing, in individuals with autism. Atypical activation of the fusiform gyrus in response to faces has been reported in autism (Malisza et al., 2011; Schultz et al., 2000), though the evidence is somewhat inconsistent (e.g. Hadjikhani et al., 2004). In adults with autism, fixations on the eye region of a face have been shown to be positively correlated with activity in the fusiform gyrus, suggesting that patterns of attention to the face may be the mediator between atypical fusiform activity and poorer face recognition skill (Dalton et al., 2005).

For the mouth, we hypothesized that the group with autism would look more at these ROIs than would the TD group and that this pattern would become more pronounced by adulthood. In line with our initial predictions, we found that the group with autism fixated more on the mouth than did the TD group and that patterns of fixations to the mouth changed with age typically, although our results were more complex than expected. We found the most consistent between-group differences in childhood, with the TD group fixating less on the mouth relative to the group with autism, in line with work by Jones et al. (2008). However, we found either no group differences, consistent with most extant literature (Norbury et al., 2009; Snow et al., 2011; Speer et al., 2007), or the opposite pattern of group differences—with the TD group fixating more on the mouth than the group with autism— in adolescence, due to an unexpected increase in fixations on the mouth from childhood to adolescence typically. By adulthood, individuals with autism again made longer fixations to the mouth than did TD individuals. This result was consistent with our initial predictions, although it conflicts with previous work, which has found no significant differences in patterns of fixations to the mouth between TD adults and adults with autism (Boraston et al., 2008; Hernandez et al., 2009; Kirchner et al., 2011; Neumann et al., 2006; Pelphrey et al., 2002; Rutherford and Towns, 2008).

We found that significant between-group differences and age-related changes for the mouth were only evident in test trials of the CFMT. This suggests that, during memorization, fixations to the mouth are relatively stable across group and age. However, when recognizing a face, strategies for using information from the mouth do differ between groups and change over development. Furthermore, fixations to the mouth during delayed recognition test trials were negatively correlated with performance on that condition of the CFMT typically, but were positively associated with performance in the group with autism. These results are consistent with recent work (Wagner et al., 2013), which found that increased attention to the mouth was associated with slower neural responsiveness to faces in TD adolescents, although no such relationship was present in autism. Thus, attention to the mouth seems to be maladaptive for face recognition skill in TD individuals, but may be a compensatory strategy in individuals with autism.

Although these results are encouraging, there are limitations that we would like to acknowledge. First, some of our results were only significant at trend level and thus must be interpreted with caution. Furthermore, although our overall sample size was quite large, the sample sizes of specific subgroups compared in follow-up a-priorihypothesis-driven analyses were relatively small. Due to the heterogeneity of autism, these small sub-sample sizes constrain the generalizability of our results. We used cross-sectional data, which do not allow us to analyze and compare the shape of developmental trajectories in TD individuals and in autism. The use of cross-sectional data also restricts the interpretation of our findings as results may reflect cohort effects; it is possible that younger groups of participants have already begun receiving interventions aimed at alleviating social challenges, such as by increasing attention to the eyes. Longitudinal data are currently being analyzed by our laboratory, which will hopefully provide further insight into the effects we have observed in this study. There were also some limitations associated with the eye-tracking methodology in this study. Eye data were not recorded during the initial calibration process, so, although calibration was verified by a trained research assistant prior to the experiment, we have no way to assess this aspect of data quality quantitatively. Additionally, we wanted to use the CFMT in our study because previous work indicated that it was sensitive to development. However, while the average size of our ROIs was considerably larger than the size of a fixation (in terms of pixels), the face stimuli in the task were sometimes quite small relative to the size of the overall display. While this may have impacted our fixation measures slightly, it is unlikely that the impact differed across groups. The contributions that fixation patterns do make to recognition performance appear to be subtle. One possibility is that instead of fixations to a given ROI, suggesting that individuals are encoding individual features, placement of fixations in a central location may instead allow individuals to get a holistic or global representation of the face; this might be an optimal strategy for face recognition. Future research should characterize the developmental trajectory of optimal fixation strategies typically and determine how other groups, including individuals with autism, deviate from this trajectory.

This work illustrates the importance of considering age and task (and likely other individual characteristics) when discussing how individuals with autism might differ from TD individuals in their patterns of fixation to faces or to other stimuli. Patterns of fixation are important for face recognition ability and contribute to between-group and age-related differences in performance typically and in autism, as is evident in our data, and in activation of the fusiform gyrus, as others have shown; however, changes in fixation patterns are clearly not the entire story. Our work highlights the importance of both patterns of fixation and other visual processes, such as holistic encoding, in explaining the differences in face recognition skill in autism. It also indicates that developmental changes and compensatory processes must be considered when discussing visual processing in autism and that these issues contribute to the inconsistencies in the literature on patterns of fixation to faces in autism.