Autism and the development of face processing

Abstract

Autism is a pervasive developmental condition, characterized by impairments in non-verbal communication, social relationships and stereotypical patterns of behavior. A large body of evidence suggests that several aspects of face processing are impaired in autism, including anomalies in gaze processing, memory for facial identity and recognition of facial expressions of emotion. In search of neural markers of anomalous face processing in autism, much interest has focused on a network of brain regions that are implicated in social cognition and face processing. In this review, we will focus on three such regions, namely the STS for its role in processing gaze and facial movements, the FFA in face detection and identification and the amygdala in processing facial expressions of emotion. Much evidence suggests that a better understanding of the normal development of these specialized regions is essential for discovering the neural bases of face processing anomalies in autism. Thus, we will also examine the available literature on the normal development of face processing. Key unknowns in this research area are the neuro-developmental processes, the role of experience and the interactions among components of the face processing system in shaping each of the specialized regions for processing faces during normal development and in autism.

1. Introduction

Faces are rich conduits of personal information. During a brief encounter, healthy adults often automatically attend to and quickly perceive the complex set of information contained in a face, recognizing the emotional state and social context, and often remembering the individual face later. This complex task of face processing in normal adults involves a distributed neural system in which specific loci are implicated in processing-specific facial information. For example, a region along the superior temporal sulcus (STS) is involved in detecting facial movements associated with eye gaze, speech, and emotional expression and intention [1-3]. The amygdala responds to faces, especially fearful faces [4-6]. And a region in the ventral-occipital cortex, the “fusiform face area” (FFA) [7] is implicated in face detection, categorization and identity recognition [7-12]. Some components of the face processing system may exist at birth, while others continue to develop during childhood and adolescence before reaching the adult level [13,14]. Some basic goals of face processing research are to determine the specific function of each of the components of the face processing system, the time course and mechanism of development of each component, and the interactions among the various components subserving normal face processing (Fig. 1).

Fig. 1.

(from Kalanit Grill-Spector) Top: Posterior view of an inflated brain show regions that respond more to faces than to other visual categories (abstract object, scenes, and scrambled images). Several face selective regions shown in shades of yellow and orange are shown in the occipitalcortex, superior temporal sulcus (STS) and fusiform gyrus (i.e., “fusiform face area”, FFA). A motion senstive region (MT) and retinotopic visual areas are also shown in the occipital cortex. Bottom: Ventral view of the FFA in relation to the collateral sulcus (CoS).

Interest in the neural mechanisms of face processing and its development is partly fueled by several developmental conditions such as autism, which are associated with anomalous face processing. Face processing impairments in autism are the focus of intense investigation, given the importance of faces in conveying social and emotional information starting soon after birth. Although there is a general agreement that autism involves deficits in face processing, several questions remain. For example, the precise nature of these deficits, the underlying mechanisms and the relationships between anomalous face processing and atypical socio-emotional function in autism remain unclear. This lack of clarity is likely due to a number of factors, although it is probable that the heterogeneity of risk factors and variety of pathogenetic mechanisms associated with autism are particularly important [15-17]. Thus, given the complexity of the face processing system, there may exist subpopulations among individuals with idiopathic autism showing deficits in different components of this essential neurocognitive system. Also contributing to uncertainty in understanding face processing deficits in autism is a relative lack of knowledge concerning the longitudinal development of neural systems underlying face processing in healthy children, adolescents and adults. Thus, research on core components of face processing and their neuromaturational time course in normal development may augment understanding of face processing deficits in autism and their relation to social and emotional maturation. Ultimately, such work may provide useful clinical tools for early diagnosis and remediation. Here, we review research relevant to several components of face processing in persons with autism and healthy controls, and their differential development during infancy and childhood, pointing to gaps in knowledge and potentially fruitful areas for future research.

2. What is Autism?

Autism is a pervasive developmental disorder whose causes or underlying biological mechanisms are not well understood. This behaviorally and developmentally defined syndrome is characterized by impairments in non-verbal communication, social relationships and stereotyped patterns of behavior (DSM-IV, American Psychiatric Association, 1994). Autism is considered a severe form of autism spectrum disorder (ASD), which includes milder forms such as Asperger’s syndrome [18]. Currently, there are no genetic or neurological markers for the majority of individuals with autism, and the diagnosis of ASD is typically determined by expert clinicians using standardized assessments, such as the Autism Diagnostic Observation Schedule [19,20]. Although the cause of autism is unknown, the various forms of ASD are likely to result from combinations of genetic and environmental factors [15-17]. Despite this heterogeneity, problems with social and emotional reciprocity are considered a hallmark of ASD. Preliminary indications may appear as early as the first year of life, including anomalies in mutual gaze, a lack of interest in the human face, and preference for inanimate objects [18,21-24].

In one view, early specific impairments in face processing eventually generalize to a lack of social interest. Supporting this model is the central importance of facial communication between infants and primary caregivers during early development, especially in attributing meaning and emotional significance to human interactions. Such early impairments in face processing may be a “bottom-up” result of anomalies in low-level visual processes, such as motion processing (reviewed in [25]). Alternatively, deficits in face processing may be among the consequences of a “top-down” disinterest in human interaction in ASD. Supporting the latter model are the varied and pervasive nature of ASD social deficits, including non-visual modalities, and the existence of individuals with ASD who process faces within the normal range. However, the absence of detectable deficits could variously result from normal face processing, successful use of alternative strategies or lack of sensitivity of current experimental methods. A detailed examination of normal face processing will set the stage for discriminating between some of these possibilities.

2.1. Viewing behavior and gaze processing

The human face is a focus of visual attention in most healthy individuals starting soon after birth. Newborns (9-min old) typically look more at a schematic drawing of a face than at other visual patterns, including a face outline with reconfigured internal parts [26-28] and also prefer direct gaze over averted or closed eyes [29,30]. The mechanisms underlying these preferences are not well understood. One model suggests that these preferences signify an innate face detecting mechanism that drives attention to faces early during development and may be mediated by subcortical neural systems [31,32]. Alternatively, these early face preferences may reflect the basic properties of the immature visual system at birth that are best stimulated by the physical properties of faces, such as the high contrast between the pupil and sclera [27,33].

In either case, infants’ attention to faces is thought to be an important component of normal development, whereby faces become a frequent and salient visual stimulus early after birth, a likely time for activity-dependent plasticity in the visual system. Indeed, some developmental models of visual cortex hypothesize that habitual patterns of fixation shape the long-term organization of face-processing cortical regions [34]. Consistent with its early onset in infancy, eye contact is also an important means of communication through the life span. Children and adults are proficient in rapidly discriminating the direction of gaze and reflexively orienting towards the corresponding direction [35], although it is not well known if or how proficiency in gaze discrimination may change during the course of development.

2.2. The neural basis of gaze processing: The role of STS

Imaging studies in adults suggest that gaze processing involves a network of brain regions including a cortical region in the posterior superior temporal sulcus (STS). Regions in the posterior STS have been shown to respond to moving and stationary eyes and mouth, but not to moving checkerboards or contracting circles [1]. The STS is more activated when subjects selectively attend to eye gaze than to face identity [1,2,9]. Regions in the STS also respond to a range of visual signals salient for social interaction, such as mutual gaze, emotional expression, speech, intentional limb movements, and biological motion in general [1,2,9,36-39]. Grossman et al. (2005) suggested that normal STS functioning is required for perception of biological motion, as repeated transcranial magnetic stimulation (rTMS) disrupted cortical activity in the posterior STS and reduced perceptual sensitivity to point-light animations in healthy adults [36]. The STS has also been implicated in social cognition and attribution of mental states and intentions [40-44]. In one study, the STS was activated by simple geometric shapes, which had no resemblance to faces or body parts except that their patterns of motion conveyed “intention” [43]. Thus, STS activations have been associated with attribution of intention, even in the absence of eyes, faces or other biological forms. Given that attribution of mental states involves activation of many brain regions (e.g., fusiform gyrus, amygdala, and prefrontal cortex), determining the specific contribution of STS to these tasks will likely require further rTMS.

Few developmental studies of the STS exist. One fMRI study found similar responses in the STS of adults and children (ages 7-10) during processing of averted gaze [45], consistent proficient gaze processing in children. However, a MEG study found gaze modulation of an early magnetic field activity (Plm, around 140 ms) in children ages 8-12, but not adults, that was localized to the inferior temporal sulcus and posterior occipital cortex [46]. Overall, this sparse literature suggests that some maturation of the timing and functional organization of gaze processing may continue throughout childhood, but future studies are needed to examine children’s perceptual sensitivity to fine-grain variations in gaze, and the effect of these variations on responses of STS and other related functional regions. Also unknown are the time course of development of STS, and the extent to which its neural organization and function depend on genetics and/or experience dependent mechanisms.

2.3. Anomalous viewing behavior in autism

Lack of reciprocal eye contact is an early and striking manifestation of autism [47-50]. Retrospective reviews of family home movies suggest signs of atypical social behavior in children who are later diagnosed as autistic [48-50]. These anomalies include poor eye contact, and slow or absent mutual gaze in infants and children, but the underlying mechanisms are not well understood. In addition, recent evidence for anomalous gaze behavior in parents of some autistic children raises the possibility of complex interactions among genes and environmental factors affecting development of eye contact [51].

Little is known about the relationship between early anomalous gaze and long-term face processing deficits in ASD, but there is evidence that other disruptions in early visual experience may lead to long-term deficits in face processing. For example, otherwise healthy infants who are temporarily deprived of high-resolution visual input due to congenital cataract for a period of 45 to 863 days (mean 199 days) show specific deficits in recognition of facial identities 8-29 years later (mean 17 years) [52]. By analogy, it may be that early anomalies in gaze behavior in ASD reduce visual exposure to internal face features, and contribute to long-term face processing deficits. Conversely, less frequent eye contact in ASD may result from difficulties in processing facial information. Longitudinal studies of face-processing development and the role of experience, in ASD and normal children, are needed to evaluate these divergent possibilities. Such studies may determine whether there is indeed a correlation between early gaze behavior (e.g., among offspring and caregivers), and subsequent competence in face processing, as well as emotional and social skills.

2.4. Anomalous gaze processing among children and adults with autism

Anomalous gaze processing remains a hallmark of ASD throughout childhood and into adulthood, although substantial variability is reported. Children with ASD (ages 9-14) are slower at detecting direct gaze relative to controls [53] even though they decode and orient to the direction of averted gaze accurately [53,54]. Consistent with deficits in reciprocal gaze behavior, the results of a recent study showed that adults with ASD typically spend less time looking at the inner features of the face, particularly the eyes, in contrast to healthy or IQ matched controls [55]. In this study, controls viewed faces in a stereotypical pattern, generally tracing a triangle that subtended the eyes, nose and mouth. In contrast, the patterns of face viewing in ASD appeared erratic and less predictable, variably focused on facial features such as the ear, chin or hair-line. In another study of adults, viewing naturalistic social scenes, reduced visual fixation on the eye region and increased fixation on objects predicted the ASD diagnosis [56]. In apparent contradiction, van der Geest et al. (2002) found no differences in how high functioning 10-year-old children with ASD and normal controls viewed upright faces, although group differences in strategy emerged for upside-down faces [57]. Similarly, others have reported that individuals with ASD look with normal frequency toward caregivers [58], or at a person’s face when attention is drawn [59,60]. The divergence among these data may reflect a combination of age or task specificity of gaze anomalies in ASD, and/or a diversity of ASD phenotypes. To resolve this controversy, it will be important to determine if there are any age dependent differences in face viewing strategies among healthy or individuals with ASD. Also important is to determine whether the viewing deficits in ASD are specific to faces as opposed to other objects, similar for familiar and unfamiliar faces, and reproducible within the same subjects (Fig. 2).

Fig. 2.

(from Pelphrey et al. 2002) Patterns of visual scanning of affective faces followed a stereotypical pattern in controls, outlining a triangle between the eyes and mouth. In contrast, scan paths in participants with ASD were more erratic and often excluded the internal features of the face.

2.5. The neural basis of anomalous gaze processing in ASD: A role for STS?

Although the neural basis of aberrant viewing strategies in ASD is not well understood, there is evidence that the STS and the amygdala may be involved in the atypical processing of facial information. For example, an MRI study revealed anatomical displacement of major sulci in frontal and temporal brain regions in children with ASD (mean age 10.7 ± 3.1 std) relative to healthy children (mean age 11.3 ± 2.9 std), including anterior and superior displacements of the superior temporal sulcus bilaterally [61]. A voxel-based morphometry analysis of the brains of children with ASD (mean age 15.4 ± 2.2 std) compared to IQ matched controls (mean age 15.5 ± 1.6 std) found total brain gray matter volume increases and localized increases that included the right superior temporal gyrus [62]. Also using voxel-based morphometry, Boddaert et al. (2004) found contrary results in children with ASD (mean age 9.3 ± 2.2 std), with decreases of grey matter volume localized to the STS compared to healthy controls (mean age 10.8 ± 2.7 std) [63]. This discrepancy between studies may reflect differences in the age range or choice of controls. Alternatively, given the evidence for anatomical displacement of STS position in ASD [61], divergent results from voxel-based morphometry studies may reflect methodological problems arising from image normalization procedures, thus, calling for further analysis with individually defined anatomical boundaries. Also, it will be important to determine the developmental time course when anatomical measures of STS diverge among normals and individuals with ASD.

FMRI of gaze processing suggests abnormal STS responses among individuals with ASD. In an event-related fMRI study, subjects with ASD (mean age 9.3 ± 2.2 std) and controls (mean age 10.8 ± 2.7 std) watched a virtual actor whose averted gaze was either directed towards a checkerboard target (“congruent” condition) or away from the target and towards empty space (“incongruent” condition) [64]. Subjects were instructed to press a button, whenever the “eyes moved”. Both groups were equally (and highly) accurate in detecting changes in eye gaze. However, in normal subjects, the incongruent condition evoked greater STS activation than did the congruent condition, a result that was not seen in subjects with ASD. This lack of modulation in the STS of the ASD group was interpreted to reflect deficits in interpreting other people’s intentions that are normally conveyed by gaze shifts. However, this study did not control for the possibility that the absence of STS modulation may be associated with a more general anomaly in motion processing, which has been found in several behavioral studies (see review [25]). Also, given the evidence for anatomical displacement of STS position among subjects with ASD relative to controls [61], individual ROI analyses of fMRI data would be prudent in future work. Despite these issues, fMRI evidence from other clinical populations support the notion that anomalous gaze processing is associated with aberrant STS activations. For example, individuals diagnosed with fragile X syndrome who share many of the social and emotional phenotypes of autism, including anomalous face and gaze processing, also showed decreased STS activation during a gaze processing task [65]. It remains to be determined if among these clinical populations, STS function is also abnormal during processing of biological and non-biological motion and interpretation of others’ mental state based on non-gaze behavior.

2.6. Development of facial identity recognition in children

Newborn infants at 3-4 days show signs of face recognition [66]. At 6 months infants are able to discriminate facial identities of human and of other species of primates equally well, suggesting a lack of bias for human faces [67]. However, by 9 months infants lose the ability to discriminate among other species faces, suggesting that their face recognition system is more narrowly tuned to human faces [67], unless they are trained on other species’ faces [68]. By age 2 children are able to recognize many human faces.

Despite these early capabilities, face recognition undergoes prolonged development, extending into adolescence[13,14,69-74]. Recognition performance for newly learned faces improves significantly during childhood. Although the details vary among reports, performance ranges from 50% to 70% of the adult level from age 6 to 14, with a dip before puberty and slower gains after age 16 [13,69,71-74]. The underlying mechanisms are not fully understood, but from ages 6-14 years several aspects of face recognition change in such a way that it seems unlikely that age related improvements in face recognition are solely due to maturation of domain general mnemonic processes. The following three examples will inform and motivate our review of literature related to the development of face recognition in ASD and normals.

2.7. Three developmental aspects of identity recognition in healthy children

Three phenomena have been identified in the development of identity recognition in healthy children. First, compared to adults, children typically show less decrement in recognition performance for faces of other races [74, Chance, 1982 #52]; even though this “race effect” may start early during infancy [75,76]. In adults, this race effect is thought to reflect accumulated experience with own race faces compared to other race faces [74,77]. Consistent with this idea, children seem to be more equally proficient at recognizing all faces, albeit below the adult level. This early breadth of face recognition capacity that narrows as face recognition skills improve with age is analogous to children’s wide range of phonemes during early language acquisition, which narrows with increasing skill and specialization in a given language. Our recent findings suggest that the development of this race effect in children is associated with maturation of a face selective cortical region, namely the “fusiform face area” [78]. Second, compared to adults, children between 3 and 11 make more errors in matching faces of the same person after transformations, such as aging, facial expression, view point, or addition of paraphernalia [79,80]. These studies suggest weaker transformation invariance in prepubescent children compared to adults, perhaps in part due to heavier reliance on individual features. Third, unlike adults, children aged 6-8 years old do not show preferential recognition memory for “distinct” faces compared to typical faces, and even at age 13 this aspect of children’s performance is far from adult-like [80]. This finding suggests that typicality is less used or well-defined in children, compared to adults.

2.8. Developmental models of normal facial identity recognition

Results from behavioral studies of face processing in healthy children are consistent with the hypothesis that normal face recognition requires perceptual skills gained by accumulated experience with many faces over years [13,81]. Several models have attempted to explain how visual experience with faces may improve face-recognition skills. These models provide a framework for the normal development of face processing and how a reduced propensity to view faces (as in the case of autism) might lead to atypical development of face processing skills.

Valentine’s influential model of face recognition proposes a multidimensional array for storage of learned faces [81]. In this array, “typical” faces form a cluster (due to their similarities) and “distinct” faces are distributed more sparsely in the periphery. After accumulated experience with faces, fine-grain differences among the “typical” faces can be effciently detected due to the many exemplars, and the large differences between “typical” and “distinct” faces are readily detected. In contrast, it would be diffcult to make fine grain distinctions among atypical faces (e.g., faces belonging to a given other race) due to the availability of fewer exemplars. Extrapolating, others have interpreted children’s lack of bias for same race or distinct faces to mean that young children do not have a well-defined prototype, due to fewer experiences with “typical” faces [80]. The neural substrates for such an experience-dependent “tuning” of face recognition are not well defined. However, several lines of evidence suggest that face selective regions in the occipito-temporal cortex, namely the “fusiform face area” (FFA) may be involved [82-84].

In another model, Diamond and Carey view the development of expertise in face recognition as a transition from an ineffcient piecemeal or “featural” representation to a more effcient processing of the whole face (“holistic” representation) or of the spatial relations among the facial features (“configural” representation) [85,86]. Due to diffculty in relating these concepts precisely to experimental stimuli [87,88] support for this model has been controversial [85,89-93]. Despite these challenges, the notions of featural and configural processing remain attractive concepts in understanding variations in face processing performance among healthy and clinical populations.

More recently, psychophysics studies of adult face recognition suggest that certain constellations of facial features are especially informative for face recognition [94-96] raising the question of whether children are similar in this regard. Other psychophysics studies found that earlier stages of face processing, such as categorization (e.g., face vs. object) are followed by a later stage of face identification (e.g., Bill vs. George) [97-99]. It is not known if maturation of face recognition involves changes in speed of recognition, or correlates with changes in the accuracy or timing of earlier stages of face processing, such as categorization. Although face selective event-related-potentials (namely the N170) have been reported to be slower in children than in adults [100], supporting both of these possibilities, more studies are needed to determine the maturational time course and the relationship between specific stages of face processing during normal development.

2.9. Anomalous identity recognition in ASD

Deficits in recognition memory for newly viewed faces have been reported in children diagnosed with ASD, as young as 2 years old and through adulthood. For example, Klin et al. (1999) found that identity recognition for newly viewed faces was more vulnerable to changes in pose or expression in children with ASD (ages 2-10) compared to controls, matched for chronological age or abilities including short-term visual memory [101]. Similarly, Boucher and Lewis (1992) reported deficits in subsequent recognition of newly viewed faces in children with autism (8 to 17 years old) compared to age matched normals, or learning disabled controls [102]. These results could not be explained by deficits in visual discrimination among faces, as the same subjects with autism accurately matched faces to concurrent samples. Furthermore, children with ASD (ages 3-11) were able to accurately sort, based on identity, pictures of faces that varied across identity and expression [103]. These findings suggest that face recognition memory is impaired in ASD even as visual discrimination among faces may be intact (but see [104]).

A potential mechanism for poor face recognition memory in ASD is anomalous face viewing strategies. For example, in a landmark study by Langdell (1978), children with ASD and controls (matched for mental age and chronological age) were asked to identify faces of their peers that were partially masked [105]. The overall performance of participants diagnosed with ASD and controls were similar. However, between group differences emerged depending on mask placement, suggesting that children with ASD relied more on facial information in the mouth area in contrast to controls who relied more on the eye region. Others have found evidence for unusual reliance on facial features as apposed to configural face information in ASD [106,107]. Further studies are needed to determine the correlation between face viewing strategies and subsequent identity recognition performance among children with ASD and typically developing controls.

Poor face recognition in ASD may not be limited to experimental settings. In one study, recognition memory for faces of frequently present individuals in a school environment was lower among children with ASD (ages 7-11), compared to controls matched for age and verbal ability [108]. Thus, according to Valentine’s model (discussed above), autistic children may encode fewer face exemplars than normal, and develop less “expertise” in face recognition. Note that this period of possible inattention to faces coincides with substantial maturation of the FFA, according to recent fMRI studies [82-84]. Longitudinal studies would be valuable in addressing this possibility.

Evidence for the domain specificity of face recognition deficits in ASD is not uniform. In support, Boucher et al. (1992) found that children with ASD and poor face recognition were similar to normals (matched for verbal ability) for visual memory of their school building relative to controls [108]. Similarly, Blair et al. 2002 found that compared to age and verbal IQ matched controls, a group of adults with ASD showed poor face recognition memory, but normal recognition memory for buildings and leaves [109]. In contrast, a number of other studies have found visual performance deficits in ASD that were generalized to non-face stimuli, including sorting of geometric shapes [110] and memory for non-social stimuli, when compared to controls who are matched for other non-verbal abilities [111,112]. These varied findings may reflect the heterogeneity of ASD or of the choice of controls, non-face objects or tasks. Future studies are needed to test recognition memory in ASD across a range of ability and age, including a variety of non-face stimuli.

2.10. ERP correlates of face processing in normal adults and children

Scalp ERPs and intracranial recordings from otherwise healthy epileptic patients have revealed a reliable and selective index of face processing at 150-200 ms latency, namely the scalp N170 [113-115]. Similarly, the intracranial N200, has been recorded at discrete locations over the fusiform gyrus, the inferior temporal sulcus (ITS) and the superior temporal sulcus (STS) [116]. Possibly marking the category-recognition stage, the N170 is sensitive to features usually predictive of faces, such as isolated eyes, nose or mouth [117,118], or back views of the head, or face outlines without features. In contrast, the N170 appears independent of face identity [113,119,120] as it is insensitive to face repetition [119,121], learning [122], familiarity [123,124], or attention [125]. Although the N170 is not dynamically responsive to face identity or memory, comparison among adult subjects suggests that its amplitude and latency may predict general proficiency at face recognition [114].

In children ages 5-14 years old, the N170 was delayed in onset, longer in duration and less negative compared to adults, and became more adult-like with age [100,117]. Following a similar time course, identity recognition progressively improved into late adolescence before reaching the adult level [73]. Thus face recognition may be a multi-stage process, where the early stage of face categorization is important for later stages of mnemonic processing that are sensitive to face identity [113,120,123].

Consistent with a multi-stage model of face processing, the N170/N200 is followed at 300-700 ms by mnemonically sensitive potentials that rapidly habituate with repetition and are modulated by semantic priming, face-name learning and identification [121]. The N700 is significantly larger for faces than for other stimuli [116], suggesting local generation by face-selective modules in the occipito-temporal cortex. These findings are consistent with identity-sensitive, late, unit responses from face-selective cells in the primate temporal cortex [126].

2.11. ERP evidence for anomalous identity recognition in ASD

Recent studies have found anomalies in face selective ERPs in the occipito-temporal cortex of individuals with ASD. In one study, the N170 was delayed, broadened, and insensitive to face inversion in ASD subjects (ages 15-42) compared to normal controls (ages 16-37), paralleling differences found between healthy children and adults (discussed above). Another study found slower and lower amplitude N170 among adults with Asperger’s syndrome compared to controls, but surprisingly no between group differences among children with Asperger’s and controls (mean age 11.2 ± 1.89; ASD mean age 11.6 ± 1.9) [127]. These results suggest that the anomalous N170 in ASD represents a progressive divergence from normal development, rather than a developmental delay. To test this hypothesis and explore mechanisms, longitudinal studies are needed to determine the time course of development of N170 anomalies in ASD, the location of N170 generators in children with and without ASD, and the relationship of N170 timing and amplitude with instantaneous and habitual patterns of face viewing.

Consistent with the idea that the N170 represents a crucial early stage of face processing, McPartland et al. (2004) found a positive correlation between N170 latency and face identification deficits among individuals with ASD [128]. In a separate study of children with ASD (ages 3-4) compared to controls, Dawson et al. (2002) found atypical modulation of late potentials in response to repeated presentation of a familiar versus an unfamiliar face in children with ASD [129]. Additional ERP and MEG studies specifying the relative developmental time course and generators of the anomalous N170 and the late familiarity-sensitive potentials would help clarify the potential role of mnemonic processing (i.e., storage of many face exemplars) in shaping the relatively early stages of face perception, both in normals and in ASD. One key question is whether the familiarity sensitive potentials are associated with the fusiform face area.

2.12. Neural substrates of identity recognition in healthy adults: The role of FFA

Face recognition involves a distributed network of brain areas, including regions in the medial temporal lobe and prefrontal cortex (see [130,131]) as well as in the fusiform gyrus [132]. Here, we focus on a face selective region in the fusiform gyrus, namely the “fusiform face area” (FFA) [7] which has received particular attention for its role in face recognition in adults for several reasons. First, some patients who have suffered focal injury to the temporal cortex are selectively impaired in face recognition (prosopagnosia), while others are selectively impaired in the recognition of non-face objects (object agnosia) [133-135]. Second, neuroimaging methods have revealed specific face-selective regions in the fusiform gyrus, namely the “FFA” [7]. Third, FFA responses to vaguely face-like stimuli correlate with the subjective experience of face perception [8,136-138], suggesting that the FFA responds to the face gestalt. Fourth, FFA responses are modulated by changes in facial features and configuration [139]. Fifth, the adult FFA is correlated with detection, categorization and identification of faces, but not non-face objects [10,11,136,138]. In contrast, activations of face-selective regions in the STS do not correlate with recognition performance [11]. Sixth, the FFA rapidly habituates to repeated presentation of a face, suggesting that it is involved in fine-grain processing of faces, not just category detection [12,134]. Seventh, the FFA is activated more by face identity than variations in eye gaze or expression [9,12], while the STS showed the opposite effects [9]. Eighth, the FFA’s activity is modulated during working memory, encoding and recognition tasks involving faces [140-144] and predicted subsequent recognition memory for faces [143]. Taken together, these findings support a key role for the FFA in face recognition. However, little is known about the specific visual properties of faces that activate the FFA, what neural interactions within the FFA underlie face processing, and how accumulated experience may shape the FFA’s development and function.

2.13. Competing models of the functional organization of the occipito-temporal cortex

The discovery of category selective regions such as the FFA led to an intense debate regarding the organization of the occipito-temporal cortex. A domain-specific model suggests that the FFA is a cortical module dedicated to processing facial information [145,146]. However, some studies have found that the FFA is activated during viewing non-face objects with which subjects have expertise [147,148], suggesting a process map model in which the FFA is an expert processing region [149,150]. A distributed-representation model questions the modularity of the occipito-temporal cortex and suggests that processing of all visual forms is distributed widely [130,151]. Although none of these models provide a full account of how highly selective functional regions develop, all three are consistent with the hypothesis that lower frequency of face viewing or inattention to faces in autism has long-term effect on the neural substrates of face processing in the ventral stream.

This hypothesis and the neural locus of ASD face-processing abnormalities would be best addressed by experiments that also discriminate between these models. The domain-specific model would predict that any deficits in face detection or identity recognition in autism would be specifically associated with functional abnormalities in the FFA, which would not subserve other expert processing. The process map model would predict that any FFA functional abnormalities in autism would be specific to faces but not other non-face objects of expertise. The distributed-representation model would predict that the functional abnormalities associated with identity recognition would be apparent across a distributed network in the ventral stream, and not confined to the FFA. Thus progress in understanding the neural basis of identity recognition deficits in autism may provide a better understanding of the normal organization of the ventral stream.

2.14. Evidence for maturation of the FFA during normal development

Recent imaging studies have examined face-specific activations in the occipito-temporal cortex in healthy children, finding varied evidence for a maturation process. For example, a PET study found greater responses to faces than to geometric shapes in the ventral-occipito-temporal cortex of two-month old infants [152], while others reported the absence of face selective responses in the fusiform gyrus relative to objects or places (i.e., the FFA) in 5-8 [82] and 8-10 year olds [83] based on group analyses. Thus, it was unclear if the reported reduction or absence of the FFA activation in children under age ten reflects lesser responsiveness to faces or greater responsiveness to objects in the fusiform gyrus, a smaller anatomical size of the fusiform gyrus, or non-specific artifacts such as higher levels of fMRI related noise in children. Furthermore, it was not known if FFA maturation specifically relates to development of face recognition performance or if it is uniquely prolonged compared to other higher-level visual regions specialized for faces (e.g., STS).

To address these questions we examined the maturation of the FFA after age seven, using a standard “localizer” to find face-selective regions within individual subjects [84]. We found striking evidence for selective FFA maturation during this period. First, the right FFA in children ages 7-11 was about half the size of that of older children (12-16 year olds), and about one-third that of adults, even after matching age groups on several measures of fMRI signal-to-noise-ratio. Second, the smaller FFA size in children was strongly correlated with face recognition memory. Third, the developmental changes we found were domain specific. For example, there were no differences among age groups in object recognition, or the size of object-selective cortex or STS.

These findings support an experience dependent model, in which the FFA emerges during a prolonged developmental process involving accumulated experience with faces [153-156]. These findings also raise a number of questions that are current foci of research, namely: does FFA maturation involve increasing face selectivity among broadly face tuned neural elements or increasing numbers of face selective elements? Is a smaller FFA in children associated with differences in face viewing strategies or the specific types of information that children extract from faces? What are the specific functions of the developing FFA in children, especially during the stages of face processing such as categorization and identification?

2.15. FFA and face processing in ASD

Given the apparent role of the FFA in categorization and recognition of faces, several studies have examined the FFA in ASD. Early studies of adults with ASD compared to controls found evidence for lesser FFA activation and greater object-area activation by faces, even though measures of attention to the faces were similar [157-159]. From the process-map perspective, this apparent failure of individuals with ASD to develop normal cortical face specialization in the FFA and “expertise” in face recognition may be the accumulated effect of reduced social interest and lack of motivation to view faces [160,161]. However, other studies variously found FFA activation by faces that was modulated by personal familiarity in ASD [162], or FFA activation that was not distinguishable from controls [163] or FFA activation that was differentially modulated by task among ASD subjects, compared to controls [164].

Using eye-tracking, a recent study suggests that this variety of findings may be explained by differences in fixation behavior [165]. Among persons with ASD but not controls, activation of the fusiform gyrus appears to be strongly and positively correlated with the time spent fixating on the eye region of the face stimuli. Whether the absence of this correlation in controls is due to ceiling effects or between group differences in FFA’s sensitivity to viewing behavior is unknown. In any event, these studies together suggest that the FFA does engage in face processing in ASD and in a manner modulated by fixation upon the eyes (mutual gaze), with implications for the possible developmental consequences of habitual gaze avoidance in ASD. These implications are considered below.

Consistent with the domain-specific model of the ventral stream, and experience dependent models of its development, one possibility is that habitual gaze avoidance (and ensuing chronic hypo-activation of the FFA during development) would result in long-term anomalies in FFA functions, such as face detection and identification, even when subjects do view the eyes in face stimuli. Consistent with the process-map model, a second possibility is that the FFA in ASD subjects would be predicted to activate to a degree correlating with measures of expertise when processing faces and other objects. Consistent with the distributed map model, a third possibility is that functional face-processing abnormalities in autism are not confined to the FFA, but are distributed more widely across the ventral stream. In view of these various model predictions, a detailed analysis of the functional properties of the FFA and the ventral stream in ASD may provide new insights into the role of experience in the normal development and organization of the occipito-temporal cortex.

2.16. Development of facial emotion recognition in children

The recognition of facial expressions of emotion is thought to develop slowly during the first two years of life and continue to mature into adolescence. Some reports indicate that in the first several months of life infants can discriminate between a variety of emotional expressions [166-170]. However, there are diffculties in assessing visual discrimination in young infants. For example, infants’ ability to discriminate among some expressions is sensitive to the order of presentation, and may be affected by differential rates of habituation to specific expressions [166,167,171]. Other data suggest that an infant’s discrimination of facial expressions is based on simple featural differences [172] (but see [173]). For example 4 to 8-month-old infants discriminated “toothy” similes from closed-mouth smiles and closed-mouth anger, but not from toothy anger [172]. Other findings suggest that discrimination of emotional expressions may be sensitive to early contextual information. For example, in one study, 3.5-month-old infants discriminated between happy or sad facial expressions (accompanied by affectively matching vocal expressions) only if the emotional expressions were displayed by their own mother in the experimental settings [174] (also see [175]). Taken together these findings suggest that during the first months of life, processing of facial expressions of emotion is at a rudimentary level, and may be sensitive to familiarity with the facial identity.

Recognition of emotion expressions improves by as much as 40% between ages 2 and 5, to within 10% of adult performance in some studies, albeit non-uniformly for different emotions. Children, ages 4 and 5, recognize facial expressions of happiness, sadness, and anger in order of descending accuracy, and are less accurate in recognizing surprise, fear and neutral expression [176-184]. Also, children’s misjudgments of facial expressions follow systematic patterns. For example, children often confuse faces depicting sadness and anger, or anger and disgust and misjudge neutral faces as sad [178, 179 Reichenbach, 1983 #4630, 182]. In one study of children between 2 and 5 years old, Bullock and Russell (1985) plotted misjudgments of facial expressions on a continuum of facial expressions arranged along two axes of pleasure and arousal, found a narrowing of the distribution of errors around the correct expression with increasing age [178]. These authors inferred a narrowing of emotion categories during development [178], perhaps due to socialization and experience with faces [185].

2.17. The neural substrates for recognition of emotion expressions: Role of amygdala

Although processing of facial expressions of emotion is thought to involve numerous brain regions with varying specificity (reviewed in [186]), much research has focused on the role of amygdala, due to its involvement in emotional learning (e.g., fear conditioning), emotional memory (e.g., “flashbulb” memories), and processing of social information involving various cues including visual inputs of objects and faces (reviewed in [187]). For example, amygdala damage in humans may impair the ability to anthropomorphize moving geometric shapes that normal subjects can interpret as characters with motives interacting in a complex social situation [188]. Amygdala damage is also associated with impairments in social judgment, such as overestimation of trustworthiness and approachability of people based on their faces in recognizing negative expressions, such as fear, anger, surprise, and sadness [189-197], but not recognizing facial identity [190].

Supporting a role for amygdala in face processing and social judgment, early neuroimaging studies found amygdala activation in response to facial expressions of emotion, particularly fear, even when subjects were not instructed to judge emotion [6, 198]. Indeed, amygdala is activated by subliminal presentations of facial expression stimuli [199]. In contrast, verbally labeling facial expression stimuli may reduce amygdala activation [200]. In general, the amygdala is thought to engage in rapid, automatic processing of facial expressions as part of its role in detecting potential danger [189, 194].

The eyes may be a particularly salient stimulus for the amygdala. For example, Kawashima et al. (1999) reported left amygdala activation when subjects interpreted gaze direction, whereas the right amygdala was activated during eye-to-eye contact [201]. In another study, the eye region was suffcient to elicit amygdala responses during fMRI [202]. More specifically, the white sclera surrounding the dark pupil in fearful eyes was a necessary component of the stimulus. Consistent with these findings, a patient with bilateral damage to the amygdala failed to use information from the eye region when viewing faces [191]. Specifically, during exposure to isolated samples (“bubbles”) of faces of fear and happiness this subject used facial information from the mouth, but not the eyes. Eye tracking methods suggested that this inability to use eye information was related to a lack of fixation on the eye region. In contrast, normal subjects spontaneously fixated upon the eyes, preferentially using eye information to recognize fear. When the amygdala damaged subject followed instructions to fixate on eyes, their ability to recognize fearful faces improved to a normal level. These findings suggest that amydala function may be important in directing gaze and attention onto eyes, as a source of social and affective information (Fig 3).

Fig. 3.

(from Adolphs et al. 2005) When exposed to isolated fragments of faces of fear and happiness, normal controls used information mostly from the eye region. In contrast, an amygdala damaged subject (SM) used information mostly form the mouth region.

These studies also raise several questions. For example, is this case study of effects of amygdala damage on fear conditioning relevant to other cases of amygdala damage and dysfunction? Could other social processing deficits after amygdala damage, such as judging trustworthiness or anthropomorphizing moving geometric shapes, arise from a correctable failure to attend to critical information in stimuli? To what extent could training remediate ineffective viewing strategies? How would age of onset of amygdala damage or dysfunction, and its specific character, affect treatment strategies? These questions have significant implications for conditions such as autism that involve face processing deficits and perhaps abnormalities of amygdala function.

2.18. Emotion expressions and amygdala development: Neuroimaging evidence

A few fMRI studies of amygdala function during childhood and adolescence have focused on its responses to fearful facial expressions [203-205]. Baird et al. (1999) examined amygdala activations to fearful faces in 12-17-year-olds and reported no effects of age or sex [203]. In another study, Thomas et al. (2001) compared amygdala activations to fear and neutral faces among children (mean age 11 years old) and adults [205]. In adults, the amygdala responded more to fear faces than to neutral faces, as expected, whereas in children the amygdala responded more to neutral faces. Although subjective ratings of facial expression were not obtained, these results were interpreted to reflect the ambiguity of neutral faces for children, consistent with behavioral studies finding that children classify neutral faces as expressing negative emotions [179, 181]. In a separate study, Killgore et al. (2001) examined developmental changes in neural response to fearful faces in children and adolescents, finding that left amygdala responses to fear faces decreased during adolescence in girls but not boys [204]. Girls also displayed increasing activation of the dorsolateral prefrontal cortex by fear faces during this period, where as boys showed the opposite pattern [204]. These findings suggest that amygdala maturation may be accompanied by age and sex dependent changes outside the amygdala, in neural systems that exert a regulatory influence on its function in adulthood [206]. Future studies are needed to characterize the developmental trajectory of amygdala responses to a wide range of emotionally salient stimuli, including faces. Salient questions include whether the adult pattern of preferential amygdala responses to fear relative to other emotional expressions develops over time during childhood, and whether children and adults use similar viewing strategies in extracting featural information when making emotional or social judgments about faces.

2.19. Impairments of facial expression recognition in ASD: A role for the amygdala?

A number of reports suggest deficits in recognition of facial expressions among persons with ASD, starting as young as 5 years old. For example, when children with ASD and verbal-ability matched controls sorted pictures of people, who differed along dimensions including emotional expression and the type of hat, most children with autism first sorted pictures according to the type of hat, whereas most controls sorted pictures based on emotion expressions before sorting based on hats [207,208]. Another study reported that children with ASD were less able in matching affective faces than objects compared to controls, and the degree of deficit predicted social impairments among the ASD group [209]. These findings suggest that emotion expressions are less salient for children with ASD than for controls and that this deficit may be related to social deficits. However, these results might be sensitive to choice of controls, as another study found that children with ASD showed deficits in sorting emotion expressions compared to controls, when matched for non-verbal mental age but not when matched for verbal mental age [103]. Finally, a study compared high functioning children with ASD (mean age 10.6 ± 2.1) to age and IQ matched controls, and found no differences in naming of expressions (angry, happy, neutral, and surprise) or in foveation during face viewing [57]. Thus, it remains unclear how to interpret reported emotion-expression processing deficits in ASD. Longitudinal studies of emotion-expression recognition and intensity discrimination in children with ASD and well-chosen control groups are needed to address these issues and expand our understanding of normal development.

Nonetheless, the predominance of data suggests that recognition of emotion expressions is impaired in adults with ASD. For example, high functioning ASD adults were reported to rate faces as more trustworthy and approachable than did normal controls [210]. Also, Pelphrey et al. (2002) found that relative to age and gender matched controls, high functioning adult males with ASD made more errors in identifying emotion expressions, being more likely to misidentify fear as anger, surprise or disgust, while identifying happiness, sadness and surprise at control levels [55]. Individuals with ASD also looked less at the eyes. As eye information has been identified as critical for recognition of fear faces [191] and activation of amygdala (in case of fear faces) [202] a difference in fixation behavior may account for the observed fear-specific emotion-expression processing deficit among subjects with ASD. However, it remains unclear whether amygdala hypo-activation might be the cause or a result of the abnormal fixation behavior [165].

Supporting the hypothesis that amygdala dysfunction contributes to face processing abnormalities in ASD are some commonalities between ASD and patients with bilateral amygdala lesions (reviewed in [211,212]), including deficits in fear-face recognition, bias towards rating faces as more trustworthy and approachable [210], atypical patterns of face viewing [55,56] (but see [57]) and failure to attribute social intention to moving geometric shapes [213]. Other lines of evidence implicating amygdala dysfunction in ASD include evidence for amygdala neuropathology from post-mortem studies [214], and autistic-like social and emotional behavior among non-human primates with amygdala lesions early during infancy [215] (but see [216]). Also, structural MRI studies suggest abnormalities in the development of the amygdala in ASD. Initial reports were contradictory, showing evidence for smaller [217], larger [218,219] and equivalent [220] amygdala volumes in subjects with ASD compared to controls. However, more recent data suggest an unusual developmental time course for the amydala in ASD, in which the amydala is larger in children with ASD (ages 7.5 to 12.5 years old) compared to several control groups, whereas no differences are detectable between adolescents with ASD versus controls [221]. It remains to be determined if this anomalous early development leads to persistent compensatory abnormalities in brain regions connected to the amygdala or aberrant intrinsic connectivity and function of the amygdala.

2.20. Neuroimaging of amygdala function in ASD

A number of fMRI studies suggest anomalous amygdala activation in children and adults with ASD. For example, Wang et al. (2004) examined modulation of amydala activation during matching versus labeling of emotional faces in children and adolescents with ASD (ages 8-16), compared to healthy controls (ages 6.9 to 19.8) [222]. Both groups engaged similar neural networks during facial emotion processing. However, in controls but not the ASD group, amygdala activation was higher during emotional face matching relative to labeling [222]. Also, a case study of a 12-year-old boy diagnosed with ASD suggests that amygdala activation in ASD may be sensitive to expertise, as amygdala hypo-activation was found when viewing pictures of realistic faces but normal levels of activation were found when viewing a familiar and well liked cartoon character [161].

FMRI studies in adults have found abnormally low activation in the amygdala among persons with ASD relative to controls for tasks involving emotion recognition from either the eye region alone [223] or the face as a whole [224]. In contrast, another fMRI study reported that levels of left amygdala activation in subjects with ASD were greater than in controls and positively correlated with the average time of fixation upon the eye region, during both emotion and familiarity judgments on faces [165]. Dalton et al. (2005) interpreted these results to suggest that viewing faces, particularly eyes, may be associated with heightened arousal among persons with ASD, and that amygdala hypo-activation may be a result of viewing strategies in this group [165]. Future studies are needed to test this hypothesis. Taken together, these findings emphasize the importance of controlling for viewing strategies and arousal levels when studying amygdala function in face processing.

3. Conclusions

Autism is a pervasive developmental condition, characterized by impairments in non-verbal communication, social relationships and stereotypical patterns of behavior. Currently, the underlying causes of autism are not known, although they are likely to be quite heterogenous and involve combinations of genetic and environmental factors. In the search for neural markers of autism, much interest has focused on a network of brain regions that are implicated in social cognition and face processing. In this review, we focused on three such regions, namely the STS for its role in processing gaze and facial movements, the FFA in face detection and identification and the amygdala in processing facial expressions of emotion. A large body of evidence suggests that aspects of face processing associated with these regions are impaired in autism, including anomalies in gaze processing, memory for facial identity and recognition of emotion expressions. There is also fMRI evidence for abnormal activation in brain regions underlying each of these neurocognitive functions in autism. Recent data suggest that some of these functional anomalies may reflect atypical face viewing strategies, as many individuals with autism do not fixate normally upon the eyes, starting early in life. These findings raise new questions regarding the development of anomalous face processing in autism. For example, could atypical gaze behavior starting early in life adversely affect the development of all or specific components of the face processing network? Recent findings on the normal development of face processing during infancy, childhood and adolescence suggest that while some aspects of face processing such as automatic orienting towards faces may exist shortly after birth, others, such as identity and emotion processing undergo prolonged development. Open questions in this research area are the brain basis of these developmental processes, the role of experience in shaping each of the specialized regions for face processing, and the impact of each region’s function upon the development of other components of the face processing network. Progress in understanding the normal development of face processing will be essential for a better understanding of face processing anomalies in autism, and the potential timing and effect of early remediation focused on faces. In this context, longitudinal studies of components of face processing using combinations of behavioral and imaging methods during typical and atypical development will be especially valuable.