Face Perception and Learning in Autism Spectrum Disorders

Abstract

Autism Spectrum Disorder (ASD) is characterized by impairment in social communication and restricted and repetitive interests (American Psychiatric Association, 2013). While not included in the diagnostic characterization, aspects of face processing and learning have shown disruptions at all stages of development in ASD, although the exact nature and extent of the impairment varies by age and level of functioning of the ASD sample as well as by task demands. In this review, we examine the nature of face attention, perception, and learning in individuals with ASD focusing on 3 broad age ranges (early development, middle childhood, and adolescence/adulthood). We propose that early delays in basic face processing contribute to the atypical trajectory of social communicative skills in individuals with ASD and contribute to poor social learning throughout development. Face learning is a life-long necessity, as the social world of individual only broadens with age, and thus addressing both the source of the impairment in ASD as well as the trajectory of ability throughout the lifespan, through targeted treatments, may serve to positively impact the lives of individuals who struggle with social understanding and information.

Introduction

Autism Spectrum Disorder (ASD) is characterized by impairment in social communication and restricted and repetitive interests (American Psychiatric Association, 2013). Within the domain of social communication, many core impairments involve perception, learning, and behavioral modification based on information obtained from the face. Early diagnostic risk signs for ASD include decreased use of facial information, including failure to look at the faces of social partners and failure to use eye gaze for joint attention. Aspects of face processing and learning have shown disruptions at all stages of development in ASD, although the exact nature and extent of the impairment varies by age and level of functioning of the ASD sample as well as by task demands.

From a normative development perspective, faces are one of the most ubiquitous stimuli in a young child’s environment. Not only are infants primed to respond to faces more than other stimuli shortly after birth (Morton & Johnson, 1991a), but they also imitate facial expressions from very early in life (Meltzoff & Moore, 1977). Nelson has proposed that the face processing system develops from a “broadly tuned, non-specific, complex figure recognition system” into one tuned for the faces seen most often in the infant’s natural environment (Nelson, 2001). In this framework, the face processing system is dependent on perceptual experience (de Haan, Johnson, & Halit, 2007; Pascalis, de Haan, & Nelson, 2002).

In this review, we examine the nature of face attention, perception, and learning in individuals with ASD focusing on 3 broad age ranges (early development, middle childhood, and adolescence/adulthood). We propose that early delays in basic face processing contribute to an atypical trajectory of social communicative skills in young children with ASD, and contribute to poor social learning throughout development. An early delay in face processing, as seen in young children with ASD, results in a cascade of altered wiring of the social brain. While plasticity in the system supporting perceptual experience allows for “catch-up growth” and maturation of basic face processing, these later accumulated experiences cannot re-wire the early atypical social brain infrastructure formed during this critical period of early development.

Early Development

Face Attention

ASD-related differences in visual attention are apparent from very early in life. For example, looking at people (and responding to name) best distinguished infants with autism from infants with developmental delay without autism at 9 to 12 months of age (e.g., Baranek, 1999; Osterling et al., 2002). Similarly, reduced social interaction, absence of social smiling, and lack of facial expression have been noted in retrospective videotape studies of early autism (Adrien et al., 1992). In prospective studies, which often compare “high risk” infant siblings of children with ASD to “low risk” infants, Ozonoff and colleagues also found that from 12 to 18 months of age, high risk infants who were later diagnosed with autism exhibited fewer looks to an experimenter’s face, as well as decreases in social smiling, social engagement, joint attention, orienting to their name, and requesting behaviors (Ozonoff et al., 2010; Rozga et al., 2011). Thus, reduced orienting and attention to others’ faces characterizes the pre-diagnostic period.

Lab based experimental tasks present a more complex picture of early facial attention in ASD. For example, 6-month-old infants who later develop ASD show reduced visual attention to the inner facial features of a face but only when the faces are speaking (Shic, Macari, & Chawarska, 2014) and reduced attention to an actress in a naturalistic video scene (Chawarska, Macari, & Shic, 2013). Others found gradual reductions in attention to the eyes of a naturalistic ‘caregiver’ video between 2 and 12 months (Jones & Klin, 2013) and to faces during a live observational assessment between 6 and 12 months (Turner-Brown, Baranek, Reznick, Watson, & Crais, 2013). When using static displays, 7- and 14-month-old infants who later developed ASD showed typical patterns of orienting to faces, more time looking at a face than controls and typical modulation of facial attention in complex displays (Elsabbagh et al., 2013). In 20-month-old toddlers with ASD, facial attention during dynamic child-directed speech scenes was also significantly reduced (Campbell, Shic, Macari, & Chawarska, 2014). Moreover, variability within the ASD group was predictive of short term development: toddlers who showed limited attention to the scene (approximately 1/3^rd of the sample) had markedly slower development from 2 to 3 years of age, whereas the remaining 2/3^rd of the toddlers, who showed good attention to the scene, had more positive communication trajectories. Among those with good attention, children who attended to the mouth made greater language progress (Campbell et al., 2014). Thus, the developing ability to use facial information in the context of joint attention and speech has specific contributions to later social communication (Chawarska, Macari, & Shic, 2012).

Concurrent with face-related differences in attention, object attention may also be altered in early ASD. At 12 months, infants who later develop ASD show increased interest to and exploration of objects (Ozonoff et al., 2008; Zwaigenbaum et al., 2005). By 2 years, toddlers with ASD show increased focus on objects (e.g., toys) compared to people (Chawarska & Volkmar, 2006). Toddlers with ASD not only look more to physical objects in natural social scenes but increase their fixation on those objects when the objects move (Shultz, Klin, & Jones, 2011).

Face Perception

Much of what is known about basic early face perception has been derived from work using event related potentials (ERPs). The N290 (or “infant N170”) is a posterior-temporal peaked, negative component with latency between 290 and 350 ms visible in evoked potentials in 3- to 12- month old infants (Halit, de Haan, & Johnson, 2003), with emerging specialization and sensitivity for face information (e.g., de Haan, Johnson, & Halit, 2003; Halit, Csibra, Volein, & Johnson, 2004). A later (lateral) posterior-temporal component, the P400, shows a facial inversion effect by 12 months of age (Halit et al., 2003).

In the first year of life, infants who are later diagnosed with ASD demonstrate “normative” P1 and N290 responses to faces, suggesting potentially normative development of sensitivity to facial information (Elsabbagh et al., 2009; 2012). Similarly, high risk infants who develop ASD do not show delays in the N290 or P400 response to repeated pictures of unfamiliar faces at 6 to 9 months (Luyster, Wagner, Vogel-Farley, Tager-Flusberg, & Nelson, 2011). In contrast, 6- to 10-month-old infants who develop ASD show reduced sensitivity to gaze shifts). By the second year of life (18 to 30 months), children with ASD show delayed developmental N290s to faces relative to age-matched children, and are more similar in speed to children matched on social mental age (Webb et al., 2011). There is little improvement in speed of the N290 from 18 to 30 months to 30–42 months, suggesting an atypical or stalled developmental process (Webb et al., 2011). Further, this delay may be specific to the left hemisphere processing of faces (e.g., Webb, Dawson, Bernier, & Panagiotides, 2006).

Similar to results from studies of visual attention, early autism may include object processing biases. At 12 months of age, high risk infants show significantly faster early neural responses to object stimuli than low risk infants, particularly over the right hemisphere (Jones et al., 2015; also McCleery, Akshoomoff, Dobkins, & Carver, 2009). Other reports suggest increased frontal alpha power to objects compared to faces in 12-month-old infants who develop ASD (Barnes et al., 2015) and a temporal benefit for objects compared to faces (Webb, Long, & Nelson, 2005). A general increase in interest or expertise for object stimuli when compared to faces may be an early-emerging feature of risk for ASD.

Face Learning and Memory

In habituation or familiarization paradigms, a repeated stimulus is presented until visual attention wanes to a predefined level; the looking duration during this learning phase is proposed as a stable measure of individual differences in infancy (Colombo, 1997; Reynolds, Zhang, & Guy, 2012) and is considered to reflect information processing efficiency and sustained attention to a stimulus (e.g., Reynolds & Guy, 2012; Shaddy & Colombo, 2004). In low risk and typically developing infants, attention to faces during habituation is often longer than to objects with significant decreases in time to habituation over the first year of life (Jones, Pascalis, Eacott, & Herbert, 2011; Jones et al., 2015; Reynolds et al., 2012; Robledo, Kolling, & Deák, 2010). In contrast, 6-month-old high risk infants who later met criteria for ASD (compared to high risk infants without early ASD) showed a significantly shorter peak look that was later in the habituation function, suggestive of disruptions to sensitization and deeper levels of processing (see Jones et al., 2015). Similarly, Chawarska et al. (in press) also found that more attention to a speaker’s face at 6 months was associated with lower autism symptoms at 24 months. In toddlers with ASD (18 to 30 months), total habituation time was related to ASD severity, with toddlers with severe ASD demonstrating significantly longer times to habituate to faces than comparison groups (low severity ASD toddlers, unaffected siblings, controls), suggesting a specific slowing in information processing of faces in the second year of life (Webb, Jones, Merkle, Namkung, et al., 2010b). In both reports, habituation to object control conditions did not differ by group. While these findings may seem contradictory, it may be that abnormal shallow processing during the first year leads to slowed information processing later in development, although this needs to be directly tested in longitudinal studies.

After habituation, memory is assessed via pairing the learned stimulus with a novel exemplar and assessing allocation of attention between the two. During infancy, a novelty preference is thought to represent discrimination (Colombo & Mitchell, 2009), although several proposals suggest that this may be an imperfect representation of memory (Pascalis, de Haan, Nelson, & de Schonen, 1998). In 6- and 12-month-old high risk infants who develop ASD, dishabituation or novelty preference is similar to low risk controls, despite the earlier differences in habituation time (Jones et al., 2015). Similarly, in toddlers with ASD, novelty preference did not differ between groups (Webb et al., 2010b). In contrast, in 3- to 4-year-olds with ASD, Bradshaw and colleagues found a specific memory impairment for faces--similar encoding times during a familiarization phase but no novelty preference during a paired comparison procedure for faces despite above chance performance for block patterns and objects (Bradshaw, Shic, & Chawarska, 2011).

Comparisons of neural processes to familiar vs. unfamiliar stimuli using EEG suggests that early memory for faces may be delayed but not until after infancy. Luyster and colleagues did not find any differences between infants at high and low risk for ASD between 6 and 36 months when comparing familiar vs unfamiliar face processing (Luyster et al., 2014). In contrast, Webb and colleagues found that differential processing of familiar vs unfamiliar faces in toddlers with ASD was more similar to social-ability matched controls (using age equivalents derived from the Vineland Adaptive Behavior Scale Socialization domain) than to a chronological-age matched group (Webb et al., 2011), suggesting that a lack of differential processing in 3–4 year olds with ASD (Dawson et al., 2002) may reflect a transient or intermediary shift in this response that is similar to that found in younger children (Carver et al., 2003; Webb et al., 2011).

Face memory difficulties may also be a “broader phenotype” of ASD. The BASIS team found that high risk children (including children with neurotypical outcomes, other developmental concerns, and ASD) who had an older sibling with ASD showed impaired face memory when the to be remembered face differed in facial expression from learning to test but not when the face was identical at the two phases (de Klerk, Gliga, Charman, Johnson, & BASIS Team, 2014) suggesting an impairment in abstraction of configural or gestalt information from the face. Of note, greater attention (i.e., longer looking) toward a face at 7 months of age in a face pop out task was associated with poorer recognition at 3 years of age, but only in the high risk group.

When faces are incorporated into other attention and learning tasks, children with ASD show specific impairments on face or social versions of the task compared to non-social versions. Toddlers with ASD are faster at initiating a saccade to a peripheral target if the central cue is a face compared to a non-face stimulus (Chawarska, Volkmar, & Klin, 2010), suggesting that children with ASD (compared to controls) have an easier time disengaging from facial information. In line with other findings of enhanced object processing, improved working memory for nonsocial stimuli is found in 9-month-old high risk infants (Noland, Reznick, Stone, Walden, & Sheridan, 2010).

Relation to Social Ability

Piecing together the pattern of findings, high risk infants who go on to develop ASD may show increased response to faces early in the first year of life, with a significant slowing of processing or decreased attention to faces emerging in the second half of the first year to second year of life dependent on the task requirements and contrasts. Thus, as Klin emphasizes, it is the development of facial perception and attention over the first year of life that is atypical, with the severity of the trajectory associated with more severe autism at the time of diagnosis (i.e., 2–3 years; Jones & Klin, 2013). In young children diagnosed with ASD, slowed face learning was related to greater autism severity, lower verbal ability, and slowed object learning speed (Webb et al., 2010b). Further, slowed neural speed during early childhood is correlated with poorer joint attention and emotion attention (Dawson, Webb, Carver, Panagiotides, & McPartland, 2004), suggesting that face systems are related to several levels of social ability. In addition, duration of unusual visual exploration of objects (e.g., prolonged visual inspection, examining object from odd angles or peripheral vision) has also been found to be related to poorer outcomes, with more unusual object exploration at 12 months related to worse autism severity and lower cognitive and language outcomes (Ozonoff et al., 2008).

Summary

Taken together, metrics of facial processing and learning in infants and toddlers are significantly influenced by the context of the measurement with ASD-related disruptions emerging in the second half of the first year of life and becoming more delayed or atypical during toddlerhood. For faces, perceptual development is proposed to occur based on visual experience such that the atypical early attention in infants who later develop ASD would alter the maturation trajectory of the systems that support face processing efficiency and fluency. Active experience is critical to learning and children who are developing ASD may have subtly different visual environments early in development. As proposed by Bruckner and Yoder, increased attention to and restricted play with objects may further lead to a reduction in attention toward people (Bruckner & Yoder, 2007). Thus, the circuitry supporting face processing develops concurrent with the social ability of the child – suggesting a dynamic interplay between the child’s increasing social atypicalities with altered attention focus and reduced social attention leading to the construction of a social brain that is built upon a different set of information. Over time, this would disrupt the integration of facial information with other forms of information needed for complex social processing.

Childhood

Although the typically-developing face processing system is tuned early to preferentially process faces, it also improves markedly over the course of childhood. Much of this change occurs prior to puberty, with both behavioral and neural indices of face processing becoming more similar to those of adults. Children with ASD, in contrast, display an altered trajectory across many measures and tasks.

Face Attention

During middle childhood, faces continue to represent highly salient visual stimuli. As in infancy, typically developing children direct visual attention toward faces more frequently than non-social objects; relative to objects, faces elicit more frequent fixations, quicker detection of changes within visual scenes), and delayed disengagement of attention (Kikuchi et al., 2011; Kikuchi, Senju, Tojo, Osanai, & Hasegawa, 2009; Snow et al., 2011). Within faces, neurotypical children and adolescents bias attention toward the eyes, and preferentially use others’ eye gaze over nonsocial information when directing their own attention (Senju, 2004). Direct eye gaze appears to carry particular weight; children are faster and more accurate to detect it versus averted eye gaze, and they exhibit differential neural responses to direct relative to averted gaze (Kimura, Kubota, Hirose, Yumoto, & Sakakihara, 2004; Kylliäinen et al., 2012; Senju, Yaguchi, Tojo, & Hasegawa, 2003). As a whole, typically developing children appear to treat faces as a special class of visual stimuli, prioritizing them above other images and actively relying upon them for social cues.

Children with ASD, in contrast, show reduced attention to faces and increased attention to non-face images within dynamic social scenes (Rice, Moriuchi, Jones, & Klin, 2012). When attention is directed toward the face, relatively less time is spent viewing the eyes, with possible overemphasis on the mouth (Papagiannopoulou, Chitty, Hermens, Hickie, & Lagopoulos, 2014). Consistent with this, children with ASD have more difficulty detecting direct eye gaze from others and appear not to prioritize information obtained from the eyes (e.g., direction of gaze) over non-social information (e.g., direction of an arrow) during cognitive tasks (Senju, 2004; Senju et al., 2003). Because of their reduced attention to faces, and to eyes in particular, children and adolescents with ASD receive fewer opportunities to engage in eye contact, joint attention, and contingent, reciprocal social interaction with peers and adults.

Face Perception

Middle childhood is typically characterized by notable changes in face perception, although the nature of those changes has been debated. Early research suggested that young typically-developing children were unable to engage in configural processing of faces and thus relied solely upon feature-based strategies for face recognition, with a qualitative switch to a configural approach later in development (Diamond & Carey, 1977). More recent work indicates that infants and young children do possess some degree of configural or holistic processing ability from early in life (Bhatt, Bertin, Hayden, & Reed, 2005; Mondloch, Le Grand, & Maurer, 2002; Pellicano & Rhodes, 2003), but also that performance on configural and holistic tasks increases with age (Neuhaus, Kresse, Faja, Bernier, & Webb, 2015). These findings suggest the possibility of a gradual, quantitative shift away from featural processing as the dominant approach, and toward increased reliance on configural or holistic strategies over the course of middle childhood (Kuefner, de Heering, Jacques, Palmero-Soler, & Rossion, 2010; Taylor, Batty, & Itier, 2004).

Changes in processing strategies during middle childhood are paralleled by changes in electrophysiological indices. Similar to the N170 in adults, children display a negative ERP component that is greatest at posterior temporal electrodes and larger in amplitude for faces than non-face stimuli (Taylor, Edmonds, McCarthy, & Allison, 2001). This precursor N170 (prN170) is sensitive to face region and to image orientation, as it is largest in response to eyes, followed by upright faces (Taylor et al., 2001). In early childhood, the latency of the prN170 is significantly slower than adults, but becomes markedly faster from 4 to 5 years of age (~270 ms to faces), 8 to 9 years of age (~220 ms), and 14 to 15 year olds (~170 ms). Right-hemisphere responses appear to increase in amplitude over this developmental window, potentially reflecting increased processing relative to the left hemisphere that mirrors the lateralization of face processing observed in adults (Bentin, Allison, Puce, Perez, & McCarthy, 1996; Taylor et al., 2001). Developmental trends have also been observed in the P1 component, a positive deflection approximately 100 ms post-stimulus indexing early visual attention. P1 decreases in both latency and amplitude during middle childhood and adolescence, suggesting improved processing efficiency with age (Hileman, Henderson, Mundy, Newell, & Jaime, 2011).

In contrast to this trajectory, children with ASD show a prolonged reliance on featural strategies. Relative to peers, they show a reduced inversion effect, and are less accurate on holistic face processing tasks (Hobson, Ouston, & Lee, 1988; Neuhaus et al., 2015; Rose et al., 2007; van der Geest, Kemner, Verbaten, & van Engeland, 2002). The developmental course of the P1 and prN170 components in ASD are not thoroughly characterized, but evidence suggests longer latencies for P1 responses (suggesting slower processing), reduced inversion effects on P1, and longer latencies for prN170 relative to peers (Hileman et al., 2011; Neuhaus et al., 2015). Within ASD samples, sex differences may also be present, with girls showing more atypical prN170 responses to faces (Coffman, Anderson, Naples, & McPartland, 2015). Whether group differences in processing reflect difficulty engaging in configural processing or a default processing style favoring featural strategies is a matter of debate (Jemel, Mottron, & Dawson, 2006) as is the degree to which featural processing characterizes visual-spatial processing in ASD across domains (Campatelli, Federico, Apicella, Sicca, & Muratori, 2013). Nonetheless, ASD is associated with altered trajectories in both behavioral and electrophysiological markers of face perception throughout middle childhood.

Face Learning & Memory

Concurrent with the gradual shift toward configural processing, typically developing children become increasingly accurate and efficient at recognizing faces. Between 6 years of age and adolescence, recognition accuracy improves continuously before slowing during adolescence as it approaches adult proficiency (for review: Chung & Thomson, 1995; Golarai, Grill-Spector, & Reiss, 2006). Even as overall accuracy improves, however, children‘s memory for faces remains vulnerable to visual changes in target faces and is easily disrupted by transformations such as aging, addition or removal of accessories, and changes in viewing angle (Diamond & Carey, 1977; Mondloch, Geldart, Maurer, & Le Grand, 2003). Such disruption implies some continued reliance on featural processing (Golarai et al., 2006), consistent with encoding findings described earlier.

Behavioral reports suggest that by middle childhood individuals with ASD perform worse than mental age and chronological age matched peers on face memory via recognition tasks (Boucher & Lewis, 1992; Boucher, Lewis, & Collis, 1998; Klin et al., 1999; Neuhaus et al., 2015). Deficits are most often noted in the presence of memory demands (Weigelt, Koldewyn, & Kanwisher, 2012), and evidence of intact memory for non-face images such as patterns, buildings, and electric fans implies a face-specific deficit rather than a domain-general impairment (Boucher & Lewis, 1992; McPartland, Webb, Keehn, & Dawson, 2011; Snow et al., 2011; Weigelt, Koldewyn, & Kanwisher, 2012). Further disruption is found in tasks utilizing face discrimination (Deruelle, Rondan, Gepner, & Tardif, 2004; Gepner, de Gelder, & de Schonen, 2007; Tantam, Monaghan, Nicholson, & Stirling, 1989) and lip reading, gender and gaze discrimination (Deruelle et al., 2004). Similarly, faces may disrupt rule learning for children with ASD--when learning the non-match to sample rule, 9-year-old children with ASD showed disrupted rule learning when the sample and test items were pictures of faces but not when they were pictures of objects (Jones, Webb, Estes, & Dawson, 2013).

Faces, as stimuli, also play an important role in the exploration of constructs such as social reward. Various forms of social stimuli (e.g., human faces, human voices) are often construed as carrying reward value – that is, activating neural regions important to reward processing and thus positively reinforcing social engagement over the course of development (Dawson et al., 2007). Though difficult to test empirically, several studies to date have used images of faces as signifiers of social reward (e.g., Delmonte et al., 2012; Kohls et al., 2011; Scott-Van Zeeland, Dapretto, Ghahremani, Poldrack, & Bookheimer, 2010). Neural response (and thus reward value) for faces appears to be diminished for children with ASD relative to their peers (Delmonte et al., 2012; Scott-Van Zeeland et al., 2010; Stavropoulos & Carver, 2014). However, findings are not entirely consistent, as documented by an alternative approach comparing level of effort (rate of key presses) exhibited to view images of faces versus non-face objects (Ewing, Pellicano, & Rhodes, 2013). In this sample, both typically developing children and those with ASD put forth greater effort to view cars relative to faces, with more attractive faces eliciting greater effort in both groups. Such findings demonstrate both the potential and the complexity of understanding how faces are integrated into motivational and reward mechanisms.

Relation to Social Ability

Consistent with its proposed role in promoting the development of more sophisticated and complex social cognition and behavior, stronger face processing is linked with stronger social skills. Among children with and without ASD, better accuracy in tasks of holistic processing and face memory are associated with stronger social skills and fewer social difficulties (McPartland et al., 2011; Neuhaus et al., 2015) as well as more cooperative social play (Corbett, Newsom, Key, Qualls, & Edmiston, 2014). Latency and amplitude of neural response to upright faces also predicts social functioning, with stronger social skills among individuals with larger P1, more negative prN170, and faster prN170 responses (Hileman et al., 2011; Neuhaus et al., 2015). Furthermore, emerging evidence suggests that differential prN170 amplitude to faces relative to houses corresponds to ASD symptom severity in adolescent girls with ASD, with less affected girls showing greater differentiation between faces and houses (Coffman et al., 2015). Thus, although longitudinal explorations of links between face processing and social behavior are lacking, cross-sectional evidence supports theoretical models placing face processing at the core of successful social cognition and behavior.

Like many systems in the brain, mechanisms involved in face processing are moderated by a variety of psychosocial and environmental factors. Child characteristics such as temperament and psychiatric symptoms also moderate face processing during this period. Children high in anxiety, for instance, differ from their peers in patterns of visual orienting, affect recognition, and brain response to emotional faces (Gamble & Rapee, 2009; Simonian, Beidel, Turner, Berkes, & Long, 2001; Thomas et al., 2001). Environmental influences such as maltreatment also affect face processing during middle childhood, with effects most marked in the domain of affect recognition and processing (da Silva Ferreira, Crippa, & de Lima Osório, 2014; Pollak, Klorman, Thatcher, & Cicchetti, 2001).

Summary

Middle childhood, then, reflects a period of increasing divergence between children with ASD and those with typical development with regard to face processing. Whereas faces represent a special type of visual stimulus for those without ASD – attracting attention, providing a rich source of social information – they are not similarly prioritized or processed by children with ASD. Behavioral and neural indices suggest differential processing approaches, speeds, and efficiencies across children with and without ASD, and correlations between these indices and social skills and difficulties underscore the continued importance of face processing for meaningful social outcomes across this developmental period.

Adolescence & Adulthood

With its foundation in place, adolescence is a time of continued refinement of the face processing system. Social demands shift, peer interactions, instead of family, become primary, and the emotions experienced during social interactions are intensified. For individuals with ASD, differences in brain activation, perceptual processes and memory patterns persist.

Attention

Eye tracking investigations of attention patterns to faces by adolescents and adults with ASD generally reveal differences in the pattern of gaze scanning of faces, although superficial attention to faces in non-demanding experiments may be similar to controls (Fletcher-Watson, Leekam, Benson, Frank, & Findlay, 2009; Hernandez et al., 2009; Pelphrey et al., 2002; Sterling et al., 2008). Recent work comparing individuals with ASD and intellectual disability (ID) to adults with ID only found that the adults with ASD had shorter looking durations to the eyes and more fixations on the nose; when compared to typical adults, adults with ASD had a different scanning pattern between areas of the face (Yi et al., 2013). In conditions involving real-time shared or joint attention, adults with ASD demonstrate different neural activation relative to age and sex-matched comparison subjects (Redcay et al., 2012).

Different attention patterns to faces are also observed when adolescents and adults with ASD must integrate basic information about the face with other information streams. For example, adolescents and adults with ASD exhibited different initial gaze patterns compared to controls when viewing complex scenes with people that varied in their emotional valence (Santos et al., 2012). Though visual attention to isolated faces with different expressions did not differ for adolescents with ASD, ERPs differed and, unlike typical comparison youth, adolescents with ASD did not demonstrate a correspondence between faster ERPs to faces and visual attention to the eyes during an eye-tracking task (Wagner, Hirsch, Vogel-Farley, Redcay, & Nelson, 2013). As well, whereas typical comparison participants integrate categories of auditory and visual attention during both easy and hard selective attention tasks, adults with ASD only show integration during the easy selective attention tasks (Magnée, de Gelder, van Engeland, & Kemner, 2011). Taken together, these investigations suggest that adolescents and adults demonstrate increasingly aberrant attention patterns to faces when the task demands increase and become more similar to those experienced in real life. That is, attention to static faces is often similar to controls, but attention to faces that are moving, forming emotional expressions, talking, and embedded in a visually complex and noisy environment show disrupted attention patterns.

Face Perception

As individuals with ASD move toward adulthood, differences in face perception also persist. At the most basic level, adults with ASD do not categorize the gender of faces with the same expertise that typically developing children and adults do, reflecting atypical prototype formation (Strauss et al., 2011). Many adults with ASD demonstrate a processing deficit for faces relative to objects. While not universal, this deficit appears to be due to impairments in encoding second-order configural information and holistic processing (Faja, Webb, Merkle, Aylward, & Dawson, 2009; Wallace, Coleman, & Bailey, 2008). The processing advantage of viewing a face as a singular entity, known as holistic processing, is reduced in adolescents and young adults with ASD, who are less disrupted by aligned composite faces than age matched comparison subjects (Gauthier, Klaiman, & Schultz, 2009; Teunisse & de Gelder, 2003) and demonstrate less robust advantages for recognizing parts of faces within the context of the face over viewing the part in isolation (Faja et al., 2009; López, Donnelly, Hadwin, & Leekam, 2010). Of note, cueing attention to key areas of the face can lead to holistic processing advantages (López et al., 2010). Adolescents and adults with ASD are able to detect some higher order configural relations between face features, but do so less consistently and efficiently (e.g., Faja et al., 2009; Rutherford, Clements, & Sekuler, 2007). Adults with ASD are less likely to form face prototypes based on subtle configural information about distances between features relative to age matched controls (Gastgeb, Rump, Best, Minshew, & Strauss, 2009).

Behaviorally, for the face inversion effect, which is an indicator of configural processing, adolescents and adults with ASD demonstrate inconsistent levels of disruption relative to unaffected subjects (Faja et al., 2009; Lahaie et al., 2006; Rutherford et al., 2007; Teunisse & de Gelder, 2003) consistent with the idea that some basic configural processing biases are in place but not as automatically available or readily used in individuals with ASD. As well, Pallet and colleagues found that adolescents demonstrated a different pattern of face versus object discrimination when stimuli were matched on visual properties. Specifically, youth with ASD had slightly reduced discrimination sensitivity to faces, but significantly enhanced discrimination of objects. During the age range sampled (13–18 years), teens with ASD also developed an inversion effect later. Higher IQ corresponded with better face discrimination for adolescents with ASD; for controls, better face discrimination was related to age (Pallett, Cohen, & Dobkins, 2014).

In addition, adults with ASD do not exhibit differences in gaze detection for upright versus inverted faces, whereas typically developing adults had a narrowing in their perception of gaze for upright but not inverted faces (Vida et al., 2013), suggesting that perception of gaze is less precisely tuned for adults with ASD. Similarly, despite comparable advantages for conscious perception of direct gaze and unconscious object detection by adolescents with ASD and a comparison group without ASD, teens with ASD fail to detect direct gaze more rapidly than averted gaze (Akechi et al., 2014). In sum, adolescents and adults with ASD may exhibit the perceptual biases associated with faces, but to a lesser degree and with less consistency.

Brain activation in a network supporting face processing (e.g., fusiform face area, superior temporal sulcus, and occipital face area) is reduced relative to that of unaffected comparison subjects, while networks linked to object and place processing are relatively intact (e.g., Humphreys, Hasson, Avidan, Minshew, & Behrmann, 2008; Schultz et al., 2000). Adults with ASD fail to reach the activation levels of the healthy comparison group in a network associated with face detection and rapid processing of emotions that included subcortical brain regions (bilateral fusiform gyrus, left amygdala, right pulvinar, and bilateral superior colliculi; Kleinhans et al., 2011). In the same sample, the pattern of relative amplitude differences and the latencies for faces versus objects at components linked to attention (the P1) and discrimination of faces from other stimuli (the N170) did not differ for adults with ASD and the unaffected comparison group (Webb et al., 2012); nor did components responsive to facial identify (N250: Webb, Jones, Merkle, Murias, et al., 2010a). The authors propose that this may be due to the use of a fixation cross that may have modulated attention to the central features of the faces. However, as a group, adults with ASD did not produce different ERP amplitudes (Webb et al., 2012) or latencies (McPartland, Dawson, Webb, Panagiotides, & Carver, 2004) to upright versus inverted faces, whereas the comparison group did. Latencies to faces corresponded with face recognition ability (McPartland et al., 2004) and adults with ASD with more normative facial inversion effects had better face memory performance (Webb et al., 2012).

Face Learning & Memory

Facial memory tests suggest that impairments continue during adolescence and adulthood, although whether this involves poorer encoding of new faces, recognition/recall for faces, or both, is unclear. Although very few studies have examined learning, reduced habituation to faces has been found in adolescents and adults with ASD, particularly faces with neutral expressions (Kleinhans et al., 2010; Swartz, Wiggins, Carrasco, Lord, & Monk, 2013; Tottenham et al., 2014). More is known about face memory, although the ASD group shows extensive heterogeneity. Adolescents with ASD exhibited reduced memory for faces but not houses on a “surprise” memory task (Arkush, Smith-Collins, Fiorentini, & Skuse, 2013). Performance of the teens with ASD across the face and object memory conditions was correlated, whereas it was not for the comparison group, suggesting that memory processing for faces may be domain general in ASD. Typically developing individuals continue to improve their memory for faces from 9 to 29 years, while improvement plateaus during adolescence for individuals with ASD (Greimel et al., 2014; O’Hearn, Schroer, Minshew, & Luna, 2010). Adults with ASD performed significantly worse on average on the Cambridge Face Memory Test (CFMT) relative to standardized norms. It should be noted that not all individuals with ASD are impaired in face memory—Hedley et al. found that while a quarter of the group performed two standard deviations below the mean, over half performed in the average range or above (Hedley, Brewer, & Young, 2011); reported standard deviations suggest that this is likely true for many other studies as well.

There is some evidence suggesting that memory deficits detected by adulthood are not unique to faces. A recent exploration of immediate memory ability for faces, face parts and objects revealed that adults with ASD had relatively more severe and more generalized impairments than children with ASD relative to age-matched comparison groups (O’Hearn et al., 2014). Specifically, memory impairment broadened from childhood to adulthood to include both objects and whole faces as well as both eyes and mouths. Given that most of the assessments of face processing, learning and memory in this age group involve individuals with average to above average ability, it is likely that there is heterogeneity in the recruitment of compensatory systems and behavioral strategies. Anecdotally, some high functioning individuals (Faja et al., 2008; Webb et al., 2012; Webb, Jones, Merkle, Murias, et al., 2010a) reported idiosyncratic learning and memory strategies that did not require configural processes and that could result in uneven performance dependent on how the stimuli and task were designed.

Relation to Social Ability

While basic face perception may show strong relations to social ability in early development, face learning and recognition demonstrates stronger relations with autism symptoms and social ability in older subjects. Decreased amygdala habituation for both adolescents and adults corresponded with autism symptom severity (Kleinhans et al., 2010; Swartz et al., 2013). ASD symptom severity during adolescence was predicted by face recognition but not emotion recognition ability during childhood above and beyond initial symptom severity (Eussen et al., 2015). In young adults without ASD, the degree of sub-clinical autism traits as well as gender and object recognition ability each predicted face recognition scores; lower face recognition was observed in males and in individuals with higher levels of ASD traits and lower object recognition scores (Halliday, MacDonald, Scherf, Sherf, & Tanaka, 2014). Similarly, neural activation during recognition tasks corresponds with autism symptoms (Lerner, McPartland, & Morris, 2013; Scherf, Elbich, Minshew, & Behrmann, 2015).

Summary

Thus, later development may represent maturation of some processes related to face processing but continues to be a period of impairment for face attention, learning and recognition with heterogeneity related to task demands and symptom severity. There is some evidence that youth with ASD show worsening impairments into adolescence/adulthood relative to age-matched controls. However, basic attention and processing mechanisms seem to be available and can be manipulated to produce more typical responses. A number of questions remain, but integral to understanding face learning is the question of how some adults with ASD are able to show relatively spared memory for faces given their less robust face attention and (lack of) configural processing biases. And, second, given the normative shift toward greater non-family social interactions with age (e.g., school, work), can plateaus in face processing be overcome through enriched focused social experiences?

A Failure of Emergent Specialization in ASD

Johnson has proposed that the cortical systems supporting face processing represent an emergent, activity-dependent specialization in response to face input (e.g., CONLERN --Johnson, Senju, & Tomalski, 2015; Morton & Johnson, 1991b). To acquire “neurotypical” face sensitivity and specialization, subcortical systems bias attention toward face-like stimuli in the first month of life; this system is then inhibited by the cortical system that comes on-line at 2 to 3 months as the infant receives intensive visual experience with faces through interactive social experiences. Klin, Shultz and Jones (2015) have proposed that ASD is marked by a delay in the transition from a reflexive orienting sub-cortical system to experience dependent cortical systems, which underlies adaptive attention to the eyes of others. Mundy et al. argue that one of the “most vital types of actions infants take involves the self control of their looking behaviors, or active vision” (Mundy, Sullivan, & Mastergeorge, 2009, p. 10). Active vision is important to the goal-directed selection of information from the environment and can be used to self-regulate arousal and affect (Posner & Rothbart, 2007). Active attention to faces and social partners would result in perceptual expertise based on these accumulated experiences. This dynamic system of attention to faces and emerging specialization of the neural structures will make facial information available to other developing cognitive systems efficiently and interactively.

From our review, among infants who go on to develop autism and toddlers with early autism symptoms, attention to faces and face parts begins to differ from neurotypical infants at 6 months of age (Chawarska et al., 2013, in press; Jones et al., 2015; Shic et al., 2014) with continued increasing atypicalities in the first years of life (Elsabbagh et al., 2009; 2012; 2013; Ozonoff et al., 2008; 2010; Rozga et al., 2011; Turner-Brown et al., 2013). Less attention to faces, and potentially increased attention to objects, will alter the types of information available to this perceptual learning system. If this system is domain specific (Kanwisher, 2010) then less experience with faces will result in less information for the system to organize around, potentially delaying the developmental trajectory or creating a system that is less efficient, robust and consistent. If the system is domain general (Gauthier & Nelson, 2001), then specialization for faces may be competing with significant experiences with other stimulus types. It may be that the altered trajectory reflects a combination of both less information on faces and increased experiences with alternative perceptual categories, all occurring in a context of delays related to early social communication (e.g., joint attention) and altered emotional tagging.

As the infant gains mobility (crawling, walking), the child begins to make active attention choices and can self-select sensory input. The extent to which the child finds social information informative, motivational, or rewarding will influence his or her exploration of the face and the integration of the face in dyadic and triadic interactions. For children with ASD, dyadic gaze for social referencing and joint attention is significantly impacted, with variability predictive of outcome (Charman et al., 2003). Social attention may also lose ground to object exploration, suggesting the rise in repetitive behaviors and circumscribed interest could result in lost opportunities for social perception.

Data from infants with visual impairment (Le Grand, Mondloch, Maurer, & Brent, 2001) suggests that there is a critical period for face perceptual development within the first 12 months of life for face neural specialization to occur, such that configural processing remains impaired regardless of socially normative post-correction visual experiences. Given that ASD more likely results in (consistent) decreased exposure to faces, it may be that the heterogeneity seen in middle childhood and adulthood results from the system maintaining some additional plasticity. Continued reduced attention to faces may be building a non-normative perceptual expertise system. Given that attention is not absent, the system may be maturing around misinformation. For example, if the child is attending to scenes or objects more than to social partners, and when looking at the social partner, the child is attending to atypical or less informative parts of the face (e.g., external features, mouths), these types of information will significantly influence perceptual prototypes. Thus, for lower functioning children or children overly focused on objects, perceptual expertise for non-social items (mechanical items, circumscribed interest items) progresses in the absence of (or very limited) information about the face. For children with some attention to mouths within faces (potentially related to language perception), the face system prototype will develop with an altered center of focus and an altered “purpose” for the face (i.e., in influencing the understanding of verbal language but potentially not in referencing or triadic information).

The process of specialization allows faster use of facial information and integrates it fluidly into complex cognitive tasks including assimilation with multimodal perception (auditory, tactile), multi-channel communicative cues (non-verbal gestures, emotional expressions), dynamic information (movement), and context (integration of semantic information). Decreased or degraded input to the system in any one of these streams is likely to result in an altered face perception and negatively influence social communication and social ability.

Thus, our supposition is not that the face processing and learning system is fundamentally broken in ASD, but that the input into the system is disrupted by decreased and altered attention, and this results in early delays in the development of perceptual specialization, potentially “breaking” other systems that rely on efficient facial information and thus resulting in atypical social communicative ability. Testing models such as this one will require longitudinal metrics that can be employed similarly across a wide age range (like EEG and eye tracking) but also integration with measures of social cognition, ability, and disability. Further integration with contrast groups that may have early attention or perceptual abnormalities, or overlapping autism phenotype (e.g., Fragile × Syndrome, Williams Syndrome), will allow for better understanding of the specificity and sensitivity of using face attention and perceptual measures as risk markers.

Given the long maturation trajectory of the face system, there is potential for continued acquired experience to have later impacts on the perceptual expertise system, potentially resulting in normalization with supplemental experience or training. However, at some point in development, this plasticity or longer path to normalization cannot correct the influence of the early atypical circuitry formation. Although many models of ASD intervention, such as Early Start Denver Model (ESDM), deliberately encourage attention to the face by fostering opportunities for a high-level of face-to-face interactions with the therapist or parent, we hypothesize that increasing attention to specific aspects of the face that contain key configural information, in combination with other social contextual information, is important in producing more normative neural social brain activity (Dawson et al., 2012; Faja et al., 2012). The facial world of the young child gives way to a larger and larger number of potential social partners via schooling and community interactions; a system that is efficient, robust and consistent will be necessary to correctly accommodate the expanding social world.

Conclusion

Individuals with ASD experience an altered social attention environment preceding symptom development and this sets the stage for early delays in face perception that result in a less efficient, robust and consistent face processing system and downstream atypicalities in social cognition and communication. We propose that there is a critical period for facial attention and learning in the first few years of life in which face attention, perception, and learning have a significant and broad impact on both social ability and social disability. However, there is also a wider temporal window of plasticity for facial perception in which experience may continue to mature the neural systems related to face sensitivity, with variable impairment in face specialization related to heterogeneity within the autism spectrum. This proposal suggests that “treating” face processing early in development (as a supplement to broader social-communicative and cognitive therapies) may positively influence social symptoms. As well, this longer period of plasticity also provides the opportunity to “correct” parts of the system throughout development and may be an important module in combination with targeting other early developing, remedial social attention systems. Face learning is a life-long necessity, as the social world of the individual only broadens with age. Thus, addressing both the source of the impairment in ASD as well as the trajectory of ability throughout the lifespan may serve to positively impact the lives of individuals who struggle with social understanding and information.