The role of racial and developmental experience on emotional adaptive coding in autism spectrum disorder

Abstract

Sensitivity to emotional faces aids in rapid detection and evaluation of others, such that by school-age, children and youth exhibit adult-like patterns when the prolonged viewing of an emotional face distorts the perception of a subsequent face. However, the developmental considerations of this phenomenon (known as emotional adaptive coding) are unclear given ongoing maturational and experiential changes, including the influence of own-race experiences or the lack of face expertise, as is evident in autism spectrum disorder (ASD). This study addressed whether emotional adaptive coding is sensitive to factors of face perception expertise, specifically self-race and developmental experience, in adults (age 19-28 years) and youth (age 10-16 years). Emotional adaptive coding was not influenced by race expertise (i.e., other versus same race identity) in White and Asian adults. Emotional adaptation coding during childhood and adolescence is consistent with adults, though youth with ASD exhibited stronger adaptor after-effects in response to other-race faces, relative to TD youth and adults. By extending prior work to examine the integration of race and emotional adaptive coding in ASD, we discovered that the strength of response in ASD is atypical when viewing other-race faces, which clarifies the role of racial and facial experience on emotional face adaption.

Short Abstract

Following prolonged viewing of a distorted emotional face, youth exhibit adultlike perceptual responses to a subsequent face known as emotional adaptive coding. This study addressed whether emotional adaptive coding is sensitive to factors of face perception expertise, specifically self-race and developmental experience, in adults (age 19-28 years) and youth (age 10-16 years). Emotional adaptive coding was not influenced by race expertise in White and Asian adults. Youth with autism spectrum disorder (ASD) did exhibit stronger adaptor after-effects in response to other-race faces, relative to non-ASD youth and adults, clarifying the role of racial and facial experience on emotional face adaption.

1. Introduction

Quick and accurate evaluation of facial information (e.g., emotional expression, identity) is critical for decoding social cues (Ito & Bartholow, 2009) and evaluating distinguishing features (Kim & Chung, 2018). Emotion perception is one crucial component in the identification of facial expressions, which provides individuals with an opportunity to establish interpersonal connections with others. Typically developing infants recognize basic emotional expressions by distinguishing between happy, sad and surprised faces as early as 7 months (Young-Browne, Rosenfeld, & Horowitz, 1977), and emotion recognition continues to develop during childhood into adolescence (Batty & Taylor, 2006; Malsert, Palama, & Gentaz, 2020; Turkstra, McDonald, & DePompei, 2001). However, facial emotion discrimination abilities are a known impairment within the hallmark social communication difficulties of autism spectrum disorder (ASD), at both the behavioral and neural level ((Webb, Faja, & Dawson, 2012; Webb, Neuhaus, & Faja, 2017). Although evidence suggests that rudimentary understanding of emotional expressions may be similar in ASD and typical development by adolescence (C.R.G. Jones et al., 2011), youth with ASD struggle with more complex emotion detection tasks (Rump, Giovannelli, Minshew, & Strauss, 2009),

One such complex task involves adaptive coding. Adaptive coding is a phenomenon in which prolonged viewing of a stimulus produces a subsequent after-effect, such that a subsequent stimulus is perceived to be opposite of the first adapting image (Rhodes & Leopold, 2012). That is, after adapting to a face, perception of a subsequent face is shifted in the opposite direction of the adapting stimulus. Evidence of adaptive coding have been found across a variety of different features, including sex (Davidenko, Witthoft, & Winawer, 2008; Jaquet & Rhodes, 2008; Little, DeBruine, & Jones, 2005; Palumbo, Laeng, & Tommasi, 2013; Pond et al., 2013), age (Little, DeBruine, Jones, & Waitt, 2008), race (Jaquet & Rhodes, 2008; Jaquet, Rhodes, & Hayward, 2007; Little et al., 2008), and emotional expression (Benton & Burgess, 2008; Hsu & Young, 2004; Russell & Fehr, 1987; Rutherford, Chattha, & Krysko, 2008). Here, we sought to evaluate if race expertise (i.e., other versus same race identity) influences emotional adaptive coding. In addition, we explored how increased experience (i.e., age) or a lack of expertise with faces, as proposed in autism spectrum disorder (ASD) (Dawson, Webb, & McPartland, 2005) may influence emotional adaptive coding.

1.1. Classic emotional adaptive coding and influences of racial experience

In standard adaptive coding paradigms, the adaptor image affects the appearance of subsequent images (Leopold, O’Toole, Vetter, & Blanz, 2000). In adults, adapting to an emotional face changes the subsequent image’s emotional category or intensity rating (Russell & Fehr, 1987). The classic adaptive coding paradigms measure the magnitude of perceptual distortion via after-effects (Webster & MacLin, 1999), which reflect the extent to which a person (after adapting to an anti-target) identifies the following neutral probe as a target.

Emotional adaptive coding follows adaption to pure emotions (Russell & Fehr, 1987; Rutherford et al., 2008) and partially occluded emotional faces (Luo, Burns, & Xu, 2017). Adults reliably indicate that sad and happy are indeed psychological opposites, such that after adapting to a happy face, adults see a neutral face as sad (Rutherford et al., 2008). An alternative is using an “anti-emotional” face, generated by morphing pure basic emotions (e.g. happy) into the opposite expression (e.g., anti-happy), which may appear sad in contrast to a normed expression (Skinner & Benton, 2010, 2012). After-effects are found to be larger for strong (i.e., further from the norm) than weak (i.e., closer to the norm) anti-emotions (Burton, Jeffery, Skinner, Benton, & Rhodes, 2013; Palermo et al., 2017). Other work systematically confirmed the generalizability of emotional after-effects across stimuli, spatial, and temporal characteristics (Hsu & Young, 2004).

Several studies targeting race adaptive coding have found that faces of different races are coded using race-specific norms. There is evidence of differences in adult same- versus other-race face processing (Meissner & Brigham, 2001) suggesting an other-race bias. For instance, White adults exhibited a larger after-effect for White-specific and Asian-specific faces compared to mixed race faces (Armann, Jeffery, Calder, & Rhodes, 2011), such that race-specific non-mixed faces produce a larger effect of distortion (i.e., participants were more likely to choose the opposite). The after-effect magnitude is also dependent on its category (or “category-contingent” per Little et al., 2008), meaning that effects will vary if the adapting stimulus and the probe are not both the same (e.g., Asian adaptor face and White probe face). However, findings are mixed regarding whether larger after-effects occur following adaptation when the adaptor and probe are race congruent (Little et al., 2008) or race incongruent (Jaquet et al., 2007).

1.2. Evidence of adaptation after-effects in typical and atypical development

During infancy, both facial emotional differentiation (Grossmann et al., 2011; Leppänen, Moulson, Vogel-Farley, & Nelson, 2007; Walker-Andrews, 1997) and other-race effects (Anzures et al., 2013; Hayden, Bhatt, Joseph, & Tanaka, 2007; Kelly et al., 2009; Sandy Sangrigoli & Schonen, 2004) are evident, suggesting early sensitivity to both within-individual changes (emotion) and between-individual differences (race). It has been proposed that the face processing system is tuned based on perceptual experience (de Haan, Johnson, & Halit, 2003; Pascalis, de Haan, & Nelson, 2002), which is supported by evidence of improvement of emotion (Herba, Landau, Russell, Ecker, & Phillips, 2006) and race (Pezdek, Blandon-Gitlin, & Moore, 2003; Walker & Hewstone, 2006) differentiation with age. In addition, discrimination of faces improves with increased social experience (McKone, Crookes, Jeffery, & Dilks, 2012), including exposure to other races (Walker, Silvert, Hewstone, & Nobre, 2007). For instance, children aged 3-9 years adopted from Korea into a majority White country have poor memory for White faces at time of adoption, but improve several years post-adoption (Heering, Liedekerke, Deboni, & Rossion, 2010; S Sangrigoli, Pallier, Argenti, Ventureyra, & Schonen, 2005).

In fact, there is evidence that neurotypical children demonstrate after-effect adaption as early as 4 years of age (Jeffery et al., 2010, 2011; Vida & Mondloch, 2009). Between 7 and 9 years of age, children exhibit larger after-effects for opposite adaptor-to-probe pairs (i.e., stronger likelihood that the adaptor distorted the perception of the probe). These effects are exaggerated for stronger adaptors (i.e., further from the average) for the face identity (Jeffery et al., 2011) and emotion adaption (Burton et al., 2013). The norm-based coding model (Webster & MacLeod, 2011) describes a developmental phenomenon by which schematic representation of faces develops as the individual becomes more familiar with more and more faces. From this perspective, we would predict that as experience with other faces increases, the strength of the adaptation effect will also become stronger (Rhodes et al., 2005). However, studies that have compared child and adult responses (Jeffery, Rathbone, Read, & Rhodes, 2013), find that by 8 years, children exhibit similar after-effect sizes to adults (Nishimura, Maurer, Jeffery, Pellicano, & Rhodes, 2008; Pimperton, Pellicano, Jeffery, & Rhodes, 2009), although some work indicates that children may need more extreme adaptors to show the effect (Anzures, Mondloch, & Lackner, 2009).

In contrast, there is growing evidence that adaptation after-effects are atypical in children and adults with ASD. (APA, 2013)(Webb et al., 2012, 2017). Recent studies using adaptation paradigms indicate reduced or atypical after-effect responses in individuals with autism compared to controls for identity in children age 8-16 years (Ewing, Pellicano, & Rhodes, 2013), judgments about perceptual events in children age 6-14 years (Karaminis et al., 2015), eye gaze direction in adults (Lawson, Aylward, Roiser, & Rees, 2017), and emotion in children age 9-16 years (Rhodes et al., 2017) and adults (Rutherford, Troubridge, & Walsh, 2011). In fact, ASD after-effect attenuation is specific to faces but not to nonsocial categories such as cars in children 8-16 years (Ewing, Leach, Pellicano, Jeffery, & Rhodes, 2013) or color in adults (Maule, Stanworth, Pellicano, & Franklin, 2016). In addition, a study compared the performance of high functioning children with ASD and age and IQ matched TD children using the anti-expression face in both Asian and White conditions (Kim & Chung, 2018). No significant interaction was found between groups and race conditions within this Korean sample, yet, overall ASD children showed significantly larger after effects than TD children regardless of race conditions, potentially signifying broad problems with adaptive coding. As the second study to target the interaction between race and emotion in adaptive coding, the current study adds to this knowledge by utilizing the same task in a multi-racial study of White and Asian adults and White youth with and without ASD.

1.3. Current study: Overview and objectives

Given that our experience with other faces continually changes over time, it is possible that the strength of these systems may also change, as is evident across development (Zhou, Liu, Ding, Fu, & Lee, 2016). Considering individual variability in interaction with other races based on self-race and community race, we would expect that maturation of the visual system, as well as experience, would alter the strength of emotional adaptive coding. In this study, we investigated the extent by which the race of a face influences the emotion after-effect in adults (N = 40), youth (N = 20), and youth with ASD (N = 20). During adaptation trials, participants viewed an image of a face (the adaptor) that varied in anti-emotional content (anti-happy, anti-sad) and racial identity (Asian, White) for a prolonged period (4 seconds, adapting phase). We predicted increased after-effect strength for within-racial group faces (adults only) due to evidence indicating adult differences in self- versus other-race processing (Meissner & Brigham, 2001). Next, we examined after-effect in the whole population of adults and youth. Considering the fact that adults and 8-year-old children exhibit similar after-effect strengths to adults (Jeffery et al., 2013), we expected similar behavioral responses in youth as in adults, but we predicted impaired performance in the ASD group, consistent with other work (Ewing, Pellicano, et al., 2013).

2. Materials and Methods

2.1. Participant characterization

A total of 79 participants (36 female) participated in this study and were included in the analyses (see Table 1 for full participant characteristics). Youth were between 10-16 years of age, and adults were between 19-28 years of age. Based upon a priori sample size estimates via G*Power (Faul, Erdfelder, Lang, & Buchner, 2007), at least 6 per adult group and 14 per child group is necessary to achieve 95% power based upon prior emotion after-effect work (Jeffery et al., 2011; Palermo et al., 2017). Thus, our final sample of 20 per adult group, 20 TD youth, and 19 ASD youth was sufficiently powered. Adults self-identified as White (N=20) or Asian (N = 20) upon enrollment, and this was confirmed via a questionnaire (Racial Experience Measure, see below). Only White youth participants were included in this study, although one non-White control child did complete this experiment. Participants were excluded if they had active seizures within the past year, current use of any benzodiazepine, barbiturate, or anti-epileptic medication for any reason. All had normal or correct-to-normal vision. Participants were recruited through the local psychology subject pool, advertisements, participation in previous research studies, or word of mouth. All groups had comparable proportion of female participants, p > .20.

Table 1. Demographic characteristics.

Abbreviations: ASD= Autism Spectrum Disorder; M = Mean; SD = Standard Deviation; AQ=Autism Quotient; BAPQ=Broader Autism Phenotype Questionnaire; RMET=Reading the Mind in the Eye Test; N/A = Data not available.

	Asian Adults	White Adults	Youth without ASD	Youth with ASD
Analytical multilevel model (MLM)	1, 2	1, 2	2	2
N (% female)	20 (50%)	20 (50%)	20 (50%)	19 (32%)
Measure	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
Age Years M (SD)	24.5 (2.3)	23.5 (7.6)	12.7 (3.2)	12.7 (1.2)
AQ Total Score M (SD)	12.0 (6.0)	16.6 (6.8)	23.6 (22.5)	53.4 (27.6)
BAPQ Total Score M (SD)	91.4 (19.6)	96.6 (30.3)	N/A	N/A
RMET Gender subscore M (SD)	N/A	N/A	26.9 (0.97)	26.3 (2.2)
RMET Emotion subscore M (SD)	N/A	N/A	21.4 (2.9)	18.6 (5.2)
Self- or parent-reported racial experience
Environmental % Asian	34.1 (19.7)	20.6 (13.6)	16.4 (15.8)	11.1 ( 15.6)
Environmental % White	42.3 (20.1)	58.4 (18.7)	64.6 (18.4)	56.4 (18.5)

TD adults (n = 40, 20 female) and TD youth (n=20, 10 female) were excluded if ever diagnosed, referred, or suspected of having a developmental or psychiatric disorder; including family history of ASD or current elevated autism symptoms on the Autism Questionnaire (AQ) (Auyeung, Baron-Cohen, Wheelwright, & Allison, 2008; Baron-Cohen, Wheelwright, Skinner, Martin, & Clubley, 2001) or Broader Autism Phenotype Questionnaire (Hurley, Losh, Parlier, Reznick, & Piven, 2007). Youth had higher scores on the AQ relative to the adults (p = .03). ASD youth participants (N = 19, 6 female) consisted of White adolescents with ASD. One additional male child with ASD was excluded due to technical issues with the experimental task and is not included in the participant demographic table or subsequent analyses. ASD diagnosis was confirmed by expert clinical judgment based on Diagnostic and Statistical Manual of Mental Disorders 5 criteria (American Psychological Association, 2013), including assessment via the Autism Diagnostic Interview-Revised (C Lord, Rutter, & Couteur, 1994), the Autism Diagnostic Observation Schedule-2 (Catherine Lord et al., 1989), and medical history interview. To be included in analyses, all TD and ASD youth had nonverbal IQ scores greater than 80, indicative of average to above average cognitive abilities. One TD youth participant was excluded due to an IQ below 80. As indicated in Table 1, consistent with their clinical diagnosis, ASD youth (compared to TD youth) had higher scores on the AQ (p < .001). In addition, ASD youth exhibited a reduction in mental abstraction via the Reading the Mind in the Eyes Task (RMET) (Baron-Cohen, Wheelwright, Spong, Scahill, & Lawson, 2001) specific to reading emotional expressions (p = .05) but not gender (p = .24).

All procedures performed in studies involving human participants were in accordance with the ethical standards of the University of Washington (adults) and the Seattle Children’s Research Institute (youth participants) and national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Researchers discussed the study procedures with all participants, and informed consent was obtained: informed consent by the adult participants, informed parental permission for youth participants, and informed assent by youth participants.

2.2. Racial Experience Measure:

2.2.1. Self-Race and Ethnicity:

To assess racial and ethnic background, we asked adult participants and the parents of youth to report on the race/ ethnicity of the participant’s biological mother and father, identifying the country and/or region of ancestor origin, specifically focusing on percent of ancestry that could be categorized as White/Caucasian or Asian. All White/Caucasian participants identified 100% maternal and paternal white ancestry (46 Western European, 6 Eastern European, 7 Mixed European descent). Asian participants identified 100% maternal and paternal Asian ancestry (12 Chinese, 3 Taiwanese, 2 Korean, 2 Vietnamese, 1 Filipino).

In addition, adult participants and parents of youth reported the percentage of Asian and White individuals within the participant’s local educational, work, or neighborhood environment (hereto referred to as “environment”; Table 1). Participants were provided with the racial demographic distribution of their self-reported education institution or neighborhood (values were derived from published census reports). The individual was then asked to “adjust” the values to match their individual experience. White adults indicated greater environmental exposure to other Whites; and similarly, Asian adults indicated greater environmental exposure to other Asians (ps < .02). Notably, in our Youth samples (all White), there were no significant differences in exposure to race (White or Asian) within the youth’s environment compared to the White adults’ experience (p > .29).

2.3. Experiment Procedure:

2.3.1. Stimuli:

See Figure 1 for stimuli overview and examples. Average face morphs were generated via Fantamorph 3.0 software (Abrosoft, www.fantamorph.com) using four identities (two males, two females) for each race (White, Asian) for each basic emotion (Happy, Sad) from a common face database (Tottenham et al., 2009) as a base for morphing. First, to create a Neutral probe for each race, expressions were averaged within identity across the six basic expressions (happiness, sadness, fear, anger, disgust, and surprise) and one neutral expression. To create the pure Happy probe and Sad probe used for probes during the control trials and as the basis of the anti-face morphs, the respective emotional expression was averaged within race and across identities. To create the test adaptor stimuli (Anti-Sad adaptor and Anti-Happy adaptor), anti-emotion faces were generated by morphing the happy probe and sad probe, respectively, such that the anti-face was extrapolated in the opposite direction beyond the average neutral probe to equal intensity, separately for each race. Lastly, a neutral white outline of a face as the same shape and size as the other stimuli was used for baseline trials. To better account for the size-tuning of the after-effect (Zhao & Chubb, 2001), adaptor stimuli varied in size, either large (visual angle 9.6° x 9.1°) or small (visual angle 7.2° x 6.8°).

Figure 1. Stimuli examples.

Examples of each of the four trial types are presented for the Asian face stimuli. The congruent (i.e., predicted perception) after-effect response is noted.

Abbreviation: ITI, Intertrial interval.

2.3.2. Directions:

Participants received verbal instructions to accurately and quickly categorize the second, briefly presented face (i.e., the probe) as ‘happy’ or ‘sad’. Prior to the task, participants were given eight trials to practice pressing the correct button identifying an emotional face (happy, sad). Button presses (e.g., sad / right) were counterbalanced across participants.

2.3.3. Experiment:

A trial was comprised of: (1) A 500 msec fixation cross; (2) The 4000 msec adapting stimulus; (3) The 200 msec neutral probe stimulus; (4) The 1000 msec response period within which participants were required to identify as a happy or sad face; (5) 500-700 msec inter-trial interval. Altogether, 160 trials (5 blocks of 32 trials) were presented in random order, and the task took approximately 30 minutes to complete.

Data collection included three trial types (as seen in Figure 1): (1) 20 Baseline trials consisted of the outline adaptor image and a neutral probe and were included to assess responses unaffected by the adaptor. These catch trials were used to determine any biases in happy and sad responses. There is no “correct” response on these trials. (2) 20 Control trials consisted of anti-emotion adaptor and an emotion probe of the same emotion label and were included to verify participants’ overall responses to a congruent adaptor and probe. The control trials consisted of two sub-conditions: Congruent sad condition and congruent happy condition. (3) 120 After-effect trials consisted of anti-emotion adaptor and a neutral probe. Thus, the after-effect condition included two sub-conditions: anti-happy and anti-sad. The adaptor and probe were of the same race. An additional manipulation of size was included (small/large) based on prior work (Zhao & Chubb, 2001). In 50% trials, the adaptor was large and the probe small; and in the other 50% of trials, the adaptor was small and the probe large.

2.4. Acquisition Method

Stimuli were presented using E-Prime 2.0 software (Psychology Software Tools, Inc, Pittsburgh, PA). The display used for adults was a SmartEye DR 120 24” widescreen monitor and the display used for youth was a Dell UltraSharp U2312HM 23-inch Widescreen flat panel monitors with soundbar. Both displays used a refresh rate of 60 Hz. The display was viewed binocularly at a distance of 75 cm. The button box was placed on top of a table positioned in front of participants. Simultaneous electroencephalography (EEG) data was acquired but is not discussed within the current study.

2.5. Analytic Method

2.5.1. Behavioral Response.

Analyses were constrained to trials with a meaningful response, such that we excluded responses that were too quick (i.e., reaction time less than 200 msec) and responses that were beyond 2 standard deviations of each participant’s mean reaction time (Izatt et al., 2014). As predicted, preliminary multilevel models (available in Supplemental Materials) indicated that for all experiments, participants performed at chance for baseline conditions and performed above chance during control conditions. Analyses here focus on anti-emotion conditions, specifically evaluating the magnitude of the after-effect.

2.5.3. Statistical models.

We opted to use a single-trial mixed multilevel model (MLM) approach as a means to best account for individual variability related to critical factors of interest (i.e., adapting stimulus anti-emotion or race). This approach (similar to general linear modeling) focused analyses on within-participant deviations across trials by modeling individual variance using a covariance matrix that represents the variation around each participant’s mean (Hoffman, 2015). In this way, by including a random intercept per subject, MLMs adjust for each individual subject’s unique response relative to the main sample to give a clearer understanding of how different factors (e.g., stimulus race, stimulus anti-emotion, group) influence response. This approach has been successfully adopted to account for ongoing intra-individual variability (Hudac et al., 2018, 2017).

All analyses were conducted via SAS 9.4 (SAS Institute) using restricted maximum likelihood (REML) to maximize the variance parameters assuming a Gaussian distribution (Corbeil & Searle, 1976; Gilmour, Thompson, & Cullis, 1995) and Satterthwaite denominator degrees of freedom commonly used to test fixed effects (Hoffman, 2015; Satterthwaite, 1946). A series of MLMs were generated using PROC MIXED procedure to describe the variances and covariances of after-effect size behavior. The accuracy of the emotional categorization of the probe stimulus was reflective of the strength of the after-effect. Within the single-trial mixed MLM approach, input data includes the subject’s averaged value per condition with a random intercept included for each participant to account for individual variability. To this extent, the MLMs account for within-person and between-factor variability and will have higher degrees of freedom than a traditional ANOVA approach. All models included effects of stimulus properties with subsequent full-factorial interactions, including: Stimulus Race (2: Asian, White), Stimulus After-Effect sub-condition (2: anti-happy, anti-sad), and Stimulus Size tuning (2: large adaptor, small adaptor). An additional Group variable was included in the full-factorial for each model: (a) in the first MLM, group represented self-identified race (2: Asian, White); (b) in the second MLM, group represented developmental group (3: TD Adult, TD Youth, ASD Youth). Lastly, all models also included a non-interacting fixed effect of racial environment, as adjusted by the participant or participant’s parent based upon the participant’s own experience and exposure to Asian individuals. Racial experience to White and Asian was inherently collinear (i.e., those with a high White environment tended to experience a low Asian environment), thus only the one predictor was added to the model. All model results are reported in Table 2.

Table 2. Multilevel model (MLM) results.

Model results are presented for (1) adults only, grouped based upon self-identified race (Asian, White); and (2) all participants, grouped based upon developmental group (TD adult, TD youth, ASD youth).

Participants include Group variable	(1) Adults only By self-identified race		(2) All participants By developmental group
	F value	p value	F value	p value
Stimulus Anti-Emotion (SAE)	F(1, 280) = 67.09	<.0001	F(1, 518) = 209.1	<.0001
Stimulus Race (SR)	F(1, 280) = 15.21	0.0001	F(1, 518) = 15.35	0.0001
Stimulus Size of Adaptor (SSA)	F(1, 280) = 5.07	0.0252	F(1, 518) = 4.46	0.0351
Group	F(1, 40) = 0.71	0.4061	F(2, 74) = 1.78	0.1755
Asian experience	F(1, 40) = 0.94	0.3386	F(1, 74) = 0.61	0.4389
SAE x SR	F(1, 280) = 171.3	<.0001	F(1, 518) = 137.66	<.0001
SAE x SSA	F(1, 280) = 0.14	0.7099	F(1, 518) = 0.06	0.8059
SR x SSA	F(1, 280) = 0.32	0.5711	F(1, 518) = 0.01	0.9403
SAE x SR x SSA	F(1, 280) = 1.43	0.233	F(1, 518) = 1.24	0.2664
SAE x Group	F(1, 280) = 0.19	0.6655	F(2, 518) = 7.77	0.0005
SR x Group	F(1, 280) = 0	0.9462	F(2, 518) = 0.33	0.7191
SAE x SR x Group	F(1, 280) = 0.54	0.4646	F(2, 518) = 8.06	0.0004
SSA x Group	F(1, 280) = 0.33	0.5666	F(2, 518) = 0.32	0.7273
SAE x SSA x Group	F(1, 280) = 1.79	0.1819	F(2, 518) = 0.32	0.7266
SR x SSA x Group	F(1, 280) = 0.02	0.8865	F(2, 518) = 0.24	0.7835
SAE x SR x SSA x Group	F(1, 280) = 1.2	0.2743	F(2, 518) = 0.09	0.9155

To fulfill our objective of evaluating the emotion after-effects manipulation within each group, post-hoc tests with Bonferroni HSD correction for multiple comparisons of sub-condition significance (i.e., anti-happy vs. anti-sad) were conducted for each stimulus race, regardless of presence of a significant interaction.

3. Results:

3.1. Influence of self-race (Asian vs. White) in adults.

We examined the influence of self-identified race within the adult participants. To visualize our results, observed after-effect sizes collapsed across participant race and size of adaptor are presented in Figure 2a. Responses were indicative of the after-effect (i.e., greater than chance in the predicted direction) for all conditions except one: the response in the condition White Anti-Happy was below chance for White participants (0.31-0.47, 90%) and below to at chance for Asian participants (0.35-.52, 90% confidence interval). This may suggest that the Anti-Happy condition did not generate an after-effect. Full presentation of main effects and interactions statistics in Table 2. Notably, there were no significant effects or interactions with participant self-race group, ps > .18. confidence interval). Our findings indicated four significant effects: (1) A main effect of after-effect sub-condition that indicated a larger after-effect for anti-sad (M =.82, SD =.24) than anti-happy (M =.62, SD =.34). (2) A main effect of stimulus race that indicated a larger after-effect for Asian faces (M =.77, SD =.23) than White faces (M = .67, SD =.36). (3) A main effect of stimulus size of adaptor that indicated a larger after-effect for small adaptors (M =. 75, SD = .29) than large adaptors (M =.69, SD = .33). Lastly, (4) an interaction between after-effect sub-condition and stimulus race, such that after-effect size were largest for White anti-sad faces (M =.93, SD =.17) and smallest for White anti-happy faces (M =.41, SD =.31). The pattern was reverse for Asian faces, such that the effect was largest for anti-happy (M = .82, SD =.22) relative to anti-sad (M =.71, SD =.24) faces.

Figure 2. Behavioral aftereffect results.

Percent responses identified as “correct” (i.e., expected emotion identification) after-effect response for stimulus race and anti-emotion sub-condition. All effects are shown collapsed across stimulus adapator size, as this did not interact with group effects. Boxplots shading represents the lower to upper quartiles, and the solid horizontal lines represent the median. The dashed line indicates the equal-probability line. Panel A represent adult participant responses based upon self-identified race. Panel B represents all participants, with adults collapsed across self-race.

The effect of racial experience was not significant

3.2. Influence of development.

In the second model, we included all participants to test the influence of development between TD adults, TD youth, and ASD youth. To visualize our results, observed after-effect sizes collapsed across adult ethnicity and size of adaptor are presented in Figure 2b. First, it is important to note that patterns similar to the first MLM were observed for stimulus properties, including main effects of all three stimulus properties and the after-effect sub-condition by stimulus race interaction (see Table 2). Specifically: (1) A main effect of after-effect sub-condition that indicated a larger after-effect for anti-sad (M =.81, SD =.22) than anti-happy (M =.56, SD =.33). (2) A main effect of stimulus race that indicated a larger after-effect for Asian faces (M =.73, SD =.25) than White faces (M =.65, SD =.35). (3) A main effect of stimulus size of adaptor that indicated a larger after-effect for small adaptors (M =.71, SD = .30) than large adaptors (M =.67, SD = .31). Lastly, (4) an interaction between aftereffect sub-condition and stimulus race, such that after-effect sizes were largest for White anti-sad faces (M =.91, SD =.16) and smallest for White anti-happy faces (M =.39, SD =.30). The pattern was reversed for Asian faces, such that the effect was largest for anti-happy (M =.73, SD =.27) relative to anti-sad (M =.72, SD =.23) faces. Similarly, the response in the condition White anti-happy was below chance (0.36-0.43, 90% confidence interval), suggesting that this manipulation did not generate an after-effect.

Second, although the main effect of group was not significant (p=.18) there were two interactions with group: (1) First, the after-effect sub-condition (i.e., anti-happy vs. anti-sad) varied by group, such that TD adults exhibited comparable adaptor after-effects (i.e., the effect of anti-sad compared to anti-happy) relative to TD youth participants, F(1,518) = 2.18, p =.14. In contrast, ASD youth had stronger adaptor after-effects relative to TD adults, F(1,518) = 15.43, p <.0001, and relative to TD youth, F(1,518) = 2.25, p =.03. In other words, although adaptor after-effects were similar in TD adults and TD youth, the ASD youth exhibited a stronger magnitude of the adaptor effect relative to both other groups. (2) Second, an additional interaction between after-effect sub-condition, stimulus race, and group indicated that this first interaction suggesting that ASD youth exhibited adaptor after-effects was specific to group effects in the representation of Asian faces (p’s <.011) but not White faces (p’s >.66). In other words, the three-way interaction indicates that ASD differences are observed only when viewing other-race Asian faces, but that ASD exhibited TD-comparable adaptor effects when viewing White faces.

The effect of racial experience was not significant.

4. Discussion

Here, we investigated self-race and developmental experience as potential contributory factors of emotional adaptive coding in samples of TD adults (18 – 28 years) and youth (10-16 years of age) with and without ASD. We captured emotion after-effects via a face adaptation task in which we measured behavioral responses to a neutral face after a brief 4-second adaptation to an anti-emotional face. First, we determined that adult participant race did not modulate emotional adaptive coding, implicating a similar mechanism for both self- and other-race adaptive processing during adulthood. We then extended these results by indicating comparable behavior between TD adults, TD youth, and ASD youth, specifically for White stimuli. However, group differences indicated a stronger adaptor after-effect for ASD youth relative to TD adults and youth in response to Asian faces (i.e., other-race faces). These effects may be modulated by larger adaptor responses to anti-happy Asian faces in TD adults or larger adaptor responses to anti-sad Asian faces in ASD youth. The effect of racial experience was not significant as measured by self- or parent-report on neighborhood environment.

4.1. Differential strength of after-effect.

Emotional adaptive coding was noted for three of the four sub-conditions in all participants, such that adaptation to an anti-emotional face changed the perception of subsequent neutral probe. These effects were observed in adults and youth and were consistent with prior work indicating that emotional adaptive coding follows adaption to pure emotions in adults (Russell & Fehr, 1987; Rutherford et al., 2008). After-effects were not atypical for youth with ASD, indicating that the developmental time course for adaptive coding was similar to neurotypical youth.

However, at the group level, this paradigm failed to capture an after-effect in response to White anti-happy faces. That is, after viewing the anti-happy White adaptor, on average, participants perceived the neutral probe as “sad”, which is incongruent to expectations and this finding is also inconsistent with prior work (Rutherford et al., 2011). Upon inspection of individual performance (evident in Figure 2), it is evident that there is more cross-subject variability in this sub-condition than other sub-conditions, such that approximately 20-25% of responses within each group performed opposite of expectations (i.e., perceives the neutral probe as “happy” in this condition). None of the planned comparisons between subjects in this study clarified what may be accounting for these individual differences. In previous work, after-effects were obtained in as little as one second of adaptation (Burton, Jeffery, Bonner, & Rhodes, 2016), thus it is unlikely that our 4 second exposure was too short. Additionally, a recent study using the same stimuli set in a Korean sample of adults, as well as youths with and without autism (Kim & Chung, 2018) implicated a weakness in the ability of the stimuli to translate cross-culturally. In other words, the Korean population may have been more sensitive to small changes of others’ expressions whereas other populations (such as in the current study) may require stimuli with stronger valence. Another possibility is that the valence of this condition was particularly weak or that properties of the stimuli (e.g., arousal) interfered with the adaptation or otherwise produced an implicit bias. For instance, the anti-happy White adaptor (expected to be perceived as a negative emotion) could potentially be perceived as “happy” or “anti-sad” due to unconscious positive bias for White faces (Dasgupta, McGhee, Greenwald, & Banaji, 2000; Monteith, Voils, & Ashburn-Nardo, 2005). Considering the success of other studies for a similar sub-condition (Armann et al., 2011), it would be helpful to evaluate these effects cross-culturally and within other racial groups. Lastly, as noted in more detail below, it is possible that the adaptation effects are driven by something other than the affective quality of the stimuli. All stimuli were generated by morphing across four identities, and it could be the case that the output of the anti-happy White stimuli morph was poor.

4.2. Relation of self-race and racial experiences to after-effects in adults

Past visual experiences generate a set of norms that are used as a reference for encoding faces (Webster & MacLeod, 2011). As a confirmation of adult processes, we hypothesized that adults would elicit stronger after-effects in response to within-racial group stimuli due to having more extended experience with self-race than other-race faces. Yet, there were no racial group-level behavioral differences between adults self-identified as Asian versus White, suggesting that after-effect patterns were not based upon the perceiver’s race during this task. These findings are largely inconsistent with decades of work related to self- versus other-race processing of an own-race bias (Meissner & Brigham, 2001) that indicate increased performance for own-race faces, but do support the perspective of growing cross-cultural agreement in emotion recognition (Yan, Andrews, Jenkins, & Young, 2016). The Asian population within the geographic region of this study has grown significantly over the past two decades (N. A. Jones & Chief, n.d.), thus, people within this region are more exposed to diverse ethnic groups. It is unclear if the lack of race effects are driven by sociocultural other-race issues or are instead related to stimulus properties. Since the stimuli were generated using average morphs of faces, it is possible that this process paired with the removal of racially specific features, such as hair and other external features, may have accounted for the lack of the other-race effect. Despite utilizing an a priori power analysis, another possibility is that a larger sample size would be required to support own-race differences, given our p=.18.

4.3. Relation of developmental status to after-effects in youth with and without ASD

Our final objective was to add to the growing literature addressing reduced anti-emotion adaptation after-effects in ASD (Rhodes et al., 2017; Rutherford et al., 2011) by evaluating across racial faces. We hypothesized that youth with ASD would have a lack of facial expertise, and thus, would have reduced responses to emotional adaptive coding. Given prior evidence of adult-comparable after-effect sizes after the age of 8 years old (Nishimura et al., 2008; Pimperton et al., 2009) we predicted that the TD youth in this study would exhibit robust, adult-like aftereffects. Indeed, TD youth and ASD youth exhibited similar behavioral after-effects patterns to adults specifically in response to White faces, suggestive of established norm-based face templates (i.e., same-race faces consistent with their experiences). There are developmental shifts related to emotional face sensitivity that may not have been observed in this study due to the older age of the youth. For instance, children reach adult-like behaviors by 5 years of age, whereas skills related to other more complicated emotions (e.g., disgust, surprise) emerge between age 5-10 years (Gao & Maurer, 2009, 2010). There continue to be similar but not adult-equivalent behaviors in adolescents such that older youth exhibit diminished accuracy and intensity for emotions that are more challenging than happy, such as sadness (Herba et al., 2008). It may be the case that if this study was repeated in a larger sample with younger children allowing for a continuous analysis of age, a linear association with age may be more apparent.

There was one critical difference between the three groups, but it was limited to Asian sub-conditions. Specifically, when considering the effectiveness of the anti-emotion condition, the ASD group exhibited stronger adaptor after-effects in response to Asian / other-race faces, relative to both TD youth and adults, which may be related to prior work in adults indicating reduced other-race effect in ASD when tasked with identifying White and Asian faces (Hadad, Schwartz, & Binur, 2019). The ASD youth exhibited stronger responses to anti-sad Asian faces than the TD groups, which may implicate a specific atypical mechanism related to negative emotions, such as sadness. This is somewhat aligned with other work suggesting that ASD participants were more likely to see neutral images as sad after adapting to negative emotions (e.g., fear, disgust) (Rutherford et al., 2011). Additionally, parents reported that the youth with ASD in this sample did experience a majority White (56.4%) relative to Asian environment (11%) within their local neighborhood and school area. Although this measure did not significantly contribute to the statistical model, the environmental differences may contribute to the lack of facial expertise, particularly norm-based facial schemes for Asian faces.

It may be helpful to further emphasize that unlike other evidence of reduced after-effects strength in ASD (Rhodes et al., 2017), this study highlights differential strength of response in ASD, but similar pattern of results all conditions and all groups. To our knowledge, although previous studies of have either focused on race (Jaquet, Rhodes, & Hayward, 2008; Little et al., 2008) or emotional expression (Benton & Burgess, 2008; Hsu & Young, 2004; Russell & Fehr, 1987; Rutherford et al., 2008), this is the first study to target how these two features interact in adults of different races and White youth with and without ASD, building upon work in a South Korean population of school-age children with and without ASD (Kim & Chung, 2018). Anti-emotion adaption after-effect reductions have been previously noted in ASD (Rhodes et al., 2017; Rutherford et al., 2011). However, research is mixed in response to neural responses differentiating primary emotions in ASD, with most studies suggesting either no difference in response in children 6-10 years of age (Wong, Fung, Chua, & McAlonan, 2008) or non-emotion specific differences in children 8-13 years of age (Tye et al., 2014) As seen in Figure 2, responses were extremely variable to the anti-happy faces. It may be that there is significant heterogeneity in the emotion and face processing system resulting in phenotypic differences, or alternatively, the specific stimuli used within the experiment may elicit varied responses.

A critical consideration regards what mechanism/s are involved in the hallmark atypical social communication in ASD. For instance, the anti-emotion adaption after-effect reductions previously noted in ASD (Rhodes et al., 2017; Rutherford et al., 2011) may stem from perceptual impairments. To better understand these other-race sensitivities, it may be helpful to disentangle the neural processes engaged during both adaptation and response selection in response to the probe. For one, prior research is mixed in response to ERP responses differentiating primary emotions in ASD, with most studies suggesting group differences but not atypicalities in emotion differentiation (e.g., overall group amplitude differences that are not specific to different emotions). For instance, Wong et al. (2008) found intact P1, N170 and P2 responses to emotions in children aged 6-10 years compared to controls, while Tye et al. (2014) found overall reduced N170 responses in children with ASD aged 8-13 regardless of emotion type. It may also be possible that learning mechanisms may be disrupted in ASD and result in atypical adaption patterns, consistent with neuroimaging findings describing delayed social brain activation in ASD during affective face perception (Kleinhans, Richards, Greenson, Dawson, & Aylward, 2016). To best address these outstanding questions, future work would benefit from examining ongoing dynamics of neural face perception with a continued examination of traits related to ASD (e.g., social responsivity, social motivation, general perceptual skills).

6.6. Conclusion

This study aimed at investigating the extent by which emotional facial adaptation is influenced by our experience with faces, such as within-racial group faces, changing experience development, and lack of experience with faces in ASD. Following adaptation to Asian and White faces expressing anti-emotions, behavioral responses indicated similar responses to White faces across TD adults, TD youth, and youth with ASD. This study clarifies the role of experience on emotional facial adaptation, and highlights potential areas of development in youth with and without ASD.

Data accessibility statement:

Data that support the findings of this study are available from the corresponding author, C.M.H., upon reasonable request.