Measuring change in facial emotion recognition in individuals with Autism Spectrum Disorder: A systematic review

Abstract

Children and adults with Autism Spectrum Disorder (ASD) are less accurate in facial emotion recognition (FER), which is thought to contribute to impairment in social functioning. Although many interventions have been developed to improve FER, there is no consensus on how to best measure FER in people with ASD. This lack of agreement has led to wide variability in how FER is measured and, subsequently, inconsistent findings related to impact of intervention targeting FER impairment. The purpose of this review is to synthesize the extant research on measurement of FER in the context of treatment. We conducted an electronic database search to identify relevant, peer-reviewed articles published between January 1998 and November 2019 to identify studies evaluating change in FER in ASD. Sixty-five studies met inclusion criteria, utilizing a total of 36 different assessment measures for FER in individuals with ASD. Only six of the measures were used in multiple studies conducted by different investigative teams. The outcomes of the studies are reported and summarized with the goal of informing future research.

Lay Abstract

Children and adults with Autism Spectrum Disorder (ASD) show difficulty recognizing facial emotions in others, which makes social interaction challenging. While there are many treatments developed to improve facial emotion recognition (FER), there is no agreement on the best way to measure such abilities in individuals with ASD. The purpose of this review is to examine studies that were published between January 1998 and November 2019 and have measured change in facial emotion recognition to evaluate the effectiveness of different treatments. Our search yielded 65 studies, and within these studies, 36 different measures were used to evaluate FER in individuals with ASD. Only six of these measures, however, were used in different studies and by different investigators. In this review, we summarize the different measures and outcomes of the studies, in order to identify promising assessment tools and inform future research.

Facial emotion recognition (FER), a component of social cognition (Adolphs, 2002), is fundamental for effective social communication and interaction (Ekman, 1992; Wang, Dapretto, Hariri, Sigman, & Bookheimer, 2004). Valid and clinically sensitive assessment of FER is critical to effectively address FER impairments. Typically, the ability to discriminate discrete emotions based on facial expression alone develops early in childhood. By 7 months of age, babies are able to discriminate dynamic happy and angry faces (Soken & Pick, 1992), and by 4 years of age, typically developing children can verbally label most basic, prototypical emotions with accuracy (Widen & Russell, 2003). Given the predictable onset, timing, and trajectory of this process in typically developing children, delay in FER may be meaningfully related to certain forms of atypical development and psychopathology. Consistent with this idea, FER impairments have been documented in several clinical conditions, including externalizing disorders (e.g., Aspan, Vida, Gadaros, & Halasz, 2013) and depression (e.g., Jenness, Hankin, Young, & Gibb, 2014). FER atypicalities are perhaps most widely documented in youth with autism spectrum disorder (ASD; e.g., Lozier, Vanmeter, & Marsh, 2014).

The FER atypicalities documented in youth with ASD vary according to identifiable dimensions of emotional expression including valence, arousal, and the characteristics of actors displaying emotion. For example, relative to typically developing peers, young children with ASD experience more difficulty with recognition of certain expressions depending on the overall affective valence of the displayed emotion (e.g., Rump, Giovannelli, Minshew, & Strauss, 2009) and are more adept at recognizing emotions in familiar people relative to strangers (Shanok, Jones, & Lucas, 2019). Thus, measurement strategies aimed at quantifying FER should, at minimum, consider known dimensions of complexity including valence, intensity, and specificity. However, a meta-analysis of studies exploring FER by Lozier et al. (2014) highlights the variability of results in terms of emotion-specific FER deficits in ASD. While some studies suggest deficits exist across a variety of emotions (e.g., Rump et al., 2009), others indicate deficits primarily for negative emotions (Ashwin, Chapman, Colle, & Baron-Cohen, 2006; Wingenbach & Brosnan, 2017), and still others point to specific emotions such as sadness (e.g., Boraston, Blakemore, Chilvers, & Skuse, 2007) or fear (e.g., Pelphrey et al., 2002). Most research indicates that from 12 years of age through adulthood, individuals with ASD do not show impairment in recognizing basic emotions (Capps, Yirmiya, & Sigman, 1992; Grossman, Klin, Carter, & Volkmar, 2000); they do, however, show difficulty when stimuli are more subtle or complex, and when presented briefly (Humphreys, Minshew, Leonard, & Behrmann, 2007). Difficulty recognizing emotions, especially complex ones, seems to stem, in part, from altered functioning of the social brain (Black et al., 2017), such as attending to mouth more than eye region of stimulus (Black, Chen, Lipp, Bolte, & Girdler, 2020). In everyday social interactions, emotional expressions tend to be subtle, complex, and brief; as such, any type of FER impairment, even if mild, is likely a direct contributor to social problems.

As summarized in a review of behavioral and neuroimaging studies of FER in ASD by Harms, Martin, and Wallace (2010), the inconsistency in research findings on FER among intellectually able individuals with ASD is due, in part, to differences in how FER is measured. Inconsistency in measurement also has likely affected the extant intervention research. Published studies of FER in ASD have yielded inconsistent findings regarding responsiveness to interventions targeting FER among individuals with ASD. A recent review, for instance, summarized that emotion recognition training is promising, but there is little data on generalizability of intervention effects (Berggren et al., 2018). In a systematic review of the effectiveness of technology-based interventions in improving FER in individuals with ASD, Lee and colleagues (2018) highlighted the difficulty in drawing conclusions regarding effectiveness of the intervention programs when outcome measures vary considerably across studies. Given the implications of FER impairments, it is important to identify valid measures of FER that are sensitive to change. The primary aim of this systematic review is to identify and summarize commonly used FER assessment tools in outcome research in individuals with ASD, with the goal of informing future research. Because sensitivity of a measure is inextricably linked to the effect of the intervention that measure is being used to gauge, we make an assumption that the interventions studied are roughly equivalent in terms of potency, in order to make comparisons of sensitivity across the measures. However, we fully acknowledge that this is a circular argument, as degree of (non)equivalence is not known, in part due to use of different FER measures.

Methods

Inclusion Criteria

To be included in the review, the articles had to be published in scholarly, peer-review journals and written in English. The study had to measure change in FER ability in individuals with ASD, specifically assessing FER as an outcome variable. Only data-based papers were included; as such, reviews, commentaries, and conference papers were excluded. In addition, in order to determine which FER measures are most sensitive to change regardless of the intervention provided, studies that utilized the same measure (i.e., protocol or stimuli) to conduct the intervention as well as to assess FER outcome were excluded. No participant age restrictions were implemented.

Search Methods

We conducted an electronic database search of PubMed and Web of Science to identify relevant, peer-reviewed articles, published between January 1998 and November 2019. Search keywords included: [ASD, AUTISM, or ASPERGER] and [EMOTION RECOGNITION, FAC* RECOGNITION, FAC* AFFECT] and [FAC* EXPRESSION]. One of the authors screened all titles and abstracts to exclude clearly irrelevant articles and reviewed full papers of the remaining articles to determine which articles met all inclusion criteria, including use of an intervention. Another author reviewed full papers of all of the included articles and 10% of the excluded articles in order to confirm eligibility or exclusion. When the first reviewer was unsure of eligibility (n = 14 articles), the secondary reviewer assigned a rating as well. When discrepancy occurred, a third rater determined the final eligibility for inclusion. After the database search, references of all included articles were reviewed for articles that were not identified in the initial database search (cf, snowball search; Greenhalgh and Peacock, 2005).

Variable Definitions and Coding

All included articles were coded for measures used to assess change in FER (Table 1). For study-specific FER measures with no given name, we provided a descriptive label. A brief description of the measure, including stimuli-specific factors such as color vs. black & white, static vs. dynamic, adult vs. child, and whole face vs. partial face, is then provided. Third, we indicated which emotions were conveyed. Fourth, we cited the studies that utilized each FER measure. In the case of multiple articles by the same author or author group and with the same sample, only the article with the largest sample size was reported to avoid confounding the findings. Fifth, we summarized findings in terms of change in FER from pre-intervention to post-intervention. When available, effect size was included. When not reported, Cohen’s d was calculated with the following formula: [(M post - M pre) / SD pre], when the requisite data were provided. Given differences in study design and statistical power across the studies, we emphasized indices of clinical significance and effect size over statistical significance. We also included a brief description of treatment intensity (i.e., number of sessions or duration of program) for each intervention. Finally, we reported participant characteristics for each study, including sample size, age, and Intellectual Quotient (IQ) (when provided) for all participants that were administered the FER measure, irrespective of condition, diagnosis, or group assignment.

Table 1.

Measures of Change in FER in Individuals with ASD

Level 1 Measures	Description	Emotions Conveyed	Study	Evidence of Change	Treatment Intensity	Participants
Diagnostic Analysis of Nonverbal Accuracy (DANVA; Nowicki & Duke, 1994; DANVA2; Nowicki 1997)	Colored static photos of adult and child facial expressions	A, F, H, S	Anagnostou et al., 2012	No significant change in performance [d = 0.33 NS]	2x/day for 6 weeks	N = 19 Age: 33.2 (13.3) IQ: 107 (24)
			Baghdadli et al., 2013	No significant between group difference post tx and no significant change in total score between baseline and post tx. For anger, adult expression only, less errors after tx; post-tx median gain was significantly different between the two groups [d = −0.8]	90 min/week for 20 sessions	N = 13 Age: 8–12 VIQ: 92.7 (24.2), 97.6 (20.3)
			Barnhill, Tapscott Cook, Tebbenkamp, & Smith Myles, 2002	No statistically significant change between pre- and posttest measures on the Child or Adult Faces [Child d = −0.40 NS; Adult d = −0.46 NS]	1hr/week for 8 weeks	N = 8 Age: 13–18 IQ: NA
			Guastella et al., 2015	No significant interactions observed between drug group and time [Post d = 0.21 NS; Follow-up d = 0.14 NS]	2x/day for 8 weeks	N = 44 Age: 12–18 IQ: 80.0 (19.2)*, 93.1 (21.1)
			Guli, Semrud-Clikeman, Lerner, & Britton, 2013	No significant effects for between-group comparisons in change over time [R2 = 0.001 NS]	1.5hr/session 2x/week for 8 weeks or 2hr/session 1x/week for 12 weeks	N = 31 Age: 8–14 IQ: 104.17 (15.47), 107.50 (14.04)
			Lerner, Mikami, & Levine, 2011	Intervention group did not demonstrate significant improvement relative to the comparison group on child or adult faces [all R2 < 0.021 NS]	145 hrs over 29 sessions	N = 17 Age: 11–17 IQ: NA (ID excluded)
			Lopata, Thomeer, Volker, Nida, & Lee, 2008	No significant main effect for either the child or the adult faces and no time by tx condition interaction [Child d = −0.19 NS; Adult d = −0.15 NS]	6 hr/day, 5days/week for 6 weeks	N = 36 Age: 6–13 IQ: 99.1 (15.7)
			Lopata et al., 2010	No significant effect after application of Bonferroni correction for multiple comparisons, although obtained effect size suggested a possible medium effect favoring the tx group [d = 0.53 NS]	5 days/week for 5 weeks	N = 36 Age: 7–12 IQ:103.0 (14.0)
			Lopata et al., 2012	Significant pre-post difference for child faces but not for adult faces [Child: d = 1.27; Adult d = 0.45 NS]	3 weeks, 5x/week preparation followed by 10 months	N = 12 Age: 6–9 IQ: 102.3 (16.0)
			Lopata et al., 2013	Significant pre-post change for child faces and adult face [child d = 0.58; adult d = 0.78]	3 weeks, 5x/week preparation followed by 10 months	N = 12 Age: 6–9 IQ: 104.0 (16.6)
			Lopata et al., 2015a	No significant time by tx (High vs. Low intensity) interaction and no main effect for child faces. However, significant main effect for time for adult faces [Adult faces: ω2 = 0.17; Child faces: ω2 = 0.05 NS]	5 days/week for 5 weeks	N = 47 Age; 7–12 IQ: 103.9 (15.7), 109.0 (13.9)
			Lopata et al., 2015b	No significant pre–post change for the Child Faces [d = −0.18 NS]	5 days/week for 5 weeks	N = 28 Age: 7–10 IQ: 105.8 (16.3)
			Richard, More, & Joy, 2015	No significant group effect at post-test for Child Faces. From pre to post, tx group had trend toward greater improvement than control group, but not significant [tx: d = 0.35 NS; control: d = 0.07 NS]	1 session	N = 19 Age: 8–14 IQ: NA
			Russo-Ponsaran et al., 2014	No significant pre-post change as only one out of the three participants improved score [d = 0.3 NS]	1 hr, 2x/week for 8 weeks	N = 3 Age: 9–11 IQ: 109.3 (NA)
			Russo-Ponsaran et al., 2016	Tx group showed improved performance on Child Faces test [post d = 0.52, maintenance d = 0.54]	1 hr/session, 2x per week for 8 weeks	N = 25 Age: 8–15 IQ: 98.7 (22.2), 106.6 (18.9)
			Schmidt et al., 2011	Significant pre-post improvement [d = NA]	20hr total: 2x/week for 10 weeks	N = 6 Age: 12–13 IQ: 103.3 (15.9)
			Solomon et al., 2004	Significant interaction of group and time for Adult Faces and Child Faces with tx group scoring higher at post-test [Adult d = 1.09; Child d = 1.07]	1.5hr session, 1x/week for 20 weeks	N = 18 Age; 8–12 IQ: 75–143
			Stichter et al., 2010	Significant pre-post improvement (improvement translates to one additional correct response from pre to post-intervention) [d = 0.55]	20hrs total: 2x/week for 10 weeks	N = 27 Age: 11–14 IQ: 103.8 (17.0)
			Stichter et al., 2012	No significant improvement in ability to correctly identify the emotional state [d = 0.07 NS]	20hrs total: 2x/week for 10 weeks	N = 20 Age: 6–10 IQ: 99.3 (15.2)
			Stichter et al., 2014	No significant pre-post difference on the Child Faces [d = 0.33 NS]	31 45-min lessons over 5 units	N = 11 Age: 11–14 IQ: 99.6 (16.8)
			Thomeer et al., 2012	No significant group difference at post-assessment controlling for pretest scores [d = 0.26 NS]	5days/week for 5 weeks	N = 35 Age: 7–12 IQ: 103.8 (13.5)
Reading the Mind in the Eyes Test, Revised (RMET; Baron-Cohen et al., 2001)	Black and white pictures of eye region of a face	Complex emotions including cautious, confident, doubtful, nervous, playful, and skeptical, among others	Anagnostou et al., 2012	Significant improvement after 6 weeks [d = 1.2]	2x/day for 6 weeks	N = 19 Age: 33.2 (13.3) IQ: 107 (24)
			Friedrich et al., 2015	Significant pre-post improvement [η2= 0.4]	1h, 2–3 x/week for 6–10 weeks (average 8 weeks)	N = 13 Age: 6–17 IQ: 93.3 (22.6), 91.3 (30.1)*
			Golan & Baron-Cohen, 2006	Ex1: No significant time x group interactions [Intervention: d = 0.10 NS; Control: d = −0.13 NS ] Ex 2: Significant time x group interaction [Software and Tutor d = 0.35; Social Skills d = −0.41]	2hr/week for 10–15 weeks	Ex1: N=41 Age: 17–52 VIQ: 108.3 (13.3), 109.7 (10.0) Ex2: N=26 Age: 17–50 VIQ: 105.7 (16.1), 96.5 (15.5)
			Guastella et al., 2010	In comparison with performance under placebo, 60% of participants in oxytocin group improved [d = NA]	1 dose	N = 16 Age:12–19 IQ: NA
			Guastella et al., 2015	No significant interactions between drug group and time [post d = 0.22 NS; follow-up d = 0.15 NS]	2x/day for 8 weeks	N = 41 Age: 12–18 IQ: 80.0 (19.2)*, 93.1 (21.1)
			Kandalaft et al., 2013	No significant pre-post change [d = 0.31 NS]	10 sessions over 5 weeks	N = 8 Age: 18–26 IQ: 111.9 (8.5)
			Quintana et al., 2017	No significant effect of treatment or time on performance [d = NA]	Two doses	N = 17 Age: 19–35 IQ: 109.8 (12.1)
Reading the Mind in the Eyes Test – Child (RMET-C; Baron-Cohen et al., 2001b)	28 black and white images of eyes depicting emotion states, with forced choice between 4 mental-state terms for each	Affective and cognitive mental states including H, Sc, S, worried, friendly, interested, and serious, among others	Guastella et al., 2015	No significant interactions between drug group and time [post d = 0.21; follow-up d = 0.33 NS]	2x/day for 8 weeks	N = 41 Age: 12–18 IQ: 80.0 (19.2)*, 93.1 (21.1)
			Schmidt et al., 2011	No significant pre-post assessment change [d = NA]	20hr total: 2x/week for 10 weeks	N = 6 Age: 12–13 IQ: 103.3 (15.9)
			Stichter et al., 2010	Significant pre-post improvement (improvement translates to one additional correct response from pre to post-intervention) [d = 0.35]	20hrs total: 2x/week for 10 weeks	N = 27 Age: 10–14 IQ: 103.8 (17.0)
			Stichter et al., 2012	No significant improvement in ability to correctly label someone’s emotional or mental state [d= 0.13 NS]	20hrs total: 2x/week for 10 weeks	N = 20 Age: 6–10 IQ: 99.3 (15.2)
			Stichter et al., 2014	No significant pre-post difference [d = 0.02 NS]	31, 30–45 min lessons	N = 11 Age: 11–14 IQ: 99.6 (16.8)
Cambridge Mindreading Face-Voice Battery (CAM; Golan, Baron-Cohen, & Hill, 2006)	Colored video clips of male and female adults	20 complex emotions including appalled, appealing, grave, insincere, resentful, stern, uneasy, and vibrant, among others	Golan & Baron-Cohen, 2006	Ex 1: Significant time x condition (intervention, control) effect, with greater improvement for the intervention condition [Intervention d = 0.70; Control d = 0.27] Ex 2: Significant improvement in Software and Tutor but not social skills group [Software and Tutor d = 0.47, Social Skills d = 0.26 NS]	2hr/week for 10–15 weeks	Ex1: N=41 Age: 17–52 VIQ: 108.3 (13.3) 109.7 (10.0) Ex2: N=26 Age: 17–50 VIQ: 105.7 (16.1), 96.5 (15.5)
Cambridge Mindreading Face-Voice Battery for Children (CAM-C Golan, Sinai-Gavrilov, & Baron-Cohen, 2015)	Colored video clips of male and female children and adults	A, D, F, H, S, Su, loving, embarrassed, undecided, unfriendly, bothered, nervous, disappointed, amused, and jealous	Lacava et al., 2007	Significant pre-post scores for the CAM-C Faces subtest [d = 0.76]	10 weeks; average 10hr/week	N = 8 Age: 8–11 IQ: NA
			Lacava et al., 2010	All participants improved ER tests scores from pre- to post-testing [d = NA]	7–10 weeks; average 12.3 hrs total	N = 4 Age: 7–9 IQ: NA
			Lopata et al., 2012	Significant pre-post improvement for face and voice composite score [d = 1.64]	3 weeks followed by 10 months	N = 12 Age: 6–9 IQ: 102.3 (16.0)
			Lopata et al., 2013	Significant pre-post improvement for face and voice composite score [d = 0.94]	10 months	N = 12 Age: 6–9 IQ: 104.0 (16.6)
			Lopata et al., 2017	Significant pre-post change for faces [d = 0.25]	two 90min sessions/ week for 18 weeks	N = 44 Age: 7–12 IQ:108.6 (14.5)
			Lopata, et al., 2016	Significant time by treatment condition interaction observed favoring the augmented group over regular group [ω2 = 0.19]	5 days/week for 5 weeks	N = 36 Age: 7–12 IQ: 105.0 (13.3), 106.3 (13.3)
			Lopata et al., 2019	Significant treatment effect with significantly greater increase for tx compared to control group [d = 1.41]	160–210 mins/week for school year	N = 102 Age: 6–12 IQ: 103.8 (12.9), 100.9 (14.8)
			Thomeer et al., 2015	Significant group difference at both posttest and follow-up [between group: ω2 = 0.23; posttest d = 1.34; follow-up d = 0.86]	two 90min sessions/week for 12 weeks	N= 43 Age: 7–12 IQ:102.6 (13.2), 101.6 (15.3)
Affect Recognition subtest of NEPSY-II (Korkman, Kirk, & Kemp, 2007)	Colored photographs of children’s faces	A, D, F, H, N, S	Corbett et al., 2011	No significant pre-post change [d = 0.62 NS]	2 hrs/day, 1–4 days/week, for 3 months	N = 8 Age: 6–17 IQ: 82.4 (16.4)*
			Corbett et al., 2014	No significant pre-post change [d = −0.18 NS]	4 hrs/day, 5 days a week for 2 weeks	N = 12 Age: 8–17 IQ: 74–118
			Didehbani et al., 2016	Significant pre-post change for total sample but no significant difference between ADHD+ASD combined compared to ASD only group [d = 0.58]	10, 1hr sessions. over 5 weeks	N = 30 Age: 7–16 IQ: 112.6 (12.1)
			Lordo et al., 2017	No significant pre-post difference in ASD group [d = 0.21 NS]	90min/session, 1 session/ week for 14 weeks	N = 29 Age: 12–17 IQ: 95.6 (14.5), 103.5 (10.8)
			Rice et al., 2015	Significant difference in post-test score between experimental and control groups, controlling for pre-test score [η2 = 0.42]	One 25min session/week for 10 weeks	N = 31 Age: 5–11 IQ: 101 (14.5)
			Russo-Ponsaran et al., 2016	Significant difference at post and follow-up between tx and control groups; no significant group by time interaction [post d = 0.40; follow-up d = 0.70]	45–60 min, 2x/week for ~6 sessions	N = 25 Age: 8–15 IQ: 98.7 (22.2), 106.6 (18.9)
			Voss et al., 2019	No significant pre-post change but larger positive mean change in tx compared to control participants [d = NA]	4, 20min sessions, for 6 weeks	N = 71 Age: 6–12 IQ: 77.8 (21.5)*, 75.7 (20.7)*
			Wieckowski & White, 2019	No significant pre-post change [r = .04]	10 sessions total: 2x/week for 5 weeks	N = 8 Age: 9–12 IQ: 110.9 (13.6)
			Williams et al., 2012	No significant intervention by time effects [Intervention group d-post = 0.52 NS]	15min/day for 4 weeks	N = 55 Age: 4–7 IQ: 77.9 (14.0)*, 74.6 (13.6)*
			Young & Posselt, 2012	Significant interaction between time and intervention type; tx group improved significantly pre-post [interaction: partial η2 = 0.53; tx: d = 1.70]	3, 5–10 min episodes a day for 3 weeks	N = 25 Age: 4–8 IQ: NA
Penn Emotion Recognition Test (Gur et al., 2002; Kohler et al., 2003)	Colored images of male and female facial expressions	A, D, F, H, N, S	Eack et al., 2013	No significant pre-post change on 40 stimuli subset, although trend-level effect in emotion perception due to improvement in accuracy of sad faces [Overall: d = 0.24 NS; Sad: d = 0.61]	60 hr computer; 45, 1.5hr group sessions over 18 months	N = 14 Age: 18–45 IQ: 117.7 (16.8)
			Lee, Kang, Kim, & Kwak, 2018	No significant pre-post change on 40 stimuli subset for ASD or ADHD group. After controlling for variables, ADHD group showed more improvement compared to ASD group [ASD pre-post d = −1.00 NS].	1.5 hr/session, 1 session per week, 24 sessions total	N = 23 Age: 7–10 IQ: 82.4 (21.0)
			Mehling et al., 2017	No significant pre-post change [effect size r = 0.23]	1hr/week for 10 weeks	N = 14 Age: 10–13 IQ: NA
Situation-Facial Expression Matching (Golan et al., 2010)	Colored photo depicting a scene without facial expression and below, 3 video clips of character’s facial expressions	A, D, F, H, S, Su, excited, tired, unfriendly, kind, sorry, proud, jealous, joking, ashamed, worried	Golan et al., 2010	Significant pre-post improvement in treatment group on all tasks [Group x time partial η2 = 0.45–0.56]	at least 3 episodes/ day for 4 weeks	N = 56 Age: 4–8 VIQ: 76–116
			Gev et al., 2017	Significant time x treatment condition interaction and pre-post change; significant Time x series interaction [time: η2 = 0.20; series η2 = 0.17; time X series: η2 = 0.24]	10 mins/day for 8 weeks	N = 59 Age: 4–7 IQ: NA
		H, A, F, sorrow, astonishment	Yan, Liu, Ye, & Liu, 2018	Significant main effect of time and interaction of group x time. Only the ASD intervention group showed significant pre-post change. [group: η2 = .09 NS; time η2 = 0.34; group X time: η2 = 0.47]	40mins/day, 5 days/week for 6 weeks	N = 21 Age: 5.59 (0.91) IQ: NA
Level 2 Measures	Description	Emotions Conveyed	Study	Evidence of Change	Treatment Intensity	Participants
Frankfurt Test for Facial Affect Recognition (FEFA; Bölte et al., 2002)	Black and white photograph of eye regions and whole faces	A, D, F, H, N, S, Su	Bölte et al., 2015	Significant improvement for ASD training group after training [d = 0.88]	60 min/week for 8 weeks	N = 57 Age: 14–33 NVIQ: 105.7 (12.0), 109.0 (12.2)
			Bölte et al., 2002	Significant improvement for treatment group only for face and eyes test [d = NA]	2hr/week for 5 weeks	N = 10 Age: 16–40 NVIQ: 58–126*
			Bölte et al., 2006	Significant improvement in trained compared with the untrained sample for both the face and eyes test [face: η2 = 0.59; eyes: η2 = 0.88]	2hr/week for 5 weeks	N = 10 Age: 29.4 (5.9) NVIQ: 94.3 (18.9), 98.6 (19.2)
MiX (Humintell ©; Matsumoto & Hwang, 2011)	Colored dynamic videos of adult faces	A, D, F, S, Su, joy, contempt	Russo-Ponsaran et al., 2016	Significant difference between tx and control group post intervention and follow-up [post d = 2.02; follow-up d = 1.68]	45–60 min, 2x/week for ~6 sessions	N = 25 Age: 8–15 IQ: 98.7 (22.2), 106.6 (18.9)
			Russo-Ponsaran et al., 2014	Significant pre-post change as all three participants increased their performance [d = 4.8]	1 hr, 2x/week for 8 weeks	N = 3 Age: 9–11 IQ: 109.3 (NA)
Comprehensive Affective Testing System (CATS; Weiner et al., 2006) – Name Affect and Three-Faces subtests	Black and white photographs of adult faces expressing different emotions	A, D, F, H, N, S, Su	Russo-Ponsaran et al., 2016	No significant pre-post change on subtests, but small to large effects [0.38 $\le$ dpost ≤ 0.80]	45–60 min, 2x/week for ~6 sessions	N = 25 Age: 8–15 IQ: 98.7 (22.2), 106.6 (18.9)
			Russo-Ponsaran et al., 2014	All three participants increased their performance on both subtests [Name Affect: d = 5.1; Three Faces d = 1.5]	1 hr, 2x/week for 8 weeks	N = 3 Age: 9–11 IQ: 109.3 (NA)
Emotion Recognition and Display Survey (ERDS; Thomeer et al., 2011)	Rating scale that assesses ability of children to recognize and display emotions	35 basic and complex emotions (e.g., H, S, silly, upset, tired)	Lopata et al., 2016	No significant time x treatment condition interactions, although main effect of time was significant for parent and clinician ratings [parent ω2 = 0.39; clinician ω2 = 0.37]	5 days/week for 5 weeks	N = 36 Age: 7–12 IQ: 105.0 (13.3), 106.3 (13.3)
			Thomeer et al., 2015	Significant between-group difference at follow-up, but not at post [between group ω2 = 0.08; post-test d = 0.46 NS; follow-up d = 0.73]	two 90min sessions/week for 12 weeks	N= 43 Age: 7–12 IQ:102.6 (13.2), 101.6 (15.3)
			Thomeer et al., 2011	Significant pre-post change in parent rating [d = 0.95]	90min/session, 12 sessions over 6 weeks	N = 11 Age: 7–12 IQ: 101.3 (17.4)
Facial Expressions of Emotion Stimuli and Tests (Ekman60; Young et al., 2002)	Black and white pictures of adults	A, D, F, H, S, Su	Didehbani et al., 2016	No significant pre-post change, or between group (ASD only and ASD+ADHD) difference at post [d = 0.29 NS]	10, 1hr sessions. over 5 weeks	N = 30 Age: 7–16 IQ: 112.6 (12.1)
			Kandalaft et al., 2013	Significant improvement following treatment [d = 0.44]	10 sessions over 5 weeks	N = 8 Age: 18–26 IQ: 111.9 (8.5)
Level 3 Measures Ekman & Friesen (1976) Stimuli Adaptations	Description	Emotions Conveyed	Study	Evidence of Change	Treatment Intensity	Participants
Modified version of Ekman Pictures of Facial Affect Series (Ekman, 1993)	Male and female faces of emotions	A, D, F, H, N, S, Su	Beadle-Brown et al., 2017	Significant improvement pre to follow-up in number of emotions correctly identified; trend was apparent pre-post, but not significant [follow-up d = 2.12; post: d = 0.82 NS]	45min/week for 10 weeks	N = 22 Age: 7–12 IQ: 29–87*
Photographs of extracted eye and mouth region (from Eckman & Friesen (1976) set of faces)	Black and white photos of eye and mouth region followed by correct or incorrect emotional label	A, D, F, H, S, Su	Domes et al., 2014	Significant drug-by-group interaction, with stronger effects of oxytocin for ASD group compared with controls; significant effect of oxytocin for emotion recognition from the eyes in participants with autism only [d = 0.75]	1 dose	N = 28 Age: 24.0 (6.9) 23.6 (5.4) IQ: 122.4 (24.1), 125.6 (15.4)
Modifications of NimStim Emotional Face Stimuli (Tottenham et al., 2009) presented alongside Ekman Stimuli (Ekman & Friesen, 1976)	Split-screen presentation with still images at 40% intensity from Nimstim stimuli on one side and images from Ekman stimuli on the other side	A, F, H, N	Hadjikhani et al., 2015	Significant improvement for bumetanide treatment group in accuracy in emotion matching [d = 0.59]	1mg/day for 10 months	N = 7 Age: 14–28 PIQ:104.4 (14.5)
Ekman and Friesen’s (1975) photos and Schematic drawings	Black and white photographs of woman’s facial expressions and schematic drawings designed to depict emotions	A, D, F, H, S, Su	Hopkins et al., 2011	Significant difference between tx group and controls for photos and drawing together, and for photos only for both low functioning (LF) and high functioning (HF) children. However, significant difference was found for schematic drawing only for HF children following intervention [LF total d = 0.44; LF picture d = 0.53, LF drawing d = 0.26; HF total d = 0.49, HF picture d = 0.28, HF drawing d = 0.47]	12 sessions total: 10–25min/ session for 6 weeks	N = 49 Age: 6–15 IQ: 75.7 (27.3)*
Picture emotion recognition (Ekman & Friesen, 1976), pictures from Mindreading software library, and schematic cartoon faces (Howlin, Baron-Cohen, & Hadwin, 1999)	Black and white pictures, color pictures, and black and white schematic cartoon faces	Not reported	Lacava et al., 2010	All subjects showed reliable pre-post improvement [d = NA]	7–10 weeks; average 12.3 hrs	N = 4 Age: 7–9 IQ: NA
Emotion Recognition Test (ERT) using photographs from Ekman’s Pictures of Facial Affect (Ekman & Friesen, 1976)	Black and white laminated photographs	A, D, F, H, S, Su	Ryan & Charagáin, 2010	Significant pre-post difference and between group post- scores [training program full sample d = 1.42]	1hr/session, 1session/week for 4 weeks	N = 30 Age: 6–14 IQ: 104.6 (17.4), 98.6 (20.2)
Emotion identification task using pictures of Facial Affect (Ekman & Friesen, 1976)	Black and white photographs of adult models displaying six basic facial expressions	A, D, F, H, S, Su	Williams et al., 2012	Significant effect of intervention by time for the identification of anger only [Intervention group Total d-post = 0.19 NS; Anger d-post = 0.28]	15min/day for 4 weeks	N = 55 Age: 4–7 IQ: 77.9 (14.0)*, 74.6 (13.6)*
Emotion matching task using pictures of Facial Affect (Ekman & Friesen, 1976)	Black and white photographs of adult models displaying six basic facial expressions	A, F, H, S	Williams et al., 2012	Significant effect of intervention by time for the matching of all four expressions of emotion and specifically for the matching of anger [Intervention group Total d-post = 0.18; Anger d-post = 0.40]	15min/day for 4 weeks	N = 55 Age: 4–7 IQ: 77.9 (14.0)*, 74.6 (13.6)*
Level 3 Measures (Other Measures)	Description	Emotions Conveyed	Study	Evidence of Change		Participants
Assessment of Perception of Emotion from Facial Expression (Spence, 1995)	Black and white photographs of facial expressions of four children and adults	A, D, F, H, S, nicely surprised	Beaumont & Sofronoff, 2008	Significant main effect of time but no main effect of group or time by group interaction [time: η2 = 0.31; group: η2 < 0.01 NS; time X group: η2 = 0.05 NS]	2hr/week for 7 weeks	N = 49 Age: 7–11 IQ: 107.2 (11.9), 107.4 (14.2)
Emotion Recognition Test (ERT: Merten, 2005)	Pictures of seven emotions displayed in faces	A, D, F, H, S, Su, contempt	Bölte et al., 2015	Significant improvement for ASD training group after training [d = 0.50]	60 min/week for 8 weeks	N = 57 Age: 14–33 NVIQ: 105.7 (12.0), 109.0 (12.2)
Complex-Emotion Scale based on Facial Affect Scoring Technique, Ekman et al, 1971)	4 complex-emotion pictures and situation-pictures in the form of cards	Not reported	Cheng, Luo, Lin, & Yang, 2018	Significant improvement in tx group at post test compared to control group [Tx d = 5.64]	3 sessions, each lasting 40mins, over 21 day period	N = 24 Age: 9–12 IQ: 75–88
Facial emoticons (Chung et al., 2016)	Facial emoticons of pleasant and unpleasant faces	45 pleasant and 15 unpleasant	Chung, Han, Shin, & Renshaw, 2016	Significant pre-post change in both tx and control groups; no significant difference in the degree of improvement between groups [tx d = 1.02; Control d = 0.74]	1hr/day, 3days/week for 6 weeks	N =20 Age: 13–18 IQ: 80.0 (4.7), 80.4 (8.0)
UNSW Facial Emotion task (Dadds, Hawes, & Merz, 2004)	PowerPoint presentation of facial emotions (2 adult, 2 adolescent, 2 child)	A, D, F, H, N, S	Dadds et al., 2014	Significant pre-post change; no main effect for group (oxytocin, placebo), or time x group interaction [Oxytocin d = 0.31; Placebo d = 0.43]	1 dose/day for 4 days	N = 38 Age: 7–16 IQ: 88.6 (8.0), 90.5 (11.7)
Face task from Mindreading (Baron-Cohen et al., 2004)	Facial expression video clips of male and female actors of various age groups and ethnicities	A, D, F, H, S, Su, interested, bored, excited, worried, disappointed, kind, frustrated, proud, ashamed, joking, unfriendly, hurt	Fridenson-Hayo et al., 2017	Significant time by group interaction. Significant improvement over time was found for intervention group but not control group [Intervention d = 0.66, 0.89; Control d = 0.14, 0.30]	2hrs/week for 8–12 weeks	N = 74 Age: 6–9 IQ: NA (subtest scores are in average range)
Reading the Mind in Films Test-Children’s Version (RMF-C; Golan, Baron-Cohen, & Golan, 2008)	Characters expressing emotions in short social scenes taken from four children’s movies	22 complex emotions including guilty, lonely, upset, mean, caring and excited, among others	Lacava, Golan, Baron-Cohen, & Myles, 2007	No pretest scores presented however no statistically significant difference from Golan’s (2006) groups of children who received no intervention and group of children who received intervention [d = NA]	10 weeks; average 10hr/week	N = 8 Age: 8–11 IQ: NA
Photographs from Japanese facial expressions (JACFEE: Matsumoto & Ekman, 1997)	Color photographs of facial expressions	A, D, F, H, S, Su, contempt	Miyahara, Ruffman, Fujita, Tsujii (2010)	No significant group x time interaction; both time and group main effects were significant [interaction: η2 = 0.24 NS; time η2 = 0.21; group η2 = 0.24]	1 session	N = 43 Age: 17–26 IQ: NA
Emotion Comprehension Test (ECT: Face Task (taken from Warsaw Set of Emotional Facial Expression Pictures (Olszanowski et al., 2015), Picto Task and Situation Task (stimuli modified from Teaching Children with Autism to Mind-Read: A practical Guide (Howlin et al., 1999)	Colored photographs of facial expressions; colored graphic representations of facial expressions; pictures of emotional scenes	H, S, A, F	Petrovska &Trajkovski (2019)	Significant overall main effect for group [η2p = .65] and intellectual functioning [η p2 = .21] on post-tx scores. Main effect for group for post-tx scores for all three tasks [Face task: η2p = .40; Picto task: η2p = .66; Situation task: η2p = .19]	12 hrs over 6 weeks	N = 32 Age: 7–15 IQ: 62.5% diagnosed with mild or moderate ID (IQ 35 to 70)*
Overt emotion sensitivity task with stimuli derived from Karolinska Directed Emotional Faces database	Images of male and female faces of ambiguous emotions. Asked “how happy /angry is this person?”	A, H	Quintana et al., 2017	Significant condition (treatment, control) effect on perception of happiness in ambiguous faces [d = 0.63]	Single dose sessions	N = 17 Age: 19–35 IQ: 109.8 (12.1)
Facial Expression Photographs from Spence (1980)	10 black and white photographs of facial expressions	Not reported	Silver & Oakes, 2001	There was only a time effect as both groups improved their scores over time and the effect of the intervention was not significantly greater [d = NA]	10, 30min sessions, over 2 weeks	N = 22 Age: 12–18 IQ: NA
Emotion Recognition Cartoons (from Teaching Children with Autism to Mind-Read: A practical Guide (Howlin et al., 1999)	Colored cartoons of emotions	8 situation-based, 6 desire-based, and 8 belief-based emotions	Silver & Oakes, 2001	Significant time x group (experimental, control) interaction in number of errors made [d = NA]	10, 30min sessions, over 2 weeks	N = 22 Age: 12–18 IQ: NA
Face Emotion Identification Test (FEIT; Kerr and Neale, 1993)	Black and white photographs of faces expressing basic emotions	A, F, H, S, Su, ashamed	Turner-Brown, Perry, Dichter, Bodfish, & Penn, 2008	Significant main effect of group (treatment, control); main effect for time and group x time interactions were not significant [treatment group d = 0.94]	One, 50min session a week, for 18 weeks	N = 11 Age: 25–55 IQ: 113.3 (20.0), 110.6 (14.7)
Emotion Guessing Game (EGG; Voss et al., 2019)	40 facial expressions expressed by live human actor (5 examples of 8 emotions)	Not reported	Voss et al., 2019	No significant pre-post change but larger positive mean change in tx compared to control participants [d = NA]. Significant gains at 6-week follow-up for tx group [d = NA] however lack of control data.	4, 20min sessions, for 6 weeks	N = 71 Age: 6–12 IQ: 77.8 (21.5)*, 75.7 (20.7)*
Facial expressions from EU-Emotion Stimulus Set (O’Reilly et al., 2016)	12 colored videos of facial expressions; half high intensity and half low intensity	F, A, D, H, S, N	Wieckowski & White, 2019	No significant change from first to last tx session [r = .26]	10 sessions total: 2x/week for 5 weeks	N = 8 Age: 9–12 IQ: 110.9 (13.6)
Video Emotion Recognition Task (Wieckowski & White, 2017)	Colored video of adult expressing emotions	H, S, F, A, Su, D	Wieckowski & White, 2019	No significant pre-post change [r = .18]	10 sessions total: 2x/week for 5 weeks	N = 8 Age: 9–12 IQ: 110.9 (13.6)
Faces Task (Baron-Cohen et al., 1997)	20 black and white photographs of faces showing basic and complex affect	H, S, A, F, Su, D, distressed, scheming, proud, guilty, thoughtful, admiring, bored, quizzical, flirting, interested	Young & Posselt, 2012	Significant group x time interaction; significant pre-post change in tx group [interaction partial η2 = 0.31; tx d = 0.92]	3, 5–10 min episodes a day for 3 weeks	N = 25 Age: 4–8 IQ: NA

Note. Emotions Conveyed: Emotion expressions presented to participants: H = happy, A = angry; S = sad, F = fear, D = disgust, Su = surprised, Sc = scared, N = neutral. Evidence of Change: When available, effect size of change is reported. When data is available, Cohen’s d is calculated with following formula: M post - M pre / SD pre. When effect size unavailable, general description of outcome is provided. Only results related to change due to intervention are provided. NS designates non-significant result. Treatment Intensity: Length and number of treatments as provided in the manuscript. Participants: Total number of participants that were administered the FER measure, irrespective of condition, diagnosis, or group assignment with age range and IQ reported. IQ = Intelligence Quotient, PIQ = Performance Intelligence Quotient, NVIQ = Nonverbal Intelligence Quotient, VIQ = Verbal Intelligence Quotient. When age range was not available, mean and standard deviation (in years) are provided. NA designates studies for which data is not available.

^*Studies that included participants with IQ below 70 are marked with.

Results

Search Results

The search resulted in a total of 1971 articles after removal of duplicates (Figure 1). From this pool, 56 articles were identified, an additional eight articles were included after review of the references, and one article was included after colleague suggestion. Across the 65 included articles, 36 specific measures were identified (Table 1). Drawing from the rubric established for determination of evidence-based treatments (Chambless et al., 1998), evaluation by two or more research teams is considered a more rigorous standard of examination. Therefore, the measures in this review were evaluated and subsequently ranked based on degree of use in the field: measures utilized across more than one study and more than one research team (Level 1), measures utilized across more than one study but only one research team (Level 2), and measures only identified in one study (Level 3).

Figure 1.

Flow chart demonstrating method of study identification and screening.

Level 1 Measures: Measures Utilized Across Multiple Studies and Research Teams

Of the 36 identified measures, only six were used in multiple studies by different research labs. Most of these measures are commonly utilized in research with diverse clinical populations and standardized, with reported psychometric properties.

Diagnostic Analysis of Nonverbal Accuracy (DANVA; Nowicki & Duke, 1994; DANVA2; Nowicki 1997). DANVA2 Child Faces and Adult Faces subtests assess the ability to identify four basic emotions (i.e., happy, sad, angry, fearful) in colored photographs. Stimuli include high and low levels of emotional intensity and both male and female faces. The examinee is asked to view the face and then select which of the four emotions was displayed. The Child Faces subtest includes 24 photographs of child male and female facial expressions while the Adult Faces subtest includes 24 photographs of adult male and female facial expressions. The manual provides normative means and standard deviations for child and adult faces based on a compilation of over 20 studies of typically developing children broken down by age (Nowicki, 2003). For the Child Faces, coefficient alpha ranged from 0.69–0.81 with modal alpha of 0.76 for children aged 4 to 16 years and reported test-retest reliability was 0.74 for 3^rd grade children. For the Adult Faces, coefficient alpha was 0.64–0.90 across studies with typically-developing children through college students, and test-retest reliability was reported to be 0.84 in college students (Nowicki, 2003).

Almost one third of the included studies (n = 21) utilized the DANVA2 to assess change in FER following intervention. Of the 21 studies, only seven reported significant change in FER ability as assessed by the DANVA/DANVA2. These studies utilized a variety of intervention approaches, including a comprehensive school-based intervention (Lopata et al., 2012; Lopata et al., 2013), a comprehensive psychosocial treatment (Lopata et al., 2015a), a modified computerized dynamic facial emotion training tool, the MiX (Russo-Ponsaran, Evans-Smith, Johnson, Russo, & McKown, 2016), a social adjustment enhancement intervention (Solomon, Goodlin-Jones, & Anders, 2004), and a group-based Social Competence Intervention (Stichter et al., 2010; Schmidt, Stichter, Lierheimer, McGhee, & O’Connor, 2011). Of the seven studies, three utilized a control group design (Lopata et al., 2015a; Russo-Ponsaran et al., 2016; Solomon et al., 2004). In addition to these studies that reported statistically significant changes, Baghdadli and colleagues (2013) found that children exhibited fewer errors for recognition of anger (Adult Faces only) following treatment. This effect was not found for the Child Faces or for the other emotions. While these articles reported findings in either child, adult, or both tests, the majority of studies (n = 13, 61.90%) using the DANVA or DANVA2 did not find significant change following the intervention.

Reading the Mind in the Eyes Test, Revised (RMET, Revised; Baron-Cohen, Wheelwright, Hill, Raste, & Plumb, 2001) and Reading the Mind in the Eyes Test, Child Version (Baron-Cohen, Wheelwright, Spong, Scahill, & Lawson, 2001). The Revised version of the RMET contains 36 black and white photographs of the eye region. After viewing each photograph, the examinee is asked to choose the word that most accurately describes the portrayed emotion. Although developed as a test of theory of mind, since it requires participants to interpret complex facial cues in order to infer the emotion, it is also often used as a measure of FER. The child version of RMET contains 28 black and white images of eyes depicting emotion states with forced choice between four mental-state terms for each. The target emotion words and foils are adapted from RMET.

In our review, six studies utilized only the Revised version of the RMET, four studies utilized only the child version of the RMET, and one study utilized both versions of the measure to assess change in FER following an intervention. Support for the ability of the measure to detect change in FER is mixed. Out of the seven studies utilizing the adult version, three indicated no significant change in FER, and four indicated significant improvement. The studies that indicated a significant improvement on the RMET utilized a wide range of interventions, including intranasal oxytocin (Anagnostou et al., 2012; Guastella et al., 2010), neurofeedback training (Friedrich et al., 2015), and an interactive multimedia program called Mind Reading (Golan & Baron-Cohen, 2006). Of these four studies, three utilized a control condition (Anagnostou et al., 2012; Guastella et al., 2010; Golan & Baron-Cohen, 2006). Of the three studies that found no significant improvement on the RMET, two utilized the same or similar interventions to those noted above. Guastella and colleagues (2015) and Quintana and colleagues (2017) found no significant effect of intranasal oxytocin on performance on the RMET. In addition, Golan and Baron-Cohen (2006) found no significant time by group interaction for the RMET in the first experiment, in which a group of adults with ASD used the software at home; in the second experiment, in which individuals used the system at home and also met in a group with tutor, they found that individuals in the treatment group improved on the RMET following the intervention.

Only one of the five studies which utilized the child version of RMET found a significant improvement, using a group-based Social Competence Intervention (Stichter et al., 2010). However, three other studies utilized the Social Competence Intervention and found no significant improvement on the RMET (i.e., Schmidt et al., 2011; Stichter, Laffey, Galyen, & Herzog, 2014; Stichter, O’Connor, Herzog, Lierheimer, & McGhee, 2012).

Cambridge Mindreading Face-Voice Battery (CAM; Golan, Baron-Cohen, & Hill, 2006) and Cambridge Mindreading Face-Voice Battery for Children (CAM-C; Golan, Sinai-Gavrilov, & Baron-Cohen, 2015).

The CAM measures emotion recognition for 20 complex emotions and mental states using faces and voices taken from the Mind Reading: The Interactive Guide to Emotions program (Baron-Cohen, Golan, Wheelwright, & Hill, 2004). The CAM-C measures FER for 15 emotion concepts (six basic and nine complex emotions) using video clips of morphing facial expressions and speech audio clips. While CAM and CAM-C are two different measures that differ in the number and nature of emotions covered and the age group for which measures are used, they share the same structure and nature of the stimuli. For both measures, examinees view or listen to a clip and select one of four emotion words that reflects the emotion expressed by the person in the clip. The CAM-C effectively discriminates between intellectually able children with ASD and typical children, especially when considering the complex emotions (Golan et al., 2015). Test–retest reliability for CAM-C (administrations 10 to 15 weeks apart) was 0.74 (Golan et al., 2015).

Only one study utilized CAM to evaluate change in FER. Golan and Baron Cohen (2006) found significant change in FER following a computerized intervention (i.e., Mind Reading: The Interactive Guide to Emotions), an interactive guide to emotions and mental states in adults with ASD. Eight studies (12.31%) were identified that utilized the CAM-C to evaluate change in FER following an intervention. In all eight studies, the authors found an improvement in FER as measured by CAM-C following the intervention. Only three out of the eight studies utilized a control condition (Lopata et al., 2016; Lopata et al., 2019; Thomeer et al., 2015). However, all but one of these studies evaluated the same computerized intervention (i.e., Mind Reading: The Interactive Guide to Emotions) (Lacava, Golan, Baron-Cohen, & Smith Myles, 2007; Lacava, Rankin, Mahlios, Cook, & Simpson, 2010; Lopata et al., 2012, Lopata et al., 2013; Lopata et al., 2016; Thomeer et al., 2015), with Lopata and colleagues (2019) evaluating Mind Reading as one part of a comprehensive school-based intervention program. The only study that utilized CAM-C outside of the Mind Reading intervention was an open pilot study of a comprehensive outpatient psychological treatment for children with ASD (Lopata et al., 2017), in which they found significant improvement on CAM-C scores following intervention.

Affect Recognition subtest of NEPSY-II (Korkman, Kirk, & Kemp, 2007). The Affect Recognition (AR) subtest assesses discrimination of happy, sad, anger, fear, disgust, and neutral emotions from colored photographs of children’s faces in four different tasks: 1) determining whether two photographs depict the same emotion, 2) selecting two faces with the same affect, 3) selecting a face that depicts the same expression as a face shown, and 4) selecting two photographs that depict the same expression as a previously shown face. Internal consistency has been reported to be 0.87 for children age 7 to 12 years, and test-rest was found to be 0.60 (Korkman et al., 2007).

Ten studies (15.38%) utilized the NEPSY-II AR subtest as an outcome measure. Six studies reported no significant pre to post difference in scores, and four studies reported significant improvement in AR scores following the intervention. The studies that found significant improvement in NEPSY-II AR evaluated different interventions, which included a modified, computerized dynamic facial emotion training tool, the MiX by Humintell © (Russo-Ponsaran et al., 2016), a computer-based social skills intervention called FaceSay (Rice, Wall, Fogel, & Shic, 2015), a home-based intervention using a video, called The Transporters (Young & Posselt, 2012), and Virtual Reality Social Cognition Training (Didehbani, Allen, Kandalaft, Krawczyk, & Chapman, 2016). Three of the four studies that found a significant improvement utilized a control condition (Russo-Ponsaran et al., 2016; Rice et al., 2015; Young & Posselt, 2012). The six studies that found no significant improvement on the NEPSY-II AR task used a variety of intervention approaches, including some previously mentioned interventions (e.g., The Transporters). While Young and Posselt (2012) found significant change utilizing The Transporters, Williams, Gray, and Tonge (2012) found no significant intervention by time effects on the NEPSY-II AR with the same intervention. It is important to note however, that Williams and colleagues utilized a sample of participants with IQ below 70 while Young and Posselt (2012) included only participants with average cognitive abilities.

Penn Emotion Recognition Test (Gur et al., 2002; Kohler et al., 2003). The Penn Emotion Recognition Test is a computer-based test. The stimuli include 96 colored images of faces. Participants are asked to determine and select what emotion the face is showing from the following options: happy, sad, anger, fear, disgust, and no emotion. There are eight low-intensity and eight high-intensity expressions presented for each emotion, as well as 16 neutral expressions. ER40, a subset of the Penn Emotion Recognition Test, includes 40 photographs with examinees choosing one of five options (happiness, sadness, anger, fear, and neutral).

Only three identified studies (4.62%) utilized the Penn Emotion Recognition Task to evaluate change in FER following an intervention. Mehling, Tassé, and Root (2017), utilized the measure to evaluate the Hunter Heartbeat Method (Hunter, 2014), which is a drama-based social skills intervention. The authors reported no significant differences in pre to post intervention scores on the measure; however, half of the participants obtained high scores (> 85%) prior to treatment, indicating minimal room for improvement. Eack and colleagues (2013) utilized ER40 during their initial evaluation of feasibility and initial efficacy of a Cognitive Enhancement Therapy (CET). While the authors did not detect significant pre to post intervention change in overall emotion recognition, they found improvement in accuracy for sad faces. Lee, Kang, Kim, & Kwak (2018) utilized ER40 to investigate the effect of social skills training on facial emotion recognition in children with ASD and Attention Deficit Hyperactivity Disorder (ADHD). Authors indicate no significant change in FER in either group after the training.

Situation-Facial Expression Matching Task (Golan et al., 2010). Created to evaluate the treatment program The Transporters, the Situation-Facial Expression Matching Task consists of three tasks, including a task in which children must match video clips depicting characters’ facial expressions to a photographic scene that excluded character facial expressions (Golan et al., 2010). The three tasks tested three levels of generalization: familiar close generalization (using situations taken from the intervention series), unfamiliar close generalization (matching novel situations with novel expressions from The Transporters characters), and distant generalization (matching novel situations with novel expressions using non-Transporters faces taken from the Mind Reading software; Baron-Cohen et al., 2004).

Only three identified studies (4.62%) utilized this measure to assess change in FER following an intervention. Golan and colleagues (2010) found that the treatment group significantly improved on all three tasks. Similarly, Gev, Rosenan, and Golan (2017) utilized the Situation-Facial Expression Matching task as a FER measure to evaluate The Transporters and found significant time by treatment interaction and significant pre-treatment to post-treatment change. Yan and colleagues (2018) utilized The Transporters to improve FER in Chinese children with ASD. They utilized a subset of five items covering five basic emotions (happiness, anger, sorrow, fear, and astonishment), selected based on frequency of emotional words used by children in China. They found that the intervention significantly improved children’s FER compared to their pre-intervention scores.

Level 2 Measures: Measures Utilized Across Several Studies but Within One Research Team

Five out of the 36 identified measures (13.89%) were utilized in multiple studies. However, they were utilized by the same research team evaluating one intervention, therefore limiting generalizability.

Frankfurt Test for Facial Affect Recognition (FEFA; Bölte et al., 2002). The stimuli include black and white photographs of female and male adult faces expressing emotions. The photographs were taken from the Ekman and Friesen (1978) set. In addition to whole faces, the eye region was cropped and assessed to evaluate a different level of FER. The whole face as well as just the eye region images were then compiled to create a computer-aided program to train and test FER on different levels. Unlike the prior four measures, FEFA was developed for both assessment and intervention. The FEFA has been shown to possess good psychometric properties in a normative sample (internal consistency: 0.91–0.95; retest reliability: 0.89–0.92).

In their initial study describing the development and evaluation of a computer-based treatment program, Bölte and colleagues (2002) found significant improvement on the FEFA for both eyes and whole face following the intervention. Similarly, Bölte and colleagues (2006 and 2015) found significant improvement on the FEFA for training group following administration of the Frankfurt Training for Facial Affect Recognition. While all three of these studies showed significant change in FER utilizing the FEFA measure, all studies were completed by one research group. Moreover, they evaluated change in FER after the same intervention (i.e., Frankfurt Training for Facial Affect Recognition), which utilizes stimuli similar to those used in the FEFA assessment.

Microexpression Recognition Training Tool (MiX; Humintell ©; Matsumoto & Hwang, 2011). The MiX^™ provides FER training and testing of seven emotions (i.e., joy, sadness, anger, fear, surprise, disgust, and contempt). MiX^™ displays dynamic videos of Facial Action Coding System (FACS)¹-coded emotional faces from Ekman and Friesen (1978), which include individuals of varying gender and ethnicity. The MiX includes didactic instruction of facial emotions, in addition to testing modules. Therefore, the MiX was used as both an assessment measure and the intervention in the following two studies. Measures utilizing MiX were included in the review even though MiX utilized the same measure to conduct the intervention as well as to assess FER outcome (an exclusion criterion for this study), because the modules used to test the progress of individuals using MiX were different from the actual training stimuli; it therefore assesses FER independent of what was specifically taught during the intervention. In both a small pilot study (Russo-Ponsaran, Evans-Smith, Johnson, & McKown, 2014) as well as a larger RCT (Russo-Ponsaran et al., 2016), experimenters found statistically significant improvement on the MiX assessment following the intervention. Although both studies showed significant change in FER utilizing the MiX assessment measure, they were completed by one research group and evaluated the change in FER after the same intervention (i.e., coach-assisted MiX).

Comprehensive Affective Testing System (CATS; Weiner, Gregory, Froming, Levy, & Ekman, 2006) – Name Affect and Three Faces subtests. CATS is a commercially available measure of FER utilizing the Ekman and Friesen (1976) stimuli. Prior studies have demonstrated acceptable internal consistency in typically developing children (α = 0.61; McKown, Gumbiner, Russon, & Lipton, 2009; Russo-Ponsaran et al., 2015) and children with ASD (Name Affect α = 0.55, Three Faces α = 0.55; Russo-Ponsaran et al., 2015) for the nonverbal awareness subtests of the CATS. The Name Affect subtest asks participants to label emotions in adult face photographs by choosing one of the feelings provided. The Three Faces subtest also uses adult faces and asks participants to match faces displaying differing intensity levels of the same emotion. The two subtests of the CATS were utilized by Russo-Ponsaran and colleagues (2014, 2016) together with the MiX, DANVA, and NEPSY-II AR to evaluate the coach-assisted MiX program. Similar to the results for the MiX, in a pilot study of three children, all improved on the CATS following the intervention, and the effect was large. However, in the follow-up study, no significant pre to post treatment change on the CATS subtests was found, despite small to large effects (Russo Ponsaran et al., 2016). The inclusion of multiple FER measures in this study makes it possible to compare the ability of the measures to detect change. While participants showed FER improvement on the DANVA, NEPSY-II AR, and MiX, they did not show significant improvement on the CATS subtest, which suggests that the CATS may be less sensitive to FER change with intervention, compared to the other measures.

Emotion Recognition and Display Survey (ERDS; Thomeer et al., 2011). The ERDS is unique in the context of the other instruments included in this review, as it is not a program or stimulus set, but rather a rating scale used to assess the child’s ability to recognize and display emotions. Thomeer and colleagues (2011) developed the 54-item measure for an open-trial pilot of Mind Reading with in vivo rehearsal. The parent provides ratings for the child’s ability to recognize and display 27 emotions, ranging from 1 (almost never) to 5 (almost always). Both basic and complex emotions are included. Thomeer and colleagues reported high internal consistency (0.90) for the Recognition scale for their sample.

In their pilot study (Thomeer et al., 2011) as well as their follow-up RCT (Thomeer et al., 2015) of the Mind Reading program with in vivo rehearsal, Thomeer and colleagues found significant improvement in parent rating on the ERDS. In their RCT evaluating the efficacy of Mind Reading as a component of a comprehensive psychosocial treatment, Lopata et al. (2016) found that, while there was no significant time by treatment condition interactions, there was a main effect of time for parent and clinician ratings, with both groups (i.e., summer treatment, summer treatment with Mind Reading component) improving. While these three studies show ERDS to be a promising measure for evaluating FER change in individuals with ASD, it has been utilized only by one group and only to evaluate programs with the Mind Reading component.

Facial Expressions of Emotion Stimuli and Test (Ekman60; Young, Perrett, Cabler, Sprengelmeyer, & Ekman, 2002). Ekman60 consists of 60 black and white images taken from the Ekman and Friesen (1976) stimuli set. The set consists of ten example photographs of facial expressions for six basic emotions (happy, sad, surprise, anger, fear, and disgust). The faces are presented one at a time for 5 seconds each, followed by a blank screen when a participant is asked to pick the emotion name that best describes the facial expression. Ekman60 has high test-retest reliability rs = 0.77 (Williams, Daley, Burnside, & Hammond-Rowley, 2009).

Didehbani and colleagues (2016) used Ekman60 Faces in addition to the NEPSY-II AR task described above to measure affect recognition in their evaluation of the impact of a Virtual Reality Social Skills Intervention to enhance social skills in children with ASD. While they found significant increases in the NEPSY-II AR task following the intervention, they did not find statistically significant changes on Ekman60. Kandalaft, Didehbani, Krawczyk, and Allen (2013), however, found significant changes on Ekman60 following this intervention, even though they did not find significant differences on the RMET measure described above.

Level 3 Measures: Measures Used in Only One Study

Each of the remaining 25 measures was utilized in a single study. However, many of these studies utilized the same stimuli (i.e., Ekman and Friesen (1976) Picture of Facial Affect stimuli) to create different measures or protocols to assess FER.

Adaptations of the Ekman and Friesen (1976) Stimuli. Pictures of Facial Affect by Ekman and Friesen consist of black and white photographs of facial expressions that have been widely used for the past four decades. The stimuli include males and females displaying the six basic emotions, as well as neutral expressions. Eight measures across seven identified studies utilized these stimuli in different ways. In addition to the Ekman60 Faces described in the prior section, studies utilized the Ekman and Friesen stimuli alongside additional stimuli, such as schematic drawings (Hopkins et al., 2011), color pictures from Mind Reading software and black and white schematic cartoon faces (Lacava et al., 2010), cartoon/emoticon faces (Beadle-Brown, et al., 2017), and NimStim stimuli (Tottenham et al., 2009) presented at the same time on the other side of the screen (Hadjikhani et al., 2015). Ryan and Charragáin (2010) utilized 24 images from the stimuli set that they laminated and presented in a book format. In addition, the stimuli have been used to create Emotion Identification and Emotion Matching tasks (Williams, Gray, & Tonge, 2012), and one study adapted these stimuli by extracting eye and mouth regions from the Ekman stimuli (Domes, Kumbier, Heinrichs, & Herpertz, 2014). While the Ekman and Friesen stimuli have been widely used, no direct comparisons across the actual measures can be made, given task differences.

Other Assessment Measures. The remaining seventeen identified measures, which did not utilize the Ekman and Friesen (1976) stimuli, were only utilized in one study of FER change in individuals with ASD. The studies are briefly discussed in Table 1.

Of note, significant improvement in FER was seen on only seven of the seventeen measures. Three of these measures include static images: ERT utilized by Bölte et al. (2015), the Faces Task developed by Baron-Cohen et al (1997), utilized by Young and Posselt (2012), and Overt Emotion Sensitivity Task with stimuli derived from Karolinska Directed Emotional Faces database, utilized by Quintana and colleagues (2017). Petrovska and Trajkovski (2019) utilized visual material from multiple sources to form Emotion Comprehension Test (ECT), in which examinees evaluated photographs of facial expressions, pictograms, and situation-based emotional scenes in three tasks. The other three measures included colored cartoons of emotions (Emotion Recognition Cartoons utilized by Silver & Oakes, 2001), facial expression video clips (Face task from Mind Reading utilized by Fridenson-Hayo et al., 2017), and complex emotion scale including pictures in the form of cards (Cheng, Luo, Lin, & Yang, 2018). Five studies found a significant main effect of time only when evaluating FER measures (i.e., Beaumont & Sofronoff, 2008; Chung, Han, Shin, & Renshaw, 2016; Dadds et al., 2014; Miyahara, Ruffman, Fujita, & Tsujii, 2010; Silver & Oakes, 2001).

Discussion

FER difficulties have been widely documented in individuals with ASD (e.g., Lozier et al., 2014). Numerous interventions have been designed to specifically address FER difficulties in this population and yet, studies have yielded mixed results. A multitude of assessments are used to examine change in FER with intervention in individuals with ASD. Differences in stimuli (e.g., static versus dynamic), modality (e.g., questionnaire, computer task), and demands (e.g., task duration) of these assessments make it difficult to determine the degree to which FER impairment is, or is not, amenable to therapeutic remediation. Of note, in order to compare sensitivity of the measures across studies, we had to presuppose that the interventions were of approximately equal potency. An assumption of equivalence is almost certainly flawed; as such, results should be interpreted in that context. However, our categorization into three levels based on use by different teams and in multiple studies partially mitigates this concern. For instance, the most widely used measures have been applied across different treatments.

In the context of the rich history of FER intervention research in ASD, results of this systematic review highlight limited agreement with respect to how to assess this process. In the 65 identified articles, 36 different measures were utilized. However, only six measures (i.e., DANVA2, RMET, CAM/-C, NEPSY-II AR, Penn Emotion Recognition Task, Situation-Facial Expression Matching Task) were utilized across study teams. Of note, the Penn Emotion Recognition Task was utilized in only two studies, both of which showed no significant change in FER following the intervention. The Situation-Facial Expression Matching Task was used in three studies all evaluating the same intervention. Research utilizing the other four main measures has found medium to large effects with respect to change in FER (DANVA2: d = 0.52 – 1.27; RMET: d = 0.35 – 1.2; CAM/-C: d = 0.25 – 1.64; NEPSY-II AR: d = 0.40 – 1.70). Even within these measures, however, the results are inconsistent, as some studies find significant effects while others do not. This pattern of mixed results is seen across both within-subject and between-subject (control group) designs. Additional small to medium effects were found for many studies, even when the change was not statistically significant. These results suggest that FER is in fact modifiable in individuals with ASD, and therefore, additional concerted work on measurement in this domain is needed.

There is little evidence pointing to a single robust measure that is sensitive to change when evaluating treatment. The CAM/CAM-C is the only measure for which all nine identified studies found a significant improvement in FER following the intervention, suggesting its potential utility. However, as only four of the studies included a control condition, and all but one evaluated the same computerized intervention in itself or as part of a larger intervention, its utility in judging the impact of treatments and comparing relative efficacy of different interventions is limited. In addition, similar stimuli from the Mindreading intervention program are utilized to measure change in FER, limiting generalizability. The NEPSY-II AR, on the other hand, has been evaluated utilizing various interventions by different research teams. While less than half of the ten identified studies found significant change in FER, the effects were medium to large, and three of these four studies included a control condition. Given the small number of studies utilizing this measure however, further research into its utility is necessary. This highlights the need for additional work in this area, in order to facilitate judgments regarding the impact of treatment or comparisons of relative efficacy.

Twenty-four of the 65 identified articles utilized multiple FER measures to assess change. In eight of these 24 articles, variability was seen across the measures, such that while improvement was seen in one measure, no change was observed on another measure. For example, Anagnostou et al. (2012) found no significant improvement following intervention on the DANVA2 but found significant improvement on the RMET. Guastella and colleagues (2015) report an opposite pattern, with no significant drug group and time interaction for RMET but a significant interaction observed for the DANVA2. Even within the same intervention and measure, inconsistent patterns emerge. For example, while Young and Posselt (2012) found significant change in NEPSY-II AR following The Transporters intervention, Williams and colleagues (2012) found no significant intervention by time effects on the same measure, using the same intervention. It is unclear what factors contribute to these inconsistencies.

The FEFA and MiX are both Level 2 measures that show promise given the significant improvements reported across studies, even though the studies were conducted by the same research team. For both the FEFA and MiX measures, however, the stimuli that are utilized to measure change in FER are similar to those presented to the participants during the intervention phase. These types of measures may detect change in the taught skills; however, generalization to other stimuli, which is the ultimate goal of any intervention program, is not demonstrated.

Overall, the current body of literature suggests that DANVA is the most commonly used measure; however, it includes only static photos of adult and child facial expressions and focuses on only a narrow set of basic emotions. The RMET is also commonly used, however it includes black and white pictures of eye region of a face only, and performance is dependent on age and full and verbal IQ (Peñuelas-Calvo, Sareen, Sevilla-Llewellyn-Jones, & Fernández-Berroca, 2019). For colored photographs of children’s faces specifically, the NEPSY-II AR may be a good option for experimenters. To measure FER dynamically, the CAM-C should be considered as it is the only measure utilizing dynamic, colored video clips that has been used across more than one study and more than one research team. However, in order to assess generalization of the skill above and beyond the immediate content taught in the intervention, a task separate from the one evaluated by the intervention, or a multi-layered assessment, rather than a single measure, may be required.

Limitations

This review should be considered in light of several limitations. Given the inconsistency in what information is presented in the identified articles, there is variability in the level of detail provided for the measures. In addition, given differences in methodologies (e.g., use of control group), data analytic approach, and presentation of results, firm conclusions about “evidence of change” across studies cannot be drawn. A variety of intervention approaches were used as well, and an in-depth examination of the interventions (e.g., dosage, format) is beyond the scope of this review. However, whenever possible, an effect size was reported to allow for comparisons across studies. Evaluating a measure based on sensitivity to change with intervention is complicated; even a sound measure cannot detect change with an ineffective intervention. Additionally, although the list of keywords used to search the databases was constrained to a relatively small number, use of a snowball search approach should have helped mitigate risk of missed studies. Finally, majority of the studies utilized samples of cognitively able participants, limiting generalizability.

Future Directions

There are inconsistencies regarding the effects of intervention targeting FER impairments in ASD. Given the current state of research, it is unclear whether this inconsistency is due to noise in the measurement, methodological differences, or true differences in the intervention impact. In order for the field to advance and more efficiently converge regarding FER intervention efficacy and outcome assessment, researchers are encouraged to utilize existing measures whenever possible, without alteration, in order to make interpretation more straightforward. In addition, future studies should evaluate measures that show promise across studies (i.e., Level 1), such as CAM-C (Golan et al., 2015). Overall, the findings from this systematic review suggest that FER is modifiable, and more concerted work on measurement of FER abilities in individuals with ASD is needed, in order to allow for sensitive evaluation of impact and determination of relative efficacy of different interventions targeting FER in individuals with ASD.