Effectiveness of virtual reality and computerized training programs for enhancing emotion recognition in people with autism spectrum disorder: a systematic review and meta-analysis

Abstract

Background: People with autism spectrum disorder (ASD) have difficulties recognizing emotions. Studies showed that virtual reality (VR) and computerized training programs might be used as potential tools for enhancing emotion recognition in such people. However, some inconsistencies were observed between the studies.

Objective: In the current systematic review and meta-analysis, the potential of computerized and VR training programs were evaluated for enhancing emotion recognition in people with ASD.

Method: Using PRISMA guidelines and a PICO model, eligible studies were retrieved and the pooled effect size was calculated.

Results: This meta-analysis obtained the pooled effect of Cohen's d = 0.69 (95% CI: [0.49, 0.89]) that showed the positive effect of VR and computerized training on emotion recognition in people with ASD. The effectiveness was confirmed for different types of study design, and for both children and adults, while it was larger for non-VR computerized programs compared with VR counterparts. Conclusion–Due to the small sample size of this study and the substantial heterogeneity between studies, the outcomes should be considered with caution in practice. However, these outcomes can be considered for optimizing suitable computerized applications or as the hypothesis for future studies.

Introduction

Emotion is one of the fundamental aspects of human life that has attracted the attention of many researchers (Farashi and Khosrowabadi 2020). It is one of the important research topics regarding the neurodevelopmental disorders. Autism Spectrum Disorder (ASD) is one of neurodevelopmental disorders that its prevalence is increasingly growing. According to the Autism and Developmental Disabilities Monitoring (ADDM), the prevalence was 1 in 60 in 2014, 1 in 54 in 2016 and in 2020, it was 1 in 34 for boys compared to 1 in 145 for girls (Knopf 2020). People with ASD have several difficulties with social and non-verbal communications (Billstedt et al. 2005), facial emotion processing (Loth et al. 2018), which is an important issue for understanding the feelings of others, and social interactions (Knott et al. 2006). Interestingly, emotion processing skills in ASD individuals show enhancement during the time evolution (Gross 2008). This indicates the existence of some brain compensatory mechanisms (Harms et al. 2010) that might be activated by training. For example, using fMRI and a facial affect recognition test, it was shown that the bilateral activation of the amygdala and fusiform in ASD individuals was enhanced after affect processing training, possibly due to neuroplasticity in the social brain in these people (Bölte et al. 2015).

Technological developments in digital media and the accessibility of electronic devices among populations make digital media interfaces interesting for intervention in many types of psychiatry and cognitive interventions, including ASD. Literature indicates that people with ASD are very interested in electronic media and devices (Parsons and Mitchell 2002). A study showed that children with ASD had longer screen time for electronic media and that screen time was correlated with the symptom of ASD severity (Dong et al. 2021). In this regard, several studies have used electronic contents produced by computerized programs to enhance emotion recognition in ASD individuals (Hernandez-Mosti et al. 2018, Kandalaft et al. 2013).

Among different computerized applications developed for people with ASD, virtual reality (VR) applications have received special attention. VR technology refers to computer-generated programs for artificially simulating the real world in which the user can be engaged for interacting with artificial objects. For people with ASD, it was shown that VR-based training induced several brain-behavior changes. For example, Yang et al. showed that following 10 hours of VR training over 5 weeks, the activation of the left inferior frontal gyrus, which is an important unit for socio-emotional processing, decreased and such decreased activation was correlated with enhancement in emotion processing (Yang et al. 2018).

The computerized programs and VR-based interventions may facilitate the learning about emotions since these tools create a motivational environment (Golan et al. 2010) and provide safe and highly controlled interactive scenarios (Ip et al. 2018). Furthermore, these methods offer visual stimuli to the ASD individuals which obtain better learning performance and greater attention and motivation for learning (for example learning emotional states) as compared with non-computerized programs (O’riordan et al. 2001, Moore and Calvert 2000). VR techniques also showed several impacts on the electrophysiological aspects of brain activities. The alpha/beta event-related desynchronization for emotion was affected during VR paradigm (Schubring et al. 2020). Immersing in a VR environment also elicited several types of event-related potentials (ERP) like P200, N200, P300 and so on (Harjunen et al. 2017).

The immersion aspects of the virtual environment contain different levels. It can be low-immersion that is usually produced by 2D tablets, computers or wide screens or can be high-immersion that is usually obtained by head-mounted devices or surround projections (Miller and Bugnariu 2016). Low-immersion virtual environments have lower sensitivity to detect performance differences during emotion recognition tasks (Bekele et al. 2013, Moore et al. 2005). However, such desktop-based technologies make it possible to extract gaze-relevant emotion recognition events (Grynszpan et al. 2012). In large screen low-immersion VR studies, people with ASD showed improvements for labeling, responding and recognizing emotions mimicked by virtual characters (Kandalaft et al. 2013). In general, high-Immersion VR technologies may cause adverse effects such as cybersickness, anxiety or sensory disturbances (Kellmeyer 2018); however, they offer a closer match to the natural environment than a desktop-based virtual environment (Dixon et al. 2020).

Before this, several systematic reviews and meta-analyses were performed and confirmed the effectiveness of VR or computerized training programs for enhancing emotion recognition in ASD individuals (Grossard et al. 2017, Grynszpan et al. 2014, Kouo and Egel 2016, Mesa-Gresa et al. 2018, Ramdoss et al. 2012, Wainer et al. 2017, Karami et al. 2021). The objective of the current study was to update the knowledge regarding the effectiveness of VR and computerized training programs for enhancing emotion recognition in people with ASD since several researches were performed during the last years and it is necessary to update the knowledge base according to the new findings. Furthermore, besides the systematic review of literature, quantitative analyses through the meta-analysis approach were performed. Compared with the previous meta-analysis, since the current study contained more studies and larger sample size, the final conclusion can be considered with a enhanced statistical power. In addition, it was investigated how the effectiveness of computerized/VR training interventions on emotion recognition was affected by different confounding factors including the severity of ASD symptoms (high-functioning ASD vs. other ASD subgroups in the spectrum), type of study design (randomized controlled trial vs. observational studies), age of participants and type of intervention (VR vs. non-VR computerized training programs).

It should be noted that even though VR applications are developed by computers (i.e. VR can be considered as a special kind of computerized programs), there are some differences between VR and non-VR computerized applications. For example, in immersive VR applications, the virtual environment replaces the user's real-world surroundings, while in traditional computerized training programs using computer screens, tablets or 2D screens users can interact with real environment (Radianti et al. 2020). Also, it was proposed that the underlying brain mechanisms and the cognitive load of VR strategies might be significantly different from non-VR applications (Redlinger and Shao 2021). In this regard, even though at the first step the effectiveness of VR and non-VR computerized training programs was investigated altogether; separate analysis was also performed for each of them for investigating their capability for enhancing emotion recognition in ASD individuals.

Material and methods

The current systematic review and meta-analysis was performed according to PRISMA guidelines (Moher et al. 2009). PRISMA statements help researchers to report systematic-reviews and meta-analyses. PRISMA statements highlight the results of each step during the review process (Page et al., 2021) and show the numbers of records identified, included or excluded. The reasons for excluding studies were also included in the diagram (see Figure 1).

Figure 1.

Flow diagram of searching procedure according to PRISMA guidelines.

Search procedure

For performing this systematic review, major databases including PubMed, Scopus and Web of Science were searched using the following search terms: (Autism* OR ‘Autism spectrum disorder’ OR ASD OR autistic OR ‘Asperger syndrome’) AND (‘Virtual reality’ OR VR OR ‘Computerized*’ OR ‘Computer*’ OR ‘Digital training’ OR ‘serious games’ OR ‘video games’ OR ‘computer training’ OR ‘training program’) AND (emotion*). Details for search terms were given in the Appendix.

Inclusion and exclusion criteria

As inclusion and exclusion criteria, (1) only research articles were considered for further analyses and other types of articles such as review papers and protocols were excluded. (2) Studies focused on the application of VR or computerized training programs on emotion recognition in ASD samples were considered and studies focused on other training methodologies or containing samples other than ASD were excluded. (3) In order to prevent language bias, there was no restriction on the language of studies and studies with different languages were included. (4) There was no restriction on the age of participants and related studies with ASD samples in any age spans were included. The type of research articles included randomized controlled trials, non-randomized control trials, clinical trials, case-control and pre-post treatments. In order to be sure that relevant studies were included in this study, the reference lists of review papers, as well as the eligible studies, were checked carefully. Since the main purpose of this work was related to technological advancements like VR and computerized training programs that have been developed mainly during the past two decades, the searching procedure was restricted to 2000 to January 2022. Furthermore, only peer-reviewed published articles were considered. In addition, to reduce the risk of database bias, different databases were used during the searching procedure.

Selection process and data extraction

Two independent authors (S.F and E.J) performed the search procedure. When relevant studies were returned by the search engine in each database, the references were imported to the EndNote reference manager software (version X7). After removing duplicate references, by screening title, abstract and finally full-text, eligible studies were retrieved by each author. Any conflict was resolved through discussion between authors. For data extraction, the PICO (Population, Intervention, Comparison and Outcome) model was used where the population was people with ASD with different degrees of severity of symptoms (i.e. high-functioning or low-functioning ASD and Asperger syndrome) and interventions were VR or computerized training programs. For investigating the effect of VR and computerized training programs on emotion recognition, different measures including standard computerized tests or manual checklists were used. The comparison considered pre-treatment and post-treatment conditions when ASD subjects were exposed to the intervention. In some cases, the change of emotional scores between exposed ASD and unexposed ASD or healthy groups were considered.

For each eligible study, information regarding the name of the first author, date of publication, study design, sample size, information of participants (i.e. age and gender), intervention type, measurement tools and the effect size were retrieved.

Statistical analysis

The effects of VR or computerized training programs on the emotion recognition of ASD individuals were calculated according to standardized mean difference (SMD) and based on Cohen’s d (d) formula (Andrade 2020). For each effect, 95% confidence interval was also calculated. In some limited numbers of studies, the effect size was reported; however, for most studies, the effect size was calculated according to the information embedded in figures or reported statistics such as F-value. All statistical analyses were performed using R software version 4.1.2. Meta-analyses were mainly performed by meta, metaphor and dmetar packages. The heterogeneity between studies was assessed based on I-square (I²) statistic (Higgins and Thompson 2002), Q-test and tau² measure (Viechtbauer 2005). Furthermore, the publication bias was assessed using funnel plot as well as Egger's (Egger et al. 1997) and Begg's (Begg and Mazumdar 1994) tests. The statistical significance was subjected to a significant level of 0.05.

Results

The flow diagram for performing this meta-analysis according to the PRISMA guidelines was depicted in Figure 1. Table 1 summarized the eligible studies included in the current systematic review.

Table 1.

Summary of studies included in the current study. Emotion column refers to emotions that participants are asked to recognize, mimic or describe.

(Kandalaft et al. 2013)

Reference	Designa	ASD Sample Size (gender)	ASD Age	IQ	ASD type	Intervention type	Emotion	Intervention duration	Measure	Main outcome
(Piana et al. 2021)	Pre-Post intervention	10(9m, 1f)	9.3(1.16)	87.2 ± 13.97	ASD	Computerized program	Happiness, Sadness, Anger and Fear	4 to 6 weeks, ten sessions of playing the Guess the Emotion serious game, each session lasting maximum of twenty min	Test of Emotion Comprehension	Increased emotion recognition accuracy after training with the computerized program
(Leung 2021)	Pre-Post intervention	12(9m, 3f)	9.91(2.23)	–	ASD	Computerized Program	Angry, Happy	144 trials of the sequence of 500 ms ISI, a 500 ms fixation cross, followed by the stimulus image for 1500 m.	Correctness, Imitation score	Emotion recognition improvements in the computer-based emotion imitation paradigm were larger in people with higher levels of autistic attributes
(Wedyan et al. 2021)	Pre-Post intervention	15(9m,6f)	5.7(0.28)	125 ± 12.9	ASD	Computerized Program	Anger, disgust, fear, happiness,, sadness, and surprise	15 min two times per week, for 20 weeks	Correct response score
(Lecciso et al. 2021)	Pre-Post intervention	12(12m)	9.33(2.19)	–	ASD	Computerized program	Happiness, sadness, Fear, Anger	–	Facial Emotion Recognition Task and Basic Emotion Production task	Computer based programs promotes facial recognition and expression of basic emotions in children with ASD
(Garcia-Garcia et al. 2021)	Pre-Post intervention	3(NA)	8–10	–	ASD	Computerized program	Happiness, anger, surprise	–	Completion rate, number of errors and spend time for emotion recognition	Computerized programs can be considered as useful tools for teaching emotions to the ASD children.
(Tsai et al. 2020)	Pre-Post intervention	3(3m)	7.73(0.64)	87 ± 13	ASD	VR	Anger, disgust, fear, happiness,sadness, and surprise	5–6 weaks, 20 tests, twice a week, 30–40 min duration for each test	Correct response score	The VR system improved significantly facial expressions.
(Metcalfe et al. 2019)	CC	27(26m, 1f)	10.9(3)	–	ASD	Computerized program	Anger, boredom, disgust, fear, happiness, sadness, surprise and worry	32 timed trials is visible for a maximum of 10 s, with a 1 s break between trials, followed by a fixation cross for half a second in the screen center	Number of errors committed	Situational computerized –based cues can aid emotion recognition ability
(Babu et al. 2018)	CC	8(4m,4f)	14.75 (3.28)	–	High function ASD	VR	Happy, Angry	1–1.25 h	Gaze-related indices correlated with Spence Children‘s Anxiety Scale (SCAS)	Response time was significantly different for ASD after VR angry emotion experiment (p = 0.039).
(Ip et al. 2018)	CC	72(64m, 8f)	9.03	–	ASD	VR	NA	28-session program that lasted for 14 weeks	Test for emotion recognition and Psych educational Profile, Third Edition (PEP-3)	ASD children from VR-exposed group scored higher on emotion expression and regulation after the training
(Yuan and Ip 2018)	CC	36 (31 m,5f)	8.97(1.10)	–	ASD	VR	NA	1 h	PEP-3 affective expressions	VR changed affective expression in ASD individuals significantly (p = 0.037)
(Hernandez-Mosti et al. 2018)	–	5(5m)	12–18	–	ASD	VR	Fear, Joy, Sadness, Anger, Love	–	–	VR as an attractive environment can be used for acquiring information about individual preferences for future interventions
(Yang et al. 2017)	Non-RCT	17(15m,2f)	2.5(3.89)	109.65 ± 13.32	High functioning ASD	VR	NA	Five weeks, two 1-h sessions per week and thus 10 h in total.	Advanced Clinical Solutions for WAIS-IV and WMS-IV Social Perception Subtest (ACS-SP)	Neuroimaging-based predictive biomarkers for treatment effectiveness of VR in adults with ASD were proposed.
(Didehbani et al. 2016)	NA	17(16m,1f)	11.6(2.8)	111.6 ± 10.8	High functioning ASD	VR	NA	10 VR-SCT sessions, 2 per week, 1 h each with a peer for across 5 weeks	NEPSY-II Affect recognition and Ekman 60	Improvements on measures of emotion recognition, social attribution, and executive function (p = 0.001) were observed.
(Chen et al. 2016)	Pre-post intervention	6(5m,1f)	11.53(NA)	103.66 ± 9.29	ASD	Augmented reality	Joy, surprise, contempt, sadness, fear, disgust, and anger	Augmented reality, emotional content videos overlaid on the storybook	Correct assessment ratio	An AR strategy proved useful for teaching ASD for understanding emotions in facial expressions
(Kim et al. 2015)	CC	19 (13 m, 6f)	11.1(2.5)	110.6 ± 15.3	High functioning ASD	Computerized program	Happiness, fear, anger, disgust, sadness, and surprise	VR system for applying forty-eight 10 s trials with different facial expressions	VR emotion sensitivity test (V-REST)	Significant difference between ASD and control was revealed for recognition happy emotion.
(Bekele et al. 2014)	CC	10(NA)	14.7(1.1)	116.5	–	VR	Joy, surprise, contempt, sadness, fear, disgust, and anger	28 randomized trials (each 15–20 s long) to seven emotional expressions (four specified levels of intensity).	Accuracy, response latency, and ratings of response confidence for emotion recognition of labeled avatars	VR can be considered as useful tool for ASD intervention
(Maskey et al. 2014)	Non-RCT	9(9m)	11.2(2)	–	ASD	VR	Phobia/Fear	Four 20–30 min sessions	Spence Children’s Anxiety Scale-parent version (SCAS-P) and child version (SCAS-C)	VR is highly effective treatment for specific phobia/fear for some ASD
(Serret et al. 2014)	Non-RCT	33(31m,2f)	11.4(3.16)	70.5 ± 27.6	ASD	Computerized program	Happiness, anger, disgust, fear, sadness, surprise	Two one-hour VR sessions per week over four weeks	Participant self-rating using a computer program	VR is promising approach for emotion recognition for ASD individuals.
	CC	8 (6 m,2f)	21.2 (2.71)	111.88 ± 8.51	High functioning ASD	VR	Happy, sad, fear, surprise, anger, disgust	10 sessions across 5 weeks	Advanced Clinical Solutions for WAIS-IV and WMS-IV Social Perception Subtest, Facial Expressions of Emotion Stimuli and Tests (Ekman 60)	Significant enhancement for social cognitive measures of theory of mind and emotion recognition was obtained following VR training.
(Young and Posselt 2012)	RCT	13 (NA)	4–8	–	ASD	Computerized program	Happy, sad, angry, afraid, surprised, disgusted, distressed, scheming, guilty, thoughtful, admiring, quizzical, flirting, bored, interested, and proud	Participants were asked to view Transporters DVD, 15 episodes on the DVD, each one lasting 5–10 min	NEPSY-II affect recognition	Improvements in emotion recognition were observed based on the content of the game.
(Hopkins et al. 2011)	RCT	13(12m, 1f)	10.05(2.30) 10.05(2.30)	92. 05 ± 18.63	High functioning ASD	Computerized program	Anger, disgust, fear, happiness, sadness, and surprise	Use the FaceSay software twice a week for 10–25 min in each session for 6 weeks .	Child’s emotion recognition score (correct responses)	Both low and high functioning autism children showed improvements for emotion recognition
(Golan et al. 2010)	RCT	20(15m,5f)	5.6(1.00)	–	ASD	Computerized program	Happy, sad, angry, afraid, disgusted, surprised, excited, tired, unfriendly, kind, sorry, proud, jealous, joking	Children were asked to watch at least three episodes per day over a period of 4 weeks.	PowerPoint slide show –based emotion recognition tasks	The computerized game significantly improves emotion recognition in children with ASD
(Cheng et al. 2010)	Pre-post intervention	3(3m)	9(1)	104 ± 13.86	ASD	Computerized program	Empathy	5 months, 40 min sessions, 7–21 days for baseline and 22 days for intervention.	Empathy Rating Scale	Computerized virtual learning had significant and positive effects on participant recognition of empathy.
(Beaumont and Sofronoff 2008)	RCT	26(23m, 3f)	9.64(1.21)	107.15 ± 11.94	Asperger	Computerized program	NA	Seven consecutive weekly sessions with different durations, 5 month follow-up	Emotion Regulation and Social Skills Questionnaire (ERSSQ).	Junior Detective Training Program is effective in enhancing emotional understanding of ASD Childs.
(Bölte et al. 2006)	CC	10(10m)	29.4(5.9)	94.3 ± 18.9	High functioning ASD	Computerized program	Happy, sad, angry, surprised, disgusted, fear-full, and neutral	1 session of virtual face recognition contains 500 emotional face	FEFA a computerized emotion recognition	No significant activation changes in the fusiform region of the brain were observed after training HFA children by a computer-based training program
(Silver and Oakes 2001)	RCT	11(NA)	13.11(0.92)	–	ASD	Computerized program	Angry, afraid, sad, happy	10 daily computer sessions (over 2 to 3 weeks	Emotion Recognition Cartoons	Computer program designed for training emotions reduced significantly the error rate of ASD children.

^a Design: case-control (CC), randomized control trials (RCT), Non-randomized control trial (Non-RCT), VR: virtual reality, HFA: High-functioning autism, LFA: low-functioning autism, ASD: autism spectrum disorder, NA: Not Available.

VR-based training technologies, as well as computerized training programs for ASD individuals, have been considered effective tools during the last two decades. Initial search obtained 1063 studies (PubMed: n = 129, Scopus: n = 605 and Web of Science: n = 329 studies). Furthermore, during the screening procedure and checking the reference lists of eligible studies and related review papers, 28 additional studies were retrieved. After duplicate removal, 773 studies remained. Following the title, abstract and full-text screening, searching procedure retrieved 26 eligible studies regarding the effectiveness of VR and computerized training programs for emotion recognition in ASD individuals (see Table 1). Among these 26 studies, 23 studies had sufficient information for calculating effect size. Since some studies reported several effect sizes (for example (Babu et al. 2018) for different emotions or (Lecciso et al. 2021) for different emotion recognition measures), in total, 34 effect sizes were available for meta-analysis.

The total sample size of this meta-analysis was 521 individuals, including subjects with different types of ASD (i.e. high-functioning autism, low-functioning autism and Asperger syndrome). It should be noted that ∼80% of the sample size were male individuals, a 4:1 ratio which was consistent with the fact that ASD is strongly male-biased (Werling and Geschwind 2013). The studies included ASD individuals of different ages, used non-VR computerized or VR training programs as intervention and used different types of measures for assessing the emotion recognition in ASD individuals. For example, Babu et al. (2018) used gaze-related indices for anxiety recognition which was correlated with Spence Children's Anxiety Scale. It is worth noting that according to the American Psychological Association, anxiety is an emotion characterized by tension or fear. PEP-3 affective expression test was used by Maskey et al. (2014) and Yuan and Ip (2018) for emotion assessment, while the NEPSY-II Affect Recognition test was used by Didehbani et al. (2016) and Young and Posselt (2012). Other studies used computer-based questionnaire programs for emotion assessment (Bölte et al. 2006, Golan et al. 2010, Kandalaft et al. 2013, Kim et al. 2015, Serret et al. 2014, Silver and Oakes 2001).

The forest plot for meta-analysis of the effectiveness of VR or computerized training programs on emotion recognition in ASD individuals was depicted in Figure 2. For this analysis, all eligible studies were included and the pooled effect of SMD = 0.69(95%CI = [0.49,0.89]) was obtained which indicated the effectiveness of computer-based training for enhancing emotion recognition in people with ASD. However, significantly high between-study heterogeneity was observed (I² = 71.0%, p < 0.05, PI = [− 0.25, 1.63], tau² = 0.2). The prediction interval contained zero, that indicated future studies for the effectiveness of computerized or VR training programs on emotion recognition in people with ASD may report contradictory results.

Figure 2.

Forest plot for effectiveness of VR and computerized training programs on emotion recognition in ASD individuals.

There were several possible sources for such observed heterogeneity between studies, including the outlier studies, study design, the severity of ASD symptoms, the type of intervention (VR vs. non-VR interventions) or age of participants. In search of the possible sources for between-study heterogeneity, at first, the test for outlier studies (using find.outliers function included in dmetar R package) was conducted. This test showed that two studies (Serret et al. 2014, Chen et al. 2016) might be potential sources for the observed heterogeneity between studies. The pooled effect and between-study heterogeneity after excluding outlier studies were SMD = 0.54(95%CI = [0.41; 0.66]), I² = 15.4%(95%CI = [0.0%; 45.3%], p = 0.22) and tau² = 0.02(95%CI = [0.00; 0.16]). This result indicated that by excluding outliers, between-study heterogeneity was resolved completely. However, the subgroup analyses were also performed by considering confounding factors such as study design, the severity of ASD symptoms, age span of ASD samples and also the type of intervention (VR vs. non-VR intervention), since such analyses could obtain useful information for the effectiveness of VR and non-VR computerized training programs on emotion recognition in people with ASD.

Figure 3 shows the forest plot for subgroup analysis according to the study design as a confounding factor. There are different strategies for performing a research study. Randomized controlled trials (RCT) are the most reliable form of study due to randomization and the existence of a control group. In case-control studies (CC), the exposed group was compared with an independent unexposed group, while in a pre-post intervention (Pre-Post) scenario there is only one group available. The pooled effect size for RCT studies (Golan et al. 2010, Hopkins et al. 2011, Silver and Oakes 2001, Young and Posselt 2012) that mainly focused on non-VR computerized training programs was SMD = 1.00(95%CI = [0.70;1.30]) which was relatively higher than the pooled effect size obtained by other types of studies. Furthermore, the pooled effect for case-control (CC) type studies and pre-post intervention (Pre-Post) studies was very close (SMD = 0.45 for CC and SMD = 0.43 for Pre-Post). There was no substantial heterogeneity between studies for none of these subgroups (I²<25%, tau²<0.01, p > 0.05). Statistical analysis using one-way ANOVA followed by Tukey HSD post-hoc analysis showed that there was no significant difference (p > 0.05) between effect sizes according to study design.

Figure 3.

Forest plot for the subgroup analysis of the effectiveness of VR and computerized training programs for emotion perception in people with ASD considering the study design. Study design refers to RCT, non-RCT and CC types.

Studies included in this meta-analysis contained samples with different severity of ASD symptoms. In some studies, high-functioning autism (HFA) samples were used as ASD target group (Yang et al. 2017, Bölte et al. 2006, Hopkins et al. 2011, Kandalaft et al. 2013, Kim et al. 2015, Babu et al. 2018, Didehbani et al. 2016). In one study, Asperger syndrome (AS) was considered (Beaumont and Sofronoff 2008), while in the remaining studies, the severity of ASD symptoms was not specified or mixed groups of autism subtypes were used. The subgroup analysis by considering the severity of ASD symptoms (Figure 4) showed that when the analysis was performed for distinct subgroups (i.e. mixed ASD or HFA), the significant large or moderate effect of VR/computerized training on emotion recognition was observed (SMD = 0.66 (95%CI = [0.42; 0.91]) and SMD = 0.49 (95%CI = [0.32, 0.66]), for mixed ASD and HFA subgroups, respectively). This result implied that computerized or VR training programs might have higher impact on more severe ASD subjects since a higher pooled effect was obtained for mixed ASD subgroup (labeled as ASD in Figure 4) that contained low-functioning ASD individuals as well as high-functioning and Asperger cases.

Figure 4.

Forest plot for subgroup analysis of the effectiveness of VR and computerized training programs for emotion recognition in ASD individuals considering the severity of ASD symptom, including HFA, AS or mixed subtypes (i.e. ASD category).

For the HFA subgroup, between-study heterogeneity was not significant (I² = 0.0%, tau² = 0, p = 0.89), however, when different subtypes of ASD were considered in the analysis, between-study heterogeneity was moderate and significant (I² = 49%, tau² = 0.09, p = 0.02). Statistical analysis using student t-test showed that there was no significant difference between HFA and ASD subgroups (t = −1.6, p = 0.06). It should be noted that for this analysis two outlier studies (i.e. Serret et al. 2014, Chen et al. 2016) were excluded. Furthermore, Beaumont and Sofronoff (2008) was the only study for Asperger type of ASD, therefore, it was excluded for subgroup analysis depicted in Figure 4. The prediction interval for ASD subgroup (PI = [−0.06;1.39]) implies that the positive effect of VR/computerized training programs for emotion recognition in the mixed ASD subgroup may be rejected by future studies.

Figure 5 shows the forest plot for subgroup analysis according to the type of intervention. For such analysis, studies were separated into two groups: subgroup 1, studies in which non-VR computerized programs (CP) were used for training ASD individuals and subgroup 2 for studies in which VR technology was used for enhancing emotion recognition in ASD samples. According to the obtained results, even though both technologies improved emotion recognition in people with ASD, the effectiveness of non-VR computerized training programs (i.e. CP category in Figure 4) was higher than VR training strategies (SMD = 0.78, 95%CI = [0.51;1.05] for the former and SMD = 0.42, 95%CI = [0.26;0.56] for the latter). Student t-test revealed that the difference between effect sizes of VR and CP groups was significant (t = 2.77, p = 0.005). However, the heterogeneity between studies for the results obtained for VR technology (I² = 0.0%, tau² = 0) was relatively lower than non-VR computerized training programs (I² = 47.0%, tau² = 0.1).

Figure 5.

Forest plot for subgroup analysis of the effectiveness of VR and computerized training programs for emotion perception in ASD individuals considering intervention type (CP: computerized non-VR programs, VR: virtual reality programs).

In Figure 6, the effect of age on the effectiveness of VR/computerized training programs on emotion recognition in ASD subjects was investigated. Studies were separated into two distinct groups (childhood: age ≤ 12 years; adolescence and adulthood group: age > 12 years). The heterogeneity between both subgroups was small and non-significant (for age ≤ 12 years: I² = 28.0%, p = 0.12, tau² = 0.04; for age > 12 years: I² = 0.0%, p = 0.48, tau² = 0.00). Furthermore, the effectiveness of intervention in both groups was nearly identical and no significant difference was found (p = 0.15).

Figure 6.

Forest plot for the subgroup analysis of the effectiveness of VR and computerized training programs for emotion recognition in ASD individuals considering the age of participants.

The publication bias was assessed using funnel plot (Figure 7) as well as Egger’s and Begg’s tests. The rank correlation Begg’s test showed that there was publication bias for this meta-analysis (z = 3.42, p < 0.05), while Egger’s test showed there was no publication bias between studies (t = 1.69, p = 0.1).

Figure 7.

Funnel plot for publication bias.

Correlation between effectiveness of VR and non-VR computerized training program and age/IQ factors

To have an estimate of the dependence of effectiveness of computerized or VR training programs on emotion recognition to the age or IQ parameters, correlation analysis using Pearson's correlation was performed and the results were shown in Table 2. According to Table 1, 14 studies had information regarding the IQ for ASD participants. After excluding two outliers (Serret et al. 2014, Chen et al. 2016), for the remained 12 studies the correlation between VR/computerized training programs on emotion recognition and IQ was calculated.

Table 2.

Correlation analysis between the effectiveness of computerized/VR training programs on emotion recognition in ASD samples.

	All available studies	HFA subgroup	Mixed ASD subgroup	VR training program	non-VR Computerized training programs
Age	r = −0.052, p = 0.78, n = 32	r = 0.47, p = 0.066, n = 16	r = −0.03, p = 0.92, n = 15	r = 0.05, p = 0.85, n = 17	r = 0.11, p = 0.7, n = 15
IQ	r = 0.176, p = 0.584, n = 12	r = 0.28, p = 0.65, n = 5	–	–	r = 0.29, p = 0.49, n = 8

r indicates the correlation coefficient and n shows the number of included studies for correlation analysis. The dashed line indicated that there was not enough data (n ≤ 3) for calculating a reliable correlation coefficient.

Discussion

Study main achievements

In this study, the effectiveness of VR and computerized training strategies on emotion recognition in ASD individuals was evaluated. The pooled effect showed that these strategies largely influence emotion recognition in ASD individuals (SMD = 0.69). This outcome was consistent with previous systematic reviews (Karami et al. 2021, Mesa-Gresa et al. 2018, Ramdoss et al. 2012, Wainer et al. 2017) and was confirmed for different types of studies (Figure 3) including RCT studies (SMD = 1.00), case-control studies (SMD = 0.45), and Pre-Post intervention-based studies (SMD = 0.43). Such effectiveness may stem from the interest of ASD individuals to be engaged with computerized applications (Tentori et al. 2015), the potential for repetition the of training (Cheng et al. 2010), the potential for training with social cues that are required for emotion processing (Golan and Baron-Cohen 2006), possibly by increasing the attention span (Cunha et al. 2016) or reducing the stress level of participants (Lindner et al. 2019).

According to Figure 4, when intervention was applied to a mixed group of high-functioning and low-functioning autistic samples, the effectiveness of VR/computerized training intervention was relatively higher as compared with high-functioning cases (SMD = 0.66 for a mixed group vs. SMD = 0.49 for the high-functioning group). Previous studies showed that low-functioning ASD individuals had more difficulties for understanding emotions (Mantziou et al. 2015), therefore, higher effectiveness for the mixed group may imply stronger effect of training for low-functioning ASD samples. This needs to be checked in future studies. The result of Figure 4 indicated that when ASD individuals with the same severity symptoms of ASD were considered (i.e. HFA subgroup), between-study heterogeneity was negligible, while with the mixed sample between-study heterogeneity was higher.

Furthermore, the results of the current study proposed that non-VR computerized training programs might be more effective for emotion recognition as compared with VR training (according to Figure 5, SMD = 0.78 and SMD = 0.42 for non-VR computerized and VR training programs, respectively). Several studies reported that the level of anxiety or negative effects experienced by ASD individuals were not significantly different from normal subjects (Malihi et al. 2020, Adjorlu and Serafin 2019); therefore, factors such as stress or anxiety should not be considered significant contributors for such a result. However, other factors such as study design might cause this outcome, since as clear in Figure 5, all RCT studies had fallen in CP subgroup and this possibly increased the pooled effect.

According to Figure 6, when studies with average population age greater than 12 were considered (adolescence and adulthood subgroups), the moderate effect of 0.51 (95% CI: [0.32, 0.72]) was obtained without any significant between-study heterogeneity (I² = 0.0%, p = 0.48, tau² < 0.01). For lower age (childhood, average age ≤ 12) even though the pooled effect of VR and non-VR computerized training programs on emotion recognition was larger (0.57, 95%CI: [0.39, 0.75]), moderate between-study heterogeneity was observed (I² = 28.0%, p = 0.12). This result indicated that computerized training was effective for emotion recognition in both children and adult groups. Previous studies showed that emotion recognition skills for ASD children reach typically developing peers among 8–12 years of old and remain comparable through adolescence. Even though such skills continue to develop in typically developing group, it is not the case for ASD samples (Rump et al. 2009). This might be the reason for the same effectiveness of computerized training of children and adults with ASD.

The correlation analyses (Table 2) showed that only there was a strong and near-significant correlation between effect size and age of high-functioning ASD (HFA) participants when the effectiveness of computerized programs on emotion recognition was considered (r = 0.47, p = 0.066). This result showed that VR/computerized training programs obtained more effects on emotion recognition in older HFA samples. Furthermore, according to Table 2, there was no significant correlation between IQ and the effect of computerized/VR interventions on emotion recognition in people with ASD.

Comparison with previous systematic reviews and meta-analyses

Before this, several systematic reviews were performed for investigating the effect of VR or computerized training strategies on emotion recognition in ASD individuals. Consistent with our result, most of them (including Wainer et al. (4 studies) (Wainer and Ingersoll 2011), Ramdoss et al. (8 studies)(Ramdoss et al. 2012), Mesa-Gresa et al. (8 studies, mainly focused on young ASD subjects) (Mesa-Gresa et al. 2018), Kouo et al. (10 studies) (Kouo and Egel 2016)), without statistical analysis, reported the effectiveness of these technological interventions on emotion recognition in ASD samples. Grossard et al. (2017) reviewed studies regarding state-of-the-art serious games for training emotions to ASD individuals and highlighted the strength and weaknesses of such technologies (Grossard et al. 2017). There were also meta-analysis studies for quantifying the effectiveness of VR or computerized training programs on emotion recognition in ASD samples. Grynszpan et al. (2014) investigated the effectiveness of innovative technologies including VR, computer programs and robotic on ASD individuals (Grynszpan et al. 2014). This study obtained a pooled effect of Cohen’s d = 0.47, without any significant correlation with age and IQ. Karami et al. (2021), by focusing mainly on VR and AR interventions (10 studies), in a systematic review and meta-analysis study reported the pooled effect of Cohen’s d = 0.46 for emotion regulation and recognition skills in ASD individuals (Karami et al. 2021). This value is very near to our estimate of the effectiveness of VR on emotion recognition in people with ASD (see Figure 5, SMD for VR intervention = 0.42).

Possible mechanisms for the observed effectiveness of computerized or VR interventions on emotion recognition in people with ASD

Most of the studies included in the current meta-analysis used VR/computerized training programs using facial avatars for enhancing emotion recognition in ASD individuals. Studies indicated that such training strategies have potential impacts on brain activities and in this way have positive effects on emotion recognition in ASD individuals. Facial emotion processing elicited an N170 event-related potential which its neural generator is localized to the occipitotemporal site (Shibata et al. 2002). In ASD, the amplitude of N170 component following the emotion recognition is shown to be smaller as compared with typically developing individuals (O’Connor et al. 2005). Furthermore, the pooled effect obtained from a meta-analysis showed that N170 latency was delayed in ASD individuals, however confounding factors like cognitive ability should be cautioned for such an outcome (Kang et al. 2018). An ERP-related study showed that face discrimination training for example using a VR training task might cause a reduction of N170 latency at the occipitotemporal area (Su et al. 2012). Reduced N170 latency following facial emotion recognition in a VR-training task might be considered as a compensatory mechanism in ASD individuals. Furthermore, 3D VR training might activate the amygdala (Dores et al. 2014), an important emotion regulation unit that is less activated in ASD individuals as compared with controls (Kleinhans et al. 2009).

Study limitations

Among eligible studies that were retrieved for this study, limited studies had enough information for calculating the relationship between the effectiveness of computerized or VR interventions on emotion recognition and confounding factors such as Autism Diagnostic Observation Schedule (ADOS) or Social Responsiveness Scale (SRS). Furthermore, the duration of intervention might be another important factor which we could not consider in our study since needed information was not available from the retrieved studies. In addition, emotion recognition is a complex process and it can be recognized using several markers such as voice (Lech et al. 2020), facial expression (Song 2021) and physiological signals (Shu et al. 2018). The majority of studies included in the current meta-analysis were mainly focused on emotion recognition and emotional training using facial expression-based tasks. This limited the generalizability of the results. In addition, studies included in this meta-analysis used different measures for emotion assessment which might have some effects on the reported results. The results of this study showed that non-VR training programs might be better suited with ASD samples compared with VR programs. This should be considered as a hypothesis for future studies since we couldn't find strong evidences in the literature for such an outcome.

Conclusion

People with ASD have difficulties with emotion perception and recognition. The effectiveness of training people with ASD using computerized programs has been the subject of many studies during the last decades. A cumulative study, for example a systematic review and meta-analysis that combines the results of several independent studies can obtain a more accurate conclusion for the effectiveness of such interventions. According to the studies analyzed in the current meta-analysis and in line with previous systematic reviews, there were significant evidences for the effectiveness of VR or non-VR computerized training programs for enhancing emotion recognition in ASD individuals. This result was confirmed for different types of study designs (i.e. RCT or non-RCT studies) and for both children and adults. The results proposed that non-VR computerized training might obtain better outcomes, however, due to the small sample size and heterogeneities between studies, this should be carefully tested in future studies. Furthermore, the result of effectiveness was more significant for high-functioning autistic people. Since the majority of studies included in the current research focused on training emotion recognition scenarios according to the face perception, for higher generalizability future researches should focus on other aspects of emotion training scenarios such as emotion recognition training through speech.

Appendix: search details for different databases

PubMed

            (Autism
            *
            [Title/Abstract] OR "Autism spectrum disorder"[Title/Abstract] OR ASD[Title/Abstract] OR autistic[Title/Abstract] OR "Asperger syndrome"[Title/Abstract]) AND ("Virtual reality"[Title/Abstract] OR VR[Title/Abstract] OR "Computerized
            *
            "[Title/Abstract] OR "Computer
            *
            "[Title/Abstract] OR "Digital training"[Title/Abstract] OR "serious games"[Title/Abstract] OR "video games"[Title/Abstract] OR "computer training"[Title/Abstract] OR "training program"[Title/Abstract]) AND (emotion
            *
            [Title/Abstract])
          

Scopus

            TITLE-ABS-KEY(
            *
            Autism OR ("Autism spectrum disorder") OR ASD OR autistic OR ("Asperger syndrome")) AND TITLE-ABS-KEY(("Virtual reality") OR VR OR 
            *
            Computerized OR 
            *
            Computer OR ("Digital training") OR ("serious games") OR ("video games") OR ("computer training") OR ("training program")) AND TITLE-ABS-KEY(
            *
            emotion)
          

Web of science

            TS=(Autism
            *
              OR  ("Autism spectrum disorder")  OR  ASD  OR  autistic  OR  ("Asperger syndrome") )  AND  TS=(("Virtual  reality")  OR  VR  OR  Computerized
            *
              OR  Computer
            *
              OR  ("Digital training")  OR  ("serious games")  OR  ("video games")  OR  ("computer training")  OR  ("training program") )  AND  TS=(emotion
            *
            )
          

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Funding Statement

This work was Funded by Hamadan University of Medical Sciences, Deputy of research and technology (Grant No. 140004293699/IR.UMSHA.REC.1400.085).

Authors’ contributions

S.F. and E.J. performed systematic searches. S.F. performed statistical analyses and wrote the manuscript. S.F., S.B. and K.R. discussed the obtained results.

Disclosure statement

There is nothing to declare.