Virtual/augmented reality for joint attention skills improvement in autism spectrum disorder: a systematic review

Abstract

Deficits in joint attention (JA) are among the earliest hallmarks of autism spectrum disorder (ASD) and are targeted in early intervention programs. Virtual reality (VR) and augmented reality (AR) are emerging technologies that have attracted the interest of scientists conducting research in the ASD field in recent years. Despite the critical role of JA skills in the development of social-communication skills, only a few studies have targeted these skills using VR/AR-based interventions for individuals with autism. This systematic literature review is the first to present the state of the art in clinical applications of VR/AR-based interventions for improving JA skills in individuals with ASD. Seven peer-reviewed articles were analyzed to clarify the experimental effect of VR/AR applications on JA skills and, consequently, on social-communications skills. According to the analysis, positive results have been reported in all but one study. However, it was revealed that there was no consensus on the JA measures employed in the studies, making it difficult to compare results and draw definite conclusions about the clinical benefits of VR/AR. Due to the importance of JA, it is highly recommended that further clinical trials be conducted on the use of VR/AR-based interventions to enrich the literature on this subject.

Introduction

Joint attention (JA), also known as shared attention, is a social communication skill that develops early in life. It occurs when one individual directs the attention of another towards an object or event (stimulus) using cues such as eye-gazing, pointing, or other verbal or non-verbal indications (Bakeman and Adamson 1984). Research has shown that children typically develop the ability to participate in JA by the age of 18 months (Baron-Cohen et al. 1992). JA involves three components: orienting attention, sustaining attention, and shifting attention (Patten and Watson 2011). Orienting attention refers to the ability to coordinate attention towards a stimulus. Sustaining attention is the ability to maintain focus on a stimulus. Shifting attention is the ability to disengage from one stimulus and redirect focus to a new one (Patten and Watson 2011). The ability to orient, sustain, and shift attention towards relevant stimuli is crucial for learning about the world (Meek and Jahromi 2021).

There are two main types of JA that have been widely researched: Responding to Joint Attention (RJA) and Initiating Joint Attention (IJA) (Bruinsma et al. 2004; Meindl and Cannella-Malone 2011; Mundy and Newell 2007). RJA refers to a person’s response to a directional cue given by another individual towards a stimulus (Bruinsma et al. 2004; Meindl and Cannella-Malone 2011). On the other hand, IJA involves an individual initiating a directional cue to direct the attention of another person towards a stimulus (Bruinsma et al. 2004; Meindl and Cannella-Malone 2011).

Research has established JA as an important skill in both typical and atypical development (Moore et al. 2014). People with autism spectrum disorder (ASD) often struggle to participate in JA acts with social partners (Dawson et al. 2004; Meindl and Cannella-Malone 2011; Mundy and Newell 2007; Schertz and Odom 2004). ASD is a lifelong neurodevelopmental disorder characterized by impairments in social communication and interaction, along with restricted and repetitive patterns of behaviour, interests, or activities (American Psychiatric Association, 2013). JA deficits are considered one of the core social-communication deficits and a red flag in ASD. Individuals with ASD may exhibit limited JA skills when they fail to make eye contact, orient towards the relevant stimulus, or follow gestures, signals, or prompts from an adult or peer in a social situation. The lack of JA skills can therefore be a strong indicator of an increased risk for ASD (Baron-Cohen et al. 1997; Mundy 2016) and represents a key deficit employed in the diagnosis of ASD in young children (Baron-Cohen et al. 1992; Osterling and Dawson 1994).

Significant associations have been found between impairment in JA skills in early childhood and limited language abilities in autistic children later on (Charman et al. 2003; Kasari et al. 2008; Loveland and Landry 1986; Mundy et al. 1990). Furthermore, researchers believe that the lack of JA to social stimuli might be one of the reasons for the social and communication barriers of ASD (Charman 2003; Sigman et al. 1999; Toth et al. 2006; Warreyn et al. 2014). Thus, a wide range of behaviour- or developmental-based intervention methods have been suggested to train JA skills in people with ASD (Murza et al. 2016), such as Applied Behaviour Analysis (ABA) (Isaksen and Holth 2009), Pivotal Response Training (PRT) (Ebrahim 2019), Reciprocal Imitation Training (RIT) (Ingersoll and Schreibman 2006), Early Start Denver Model (ESDM) (Rogers et al. 2006), Joint Attention-Mediated Learning (JAML) (Schertz et al. 2013), and Joint Attention, Symbolic Play, Engagement, and Regulation (JASPER) (Kasari et al. 2006) intervention techniques. Though such conventional interventions have been modestly successful in improving JA skills in individuals with ASD, they suffer from some limitations. High costs and the need for long and intensive therapy sessions, which usually require trained therapist, limits the availability of conventional interventions (Murza et al. 2016). In addition, these approaches can be prone to subjectivity (Chasson et al. 2007). Furthermore, there is low flexibility to bring in variations to the task environment and as a result, evidence for generalization to novel settings was scarcely reported.

With the rapid progress of innovative technologies, along with the affinity of many autistic people toward them (Grynszpan et al. 2014), technology-based intervention studies have increasingly drawn the attention of researchers. Although not posited as a replacement for skilled clinical care, technology-based interventions can complement and support conventional interventions. Social robots (Sani-Bozkurt and Bozkus-Genc 2023) and multimedia-based interventions like animated video modelling (Ho et al. 2019), Kinect-based videogame (Crowell et al. 2019), Virtual Reality (VR) (Amat et al. 2021) and Augmented Reality (AR) (Pérez-Fuster et al. 2022) are examples of technology-based interventions employed in studies have focused on enhancing the JA skills of individuals with ASD. Robots are powerful tools, but they are expensive and need expert knowledge to operate and modify their functions, which mostly restricts them to research lab-based settings. Results of a recent systematic review indicate that autistic children positively respond to social robots as social partner; however, more evidence is required to demonstrate the clinical relevance of the JA improvements and generalization (Sani-Bozkurt and Bozkus-Genc 2023).

VR and AR are among the most popular multimedia-based interventions in ASD research (Berenguer et al. 2020; Bradley and Newbutt 2018; Bravou et al. 2022; Dechsling et al. 2022; Farroni et al. 2022; Glaser and Schmidt 2022; Howard and Gutworth 2020; Karami et al. 2021; Lorenzo et al. 2019; Lorenzo et al. 2022; Mak and Zhao 2023; Marto et al. 2019; Mesa-Gresa et al. 2018; Miller et al. 2020; Parsons 2016; Thai and Nathan-Roberts 2018; Zhang et al. 2022b). VR provides sensory experiences in artificial environments through the computer, enabling virtual interactions. VR systems share three main features: immersion, interaction, and a sense of presence (Savickaite et al. 2022). AR, which constitutes a part of VR, allows interaction in a physical world that is not as artificial as in the case of VR (Berenguer et al. 2020). VR is known to be beneficial to ASD research for its flexibility, controllability, replicability, and modifiable sensory stimulation (Strickland et al. 1996). Additionally, VR allows for the individualization of intervention approaches and reinforcement strategies pragmatically (Strickland et al. 1996). VR scenarios can be easily modified to demonstrate various scenes that are often not feasible in usual therapeutic settings with space limitations and lack of resources. On the other hand, AR features a more tangible presence compared to VR, maintaining body awareness and better generalization from the virtual world to the real world (Pérez-Fuster et al. 2022). Compared to social robots, VR and AR are more commercially available, relatively less expensive, and easier to learn how to work with them (Jyoti and Lahiri 2022; Liu et al. 2017). So, they can be used even outside of research labs, such as schools and clinics.

Based on the definition presented by Savickaite et al. (2022), VR equipment used in autism research can be classified as Desktop, Cave Automatic Virtual Environments (CAVE), Static, and Head-Mounted Display (HMD). Desktop-VR simply displays stimuli on a screen. CAVE-VR is a projection of the stimuli onto the walls by three to six projectors, with the possibility of user position and location tracking. Both Static-VR and HMD-VR use head-mounted displays. The former provides only a stereoscopic view, whereas the latter affords position and location tracking as well. Desktop and HMD offer the lowest and the highest levels of immersion, respectively. Similarly, we can generalize this classification for AR systems.

In the last decade, the number of review papers on the application of VR and AR systems involving individuals with autism has increased significantly. These papers focus, for example, on social skills, emotional skills, cognitive skills, daily living skills, motor skills, speech and language training, or job interview training (Berenguer et al. 2020; Dechsling et al. 2022; Karami et al. 2021; Lorenzo et al. 2019; Mesa-Gresa et al. 2018; Parsons 2016; Zhang et al. 2022b), school-related procedures and educational approaches (Bradley and Newbutt 2018; Bravou et al. 2022), design characteristics of VR systems as training tools (Glaser and Schmidt 2022), and analyzing global trends in the use of VR for learning (Lorenzo et al. 2022). However, despite the importance of JA in ASD, there is no comprehensive review to systematically investigate studies in which training JA skills have been the intervention target. Therefore, this study focused on this specific issue i.e. analysing the use of VR and AR technologies in interventions promoting JA skills in individuals with ASD. The current work aims to fill a significant gap in the literature by providing a comprehensive descriptive analysis of the studies conducted on the use of VR and AR applications for improving JA skills in autistic individuals. The main focus of this study is on (a) collecting the studies conducted with VR and AR applications to discover how clinicians can teach JA skills to individuals with ASD, (b) listing the general characteristics of these studies via descriptive summaries, and (c) evaluating the effectiveness of VR and AR applications for developing JA skills by descriptive analysis. To reach these goals, the following research questions were developed:

• Which type of VR/AR equipment was most preferred for teaching JA skills in ASD?

• Which part of JA skills (RJA or IJA) has been studied more?

• What measures have been employed to assess JA skills in ASD?

• What are the appropriate characteristics of autistic individuals (age range and IQ range) in terms of JA development?

• Does increasing JA skills based on VR/AR interventions have a positive effect on social- communication skills?

Methods

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Page et al. 2021). In Study Selection section, the PRISMA flow diagram that presents the search process was provided.

Search strategy

Systematic searches were conducted at the end of September 2022 using the following electronic databases: Scopus, Psychology and Behavioural Sciences Collection (EBSCO), Web of Science (WoS), PubMed, Institute of Electrical and Electronics Engineers (IEEE) Xplore, Education Resources Information Centre (ERIC), and Cochrane. Google Scholar was also searched as a secondary tool to seek literature that may have been missed in the full systematic review of the electronic databases (Gusenbauer and Haddaway 2020). These databases were selected since they each cover a large part of the relevant literature on these topics.

Searches were carried out using Boolean operators AND/OR by the variations of ‘virtual reality’, ‘augmented reality’, ‘autism’ and ‘joint attention’. See Appendix A for our full keyword strategy. Filters and limits were applied based on the nature of the index and provided functionality.

Following the inclusion and exclusion criteria, the title and abstract of the studies were screened by the first and third authors independently. In addition, the reference lists of studies meeting the inclusion criteria were explored by a hand search. Contributions of grey literature including reviews, conference proceedings, conceptual papers, theses, dissertations, abstracts, notes, protocols, letters, and editorials, were not included. The search was limited to peer-reviewed studies where the authors evaluated the impact of VR/AR-based interventions on subjects with ASD to improve JA skills. Since the first work concerning the use of VR in ASD was published in 1996 (Strickland et al.), only articles published after 1995 were considered. It is worth noting that the first work on the use of AR in ASD was published in 2007 (Richard et al. 2007).

Study selection

This comprehensive review focused on empirical research on the use of VR/AR for developing JA skills in children with ASD. The inclusion criteria applied in the study were that the reports (1) were published after 1995, (2) were published in peer-reviewed journals, (3) were written in English, (4) were published in full-text, (5) applied to the ASD population (considering combined populations, i.e. ASD and other disabilities or typically developing peers), and (6) targeted JA skills as one of intervention targets in the study protocol. The exclusion criteria were as follows: (1) non-interventional studies, such as usability studies like those conducted by Amaral et al. (2017) and Jyoti and Lahiri (2022); (2) studies that did not focus on improving JA skills, such as those that evaluated and quantified JA skills like those conducted by Caruana et al. (2018), He et al. (2022) and Jyoti and Lahiri (2020); and (3) studies that did not included at least one quantifiable outcome measure.

First, a search was conducted and results were imported into EndNote 20 (http://endnote.com). The number of records accessed in the first stage was 1223. The results were compared to identify duplicates. A combination of automated searches (Find Duplicates) and manual reviews were used. The number of found duplicates was 744. Identified duplicates were removed. Second, the titles and abstracts of the remaining records were reviewed. Inclusion criteria were applied, and 724 records were removed. Third, the full text of the remaining 20 reports was reviewed, and exclusion criteria were independently applied by the first two authors. The final result was 6 reports. Furthermore, one report was identified from other methods (see Figure 1). Thus, the final research corpus is 7 peer-reviewed journal articles that met all the inclusion criteria (Table 1). The study selection process is illustrated in Figure 1.

Figure 1.

PRISMA flow diagram illustrating systematic search and selection process.

Table 1.

Methodological characteristics and findings from included studies.

Authors and Date	Technology Equipment	Experiment Group Characteristics				Measurement		Protocol				VR/AR paradigm			Design of study	JA type	Strength Rating (Reichow 2011)
		No.	IQ mean (SD)	Age Range mean (SD) (years)	Gender	Baseline	Outcome	VR/AR Session No.	VR/AR Session duration (min)	Intervention duration (weeks)	Follow-up (months after baseline)	Setting	Task	DL
Hopkins et al. (2011)	Desktop-VR (PC)	24	75.71(27.3)	6-15 10.17(3.02)	22 M 2 F	CARS KBIT	SSRS SSO	12	10 ∼ 25	6	–	Realistic human faces	Following non-verbal social agent cues	yes	Group (RCT)	RJA	Adequate
Cheng and Huang (2012)	Desktop-VR (data glove + projectors)	3	61.67(4.2)	9-12 10.33(1.53)	3 M	Wechsler	PCJA JASS	6	30 ∼ 40	6	3	Play-room scene	Following non-verbal social cues	no	SSED	RJA IJA	Weak
Fletcher-Watson et al. (2016)	Desktop-VR (iPad)	27	NA	NA 4.14(1.0)	21 M 6 F	MSEL ADOS-2 BOSCC MCDI CSBS-DP	ADOS-2 BOSCC MCDI CSBS-DP	28	5 ∼ 10	10	6	VR scenes (plain, safari, store)	Looking at others and following their non-verbal social cues	no	Group (RCT)	RJA	Adequate
Amaral et  al. (2018)	HMD-VR (Oculus Rift + Eye Tracking + EEG P300)	15	102.53(11.6)	16-38 22.17(5.5)	15 M	ADI-R ADOS Wechsler	JAAT ATEC VABS	7	NA	16	10	Virtual bedroom	Following non-verbal social cues	no	SSED	RJA	Weak
Ravindran et al. (2019)	Static-VR (Google Cardboard +  smartphone)	12	NA	9-16 13.5(NA)	10 M 2 F	NA	Novel JA measure based on CSBS	14	5	5	2	Virtual safari	Following verbal and non-verbal social cues	no	SSED	RJA IJA	Weak
Amat et al. (2021)	Desktop-VR (PC + Eye-tracker)	9	NA	7-13 11(1.4)	5 M 4 F	ADOS SCQ SRS-2	Bubble popping VR game	3	NA	2 ∼ 4	–	Virtual Tangram puzzle	Following non-verbal social cue	yes	Group (Quasi experimental)	RJA	Weak
Pérez-Fuster et al. (2022)	Desktop-AR (digital whiteboard +  projector + Kinect sensor)	6	66 (11.7)	3 − 8 6.15(1.9)	5 M 1 F	SCQ GARS-2 Leiter-R	ADOS ESCS An ad-hoc measure	9	15	8	3	Class-room	Following non-verbal social cue	yes	SSED	RJA	Strong

VR: Virtual Reality; AR: Augmented Reality; JA: Joint Attention; RJA: Response to JA; IJA: Initiating JA; M: Male; F: Female; CARS: Childhood Autism Rating Scale; KBIT: Kaufman Brief Intelligence Test; MSEL: Mullen Scales of Early Learning; ADOS: Autism Diagnostic Observation Schedule; BOSCC: Brief Observation of Social Communication Change; MCDI: MacArthur Communicative Development Inventory; CSBS-DP Communication and Symbolic Behavior Scale – Developmental Profile; ADI-R: Autism Diagnostic Interview-Revised; SCQ: Social Communication Questionnaire; SRS: Social Responsiveness Scale; GARS-2: Gilliam Autism Rating Scale, Second Edition; SSRS: Social Skills Rating System; SSO: Social Skills Observation; PCJA: Picture Cards of Joint Attention; JASS: Joint Attention Skills Scale; JAAT: Joint Attention Assessment Task; ATEC: Autism Treatment Evaluation Checklist; VABS: Vineland Adaptive Behavior Scales; ESCS: Early Social Communication Scales; RCT: Randomized Clinical Trial; SSED: Single-Subject Experimental Design.

Data extraction

As shown in Table 1, the authors extracted data from the eligible articles, including information on authors and dates, technology equipment, experiment group characteristics, measurement tools, study protocol, VR/AR paradigm, design of the study, JA types, and report strength rating (see next section). These variables were the content variables of this study. The first three authors performed the coding process on the table independently of each other. Subsequently, all authors came together and compared all the information in the table one by one. In the case of disagreement on data coding, the authors reviewed and discussed it to reach a consensus.

Quality indicator

Eligible studies were independently assessed for rigor by the first and second authors using the standards created by Reichow et al. (2008). Disagreements were discussed with the fourth author. This method is best suited to evaluating empirical research on specific interventions for individuals with ASD. This assessment used two rubrics to measure research quality, one for group research and one for single-subject research. Both consist of methodological elements deemed important for research rigor. Eligible studies were assessed against the appropriate rubrics and an overall rigor rating was created using guidelines on how to synthesis rubrics ratings (Reichow 2011). This process evaluated study quality across two levels of methodological features: primary and secondary indicators. Primary indicators were considered vital components in research design in order to demonstrate validity. Secondary indicators were deemed as important but not vital components of research. To receive an overall strong rating, studies must have received a high rating for all primary indicators and meet four of the secondary indicators. An overall adequate rating was awarded to a study that received a high rating for four primary indicators and meet two secondary indicators with no unacceptable ratings in primary indicators. A study received an overall weak rating if it was awarded less than four high ratings in primary indicators and less than two secondary indicators.

Effect size calculation

The eligible studies are a mixture of two major types of experiment designs, namely, group research design and Single-Subject Experimental Design (SSED). In group research design, one group received the training, and the other group served as a control. The difference between the groups on the outcome measure was used as an estimate of the treatment effect (Between-Subject Effect). On the other hand, in the SSED or pretest-posttest design, each individual was measured before and after treatment, and the difference between the individuals’ scores before and after it was used as an estimate of the treatment effect (Within-Subject Effect). Based on the available data in the eligible studies, the standardized mean difference was used to calculate the effect size (ES) $d_p$ for both designs as follows (Goulet-Pelletier and Cousineau 2018):

$$d_{}=\frac{M_2-M_1}{S_p}$$ (1)

Where $M_1$ and $M_2$ are the mean of Group 1 (or measurement time one) and Group 2 (or measurement time two), respectively, and $S_p$ is the pooled standard deviation caclualated as follows:

$$S_p=\sqrt{\frac{\left(n_1-1\right)S_1^2+\left(n_2-1\right)S_2^2}{n_1+n_2-2}}$$ (2)

where $n_1$ and $n_2$ are the sample size of Group 1 (or measurement time one) and Group 2 (or measurement time two), respectively.

Data were extracted on ES from eligible studies by the first author. For group research design, both between-subject (for the targeted group) and within-subject ES were calculated. As discussed in Durlak (2009), ESs were calculated irrespective of their pvalue.

Results

Technology equipment: VR or AR

As can be seen from Table 1, VR-based interventions were used in 6 of 7 studies examined within the scope of the research (Amaral et al. 2018; Amat et al. 2021; Cheng and Huang 2012; Fletcher-Watson et al. 2016; Hopkins et al. 2011; Ravindran et al. 2019). Only in a very recent study AR-based intervention was employed (Pérez-Fuster et al. 2022).

Desktop-VR/AR was used in five of the seven studies surveyed in our research (Amat et al. 2021; Cheng and Huang 2012; Fletcher-Watson et al. 2016; Hopkins et al. 2011; Pérez-Fuster et al. 2022). In these studies, tasks for the improvement of JA skills were administered by a personal computer (Amat et al. 2021; Hopkins et al. 2011), video projector (Cheng and Huang 2012; Pérez-Fuster et al. 2022), or iPad (Fletcher-Watson et al. 2016). In one study, static-VR (Ravindran et al. 2019) and in another study, HMD-VR (Amaral et al. 2018) were used. It is worth noting that only in two studies were eye-tracker employed (Amaral et al. 2018; Amat et al. 2021). Furthermore, Ravindran et al. (2019) introduced a novel supervised VR which enables therapists to control the VR session through a tablet. Amaral et al. (2018) additionally extracted attentional responses by equipment for the measuring the P300 component of the electroencephalogram (EEG) signal. The P300 is a positive deflection in the EEG waveform appearing about 300 ms after a stimulus is presented to the user.

Characteristics of ASD participants

In the studies, a total of 96 participants with ASD took part in the experiment group, with a minimum of 3 and a maximum of 27 participants. Hopkins et al. (2011) and Fletcher-Watson et al. (2016) recruited 25 and 27 children with ASD, respectively, for their control groups. Furthermore, Amat et al. (2021) employed 9 typically developing (TD) children as their control group.

The average age of participants with ASD ranged from 4.14 (SD: 1.0) to 22.17 (SD: 5.5) years old. In the studies that reported IQ scores, the mean scores ranged from 61.67 (SD: 4.16) to 102.53 (SD: 11.64). However, IQ scores were not reported in 3 of the studies. In terms of gender, it was found that out of the total participants with ASD in the experiment group, 81 were male and 15 were female.

Outcome measures

There was no consensus on the measures of JA used in the studies. The use of non-standardized, self-developed measures in five of seven studies is evidence of this claim (Amaral et al. 2018; Amat et al. 2021; Cheng and Huang 2012; Pérez-Fuster et al. 2022; Ravindran et al. 2019).

Cheng and Huang (2012) employed a self-developed measure consisting of ten picture cards of joint attention (PCJA) and a joint attention skills scale (JASS) with eight items to evaluate RJA and IJA skills. The JASS was adapted from the early social communication scales (ESCS) developed by Mundy et al. (2003).

Fletcher-Watson et al. (2016) employed communication and symbolic behaviour scale - developmental profile (CSBS-DP) (Wetherby and Prizant 2002). They also used the autism diagnostic observation schedule, second edition (ADOS-2) (Lord et al. 2012) and the brief observation of social communication change (BOSCC) (Grzadzinski et al. 2016) to measure broad ASD symptoms.

Amaral et al. (2018) administrated a self-developed measure called joint attention assessment task (JAAT) in virtual environments and using an eye-tracker. They considered to two types of responses in this measure: JAAT_No face and JAAT_Face. The former only considers gaze fixation on the JA target object, while the latter also considers the gaze fixation on the virtual social partner (avatar). The authors used the autism treatment evaluation checklist (ATEC) (Rimland and Edelson 1999) (Rimland and Edelson 1999) and the Vineland adaptive behaviour scales (VABS) (Sparrow et al. 1984) to measure treatment effectiveness and adaptive functioning, respectively.

Ravindran et al. (2019) developed a new JA measure based on the communication and symbolic behaviours scale (CSBS) (Wetherby and Prizant 2002) for JA assessment. This measure included play-based activities appropriate for school-aged children, and focused on the key joint attention behaviours targeted in their study.

Amat et al. (2021) developed a bubble-popping game for pre- and post-assessment. In this game, the bubble would pop when the participant looked at the bubble cued by an avatar, and new bubbles will be generated.

Pérez-Fuster et al. (2022) used the autism diagnostic observation schedule, second edition (ADOS-2) (Lord et al. 2012) and early social communication scales (ESCS) (Mundy et al. 2003) to measure the participants’ RJA skills. Besides, they developed an ad-hoc RJA skill assessment using three types of items: a dummy, posters, and turtles. Posters and turtles were used as JA target objects, while the dummy was used to issue cue towards the target.

Hopkins et al. (2011) did not employ an explicit measure for JA skills. In their study, RJA skills were one of intervention targets to improve social skills. Thus, they used the social skills rating system (SSRS) (Gresham and Elliott 1990) and social skills observation (SSO) to evaluate social skills.

Effect size

Table 2 shows ESs for the outcome measures used in the examined studies. According to Cohen’s classification (Cohen 2016), ESs were divided into five levels: trivial ($d_p\le 0.2$), small ($0.2<d_p\le 0.5$), moderate ($0.5<d_p\le 0.8$), large ($0.8<d_p\le 1.3$), and very large ($d_p>1.3$). Based on this classification, 25% of calculate ESs were trivial, 25% were small, 16% were moderate, 11% was large, and 23% were very large.

Table 2.

Calculated effect sizes (ESs) of intervention outcome.

Study	Measure	Within-Subject ESa		Between-Subject ES
		Baseline to Outcome (p-value)	Baseline to Follow-up (p-value)	Outcome (p-value)	Follow-up (p-value)
Hopkins et al. (2011)	SSRS – Composite (LFA)	0.75	–	0.95 (<0.01)	–
	SSRS – Composite (HFA)	0.26	–	0.3 (0.05)	–
	SSO – Total (LFA)	0.09	–	0.82 (<0.05)	–
	SSO – Total (HFA)	0.41	–	1.3 (<0.01)	–
Cheng and Huang (2012)	JASS	3.55	4.79	N/A
Fletcher-Watson et al. (2016)	ADOS2 – comparison score	–	0.39	–	0.55 (0.45)
	BOSCC– overall total	–	0.01	–	0.07 (0.29)
	BOSCC (SC total)	–	0.01	–	0.10 (0.56)
	CSBS-DP (SC total)	0.18	0.25	0.00 (0.55)	0.34 (0.31)
	CSBS-DP (gestures total)	0.13	0.19	0.04 (0.29)	0.23 (0.93)
Amaral et al. (2018)	JAAT_No face	0.30	0.14	N/A
	JAAT_Face	0.32	0.36
	ATEC total	0.89	1.41
	VABS composite	0.14	0.25
Ravindran et al. (2019)	Total number of interactions	–	0.5 (0.02)	N/A
	Use of eye contact	–	0.57 (0.04)
	Initiation of interactions	–	0.54 (0.08)
Amat et al. (2021)	Highest score	0.67 (0.13)	–	0.46	–
	Time to complete	1.33 (0.01)	–	0.08	–
	Response time	0.78 (0.09)	–	0.72	–
	Gaze prompt speed	1.32 (0.01)	–	0.41 (0.54)	–
	Total Face Fixation	0.88 (<0.01)	–	0.20	–
	Normalized Eye Fixation	0.87 (0.65)	–	0.05	–
	Other Facial Fixation	0.87 (0.03)	–	1.30	–
Pérez-Fuster et al. (2022)	Poster	8.76 (<0.01)	10.99 (<0.01)	N/A
	Turtle	9.86 (<0.01)	7.262 (<0.01)
	ESCS (L/R RJA)	3.045 (<0.01)	–
	ESCS (Behind RJA)	2.49 (<0.01)	–
	ADOS-2	0.58 (0.08)	–

The bold values indicate a statistically significant difference.

^a In group research design was calculated for the targeted group.

SSRS: Social Responsiveness Scale; SSO: Social Skills Observation; LFA: Low Functioning Autism; HFA: High Functioning Autism; JASS: Joint Attention Skills Scale; ADOS: Autism Diagnostic Observation Schedule; BOSCC: Brief Observation of Social Communication Change; SC total: Social Communication total; JAAT: Joint Attention Assessment Task; ATEC: Autism Treatment Evaluation Checklist; VABS: Vineland Adaptive Behavior Scales; ESCS: Early Social Communication Scales; L/R RJA: Left/Right Response to Joint Attention; N/A: Not Applicable.

Protocol

The review reveals that the number of intervention sessions varied, with a minimum of 3 sessions and a maximum of 28 sessions. The duration of these sessions ranged from 5 to 40 min. In studies examined, the intervention took place over periods of 2 weeks to 16 weeks. Follow-up assessments were conducted in five of the studies (Amaral et al. 2018; Cheng and Huang 2012; Fletcher-Watson et al. 2016; Pérez-Fuster et al. 2022; Ravindran et al. 2019) and were performed between 2 and 10 months subsequent to the baseline measurements.

In the study by Hopkins et al. (2011), 12 VR sessions were conducted over a period of 6 weeks, with each session lasting approximately 10– 25 min. However, no follow-up assessment was performed. In the study by Cheng and Huang (2012), 6 VR sessions were administered over 6 weeks, with each session lasting approximately 30–40 min. Additionally, a maintenance phase was conducted once a week for 12 days following the intervention.

In the study by Fletcher-Watson et al. (2016), 28 VR sessions were carried out over a period of 10 weeks, with each session lasting approximately 5–10 min. Follow-up assessments were completed 6 months after the baseline. In the study by Amaral et al. (2018), the intervention was spread over 7 VR sessions over a period of 4 months. However, the duration of each session was not specified in the paper. Follow-up assessments were performed 10 months after the baseline.

In the study by Ravindran et al. (2019), 14 VR sessions were completed over a period of 5 weeks, with each VR episode lasting no more than 5 min. Baseline assessments were repeated 2 months after the baseline. In the study by Amat et al. (2021), 3 VR sessions were administered over a period of 2–4 weeks. However, the duration of each session was not specified in the paper. The authors did not conduct a follow-up study.

In the study by Pérez-Fuster et al. (2022), an AR paradigm was conducted in two phases: the learning phase and the intervention phase. The learning phase was completed after a maximum of three sessions over a period of one week for each participant. Following this, the intervention phase began and was completed after a maximum of five sessions over a period of 2 weeks for each participant. Each AR session lasted 15 min. Three months after the baseline, follow-up assessments were administered to participants.

VR/AR paradigm

The review showed that humanoid avatars were used as mediators for administering JA tasks in six out of seven studies (Amaral et al. 2018; Amat et al. 2021; Fletcher-Watson et al. 2016; Hopkins et al. 2011; Pérez-Fuster et al. 2022; Ravindran et al. 2019). In three of these studies, the full human body was used as the avatar (Fletcher-Watson et al. 2016; Ravindran et al. 2019), while in the other three studies, only the human face was used (Amat et al. 2021; Hopkins et al. 2011). In one study, a data glove was used by participants to answer questions and practice JA skills (Cheng and Huang 2012).

Hopkins et al. (2011) used a realistic human face as the JA administrator. The overall goal of the VR paradigm was to promote awareness of the movements and features of the face, particularly the area around the eyes. Specifically, a game was designed to teach children to attend to eye gaze, respond to JA, and understand that eye gaze can convey intent. JA was taught by instructing the children to follow the avatar’s eyes to determine what face or object the avatar was attending to. The VR paradigm increased in difficulty to assure that participants of various levels could be successful as well as challenged by the tasks.

Cheng and Huang (2012) employed a virtual playroom scene in which the participant could play with other children in the virtual environment. The authors formulated twenty-four questions rooted in social scenarios, encompassing the understanding of JA concepts. Then, a data glove was used to practice JA skills i.e. pointing, showing, sharing, and interacting. Animated social events, in addition to the text content, prompt instructions, voice, and feedback, were implemented in the designed system.

Fletcher-Watson et al. (2016) designed an app to give children an opportunity to rehearse two key social-communication skills: attending to people and following social cues. In the app, various VR environments, such as safari and store, were implemented. Animated child avatars were used as mediators for administering JA tasks by pointing and head movement. The game complexity increased with more distractors appearing on screen, and the avatar moved from pointing and looking to just looking at a target.

Amaral et al. (2018) utilized a VR paradigm within an immersive virtual environment. This environment simulated a bedroom furnished with common items such as a bed, table, and shelves. It also included various objects like frames, books, lights, a ball, and a laptop. The participants were asked to respond to the head cue of the avatar in the centre of the scene, looking at the object of interest.

Ravindran et al. (2019) developed a safari-themed setting in a VR environment that was completed with animals designed to draw children’s attention. An avatar in the virtual environment initiated and responded to JA bids and could also verbally prompt the child when needed. Five different scenarios were designed to train target JA subskills such as sharing, following, and directing attention.

Amat et al. (2021) initiated an RJA task with a virtual avatar (an animated woman’s face) as an interactive partner that provided participants with head and/or gaze prompts. Children played a tangram puzzle game with the virtual avatar. Prompts and visual aids were administrated using the least-to-most prompting mechanism employed to increase the difficulty level in the virtual game.

Pérez-Fuster et al. (2022) used an AR paradigm focusing on JA skills. Participants saw their skeletons –instead of their live images– reflected on the screen (interactive digital whiteboard). The AR game consisted of a virtual child’s face with two big eyes that appeared surrounded by closed windows. The virtual child then pointed to and/or looked at one of the windows. Participants were then asked to touch the window that the virtual child was looking at. The difficulty level of the game increased according to the progress of the child.

Design of study

In four studies examined within the scope of this paper, SSED was considered (Amaral et al. 2018; Cheng and Huang 2012; Pérez-Fuster et al. 2022; Ravindran et al. 2019). Amaral et al. (2018) and Ravindran et al. (2019) conducted a pre-experimental (or AB) design, while Pérez-Fuster et al. (2022) and Cheng and Huang (2012) carried out a multiple baseline SSED. The studies conducted by Amaral et al. (2018) and Ravindran et al. (2019) were pilot and feasibility trials, respectively.

Three studies were conducted using group research methods. One study executed a quasi-experimental design (Amat et al. 2021). In this study, a pilot study was conducted to evaluate the hypothesis that practicing VR would be able to improve gaze sharing and gaze following skills in autistic children. The authors compared participants’ performance in ASD and TD groups to identify any meaningful differences. In two other studies, a Randomized Controlled Trial (RCT) was conducted (Fletcher-Watson et al. 2016; Hopkins et al. 2011). Hopkins et al. (2011) did not declare the process of randomly assigning a trial participant to treatment or control arms. In the study, parents were blind to their child’s group assignment. Fletcher-Watson et al. (2016) used stratified randomization by the ADOS social-communication algorithm and employed block randomization to produce randomization lists. In this study, participants and parents were not blind to intervention allocation. However, they were blind to hypotheses regarding the skills being targeted by the intervention.

JA type and general results

In observation of the type of JA studied, it was found that RJA was the main criterion in all the studies. Only two interventions studied both RJA and IJA together as dependent variables (Cheng and Huang 2012; Ravindran et al. 2019). In terms of communication, the reviewed studies mostly focused on nonverbal communication, such as following non-verbal social agent cues (gaze, head, and finger pointing), and in one study both verbal (e.g. the avatar says: ‘hey’, ‘look’, ‘who made the sound?’) and non-verbal cues were examined (Ravindran et al. 2019). Positive results were observed in all but one of the studies investigating the effect of an iPad™ intervention targeting social-communication skills through the developmental precursors of JA (Fletcher-Watson et al. 2016). The results from this study showed positive attitudes among participants, a lack of harm, and the potential of the intervention to deliver therapeutic content at a low economic cost.

Quality assessment

The research strength of all included studies was calculated according to the criteria established by Reichow (2011). The last column of Table 1 provides a summary of the strength ratings for each. One study (14.3%) received a strong rating, seven studies (28.6%) received an adequate rating, and four studies (57.1%) received a weak rating. The details for these rating can be found in Appendix B.

Limitations of studies

The most significant limitation of the studies reviewed here was the lack of appropriate JA measures for use without conditions. For instance, the CSBS and ESCS are only suitable for use with infants, toddlers, or young children with delayed communication skills. The CSBS was only normed up to 24 months of age, while the ESCS was designed to be used with children aged 8–30 months. Some studies proposed customized and self-developed JA measurement tools (Amaral et al. 2018; Amat et al. 2021; Cheng and Huang 2012; Pérez-Fuster et al. 2022; Ravindran et al. 2019) but the reliability, validity, and psychometric properties of the measures were not evaluated. Pérez-Fuster et al. (2022) acknowledged this limitation in their study.

Only two RCT studies were conducted which is understandable considering the design challenges associated with RCTs, such as ethical difficulties and higher costs. As a result, it is reasonable for most studies to choose SSEDs. It is noteworthy that SSEDs can provide reliable results, especially with heterogeneous populations such as ASD, where SSED is the most common research design (Horner et al. 2005).

Other limitations included the short duration of some studies (Amat et al. 2021; Hopkins et al. 2011; Ravindran et al. 2019) and small sample sizes (Amaral et al. 2018; Amat et al. 2021; Cheng and Huang 2012; Pérez-Fuster et al. 2022; Ravindran et al. 2019). Longitudinal studies with a larger sample size would allow more complex analyses of VR/AR assistive capabilities and their impact. The lack of follow-up assessments (Amat et al. 2021; Hopkins et al. 2011) can also be considered as the limitations of some studies.

Furthermore, only one study addressed the generalizability of skills learned in VR/AR environments to real-world situations (Pérez-Fuster et al. 2022). Additionally, there is a need to directly compare VR/AR-based interventions with other treatment methods.

Discussion

VR/AR-based interventions for individuals with ASD are an innovative approach that has attracted the interest of the scientific community for special education. In this paper, all papers on the clinical applications of VR/AR-based interventions were reviewed to determine their effectiveness for improving JA skills. Although there is a commercial VR product for individuals with ASD (Floreo Inc.), this research field is currently premature and it is at the stage of exploration in many aspects. VR/AR-based interventions offer an alternative option that can help develop skills in individuals with ASD and can complement and support conventional interventions. However, there are few experimental studies on the clinical effectiveness of VR/AR-based interventions for improving JA skills of individuals with ASD. To offer a more comprehensive understanding of this innovative approach, the results revealed by experimental studies were collectively analysed, with an effort to enhance the existing body of research by incorporating diverse elements.

Which part of JA skills (RJA or IJA) has been studied more?

The review showed that RJA was the main skill among all the studies. Only in two interventions, both RJA and IJA were studied together (Cheng and Huang 2012; Ravindran et al. 2019). Since RJA is a passive action, it generally is much easier to identify and quantify (Zhang et al. 2022a). Furthermore, implementing IJA paradigms in virtual environments is technically more complicated than RJA paradigms. Because in an IJA paradigm, the autistic individual should use non-verbal or/and verbal cues to direct the avatar’s attention to objects or events; implementing such scenarios in the virtual environment is not too easy.

Additionally, research has shown that individuals with ASD find it easier to respond to the gestures of others rather than to spontaneously initiate joint attention gestures (Mundy et al. 1986). As a result, they exhibit more significant deficits in IJA (Mundy et al. 1990). It is therefore natural that most studies have focused on RJA.

Which type of VR/AR equipment was most preferred for JA skills teaching in ASD?

According to the reviewed literature, Desktop-VR/AR seems to be the most common equipment used in the studies we reviewed to improve the JA skills of individuals with ASD (Amat et al. 2021; Cheng and Huang 2012; Fletcher-Watson et al. 2016; Hopkins et al. 2011; Pérez-Fuster et al. 2022). It is simple and affordable, and does not require any special devices. These are the reasons for its success in research. However, it is the least immersive type of VR/AR equipment. Participants’ interaction is restricted, and they can often only respond by using a keyboard or touchscreen while viewing the stimuli on a screen. This may affect the effectiveness and generalizability of this methodology.

More immersive VR/AR technologies were used only in two studies (Amaral et al. 2018; Ravindran et al. 2019). Ravindran et al. (2019) and Amaral et al. employed Static-VR and HMD-VR, respectively. These technologies are more complex and expensive compared to Desktop-VR. In Static-VR, an individual can view the world in 360 degrees but cannot interact or walk in a three-dimensional environment. In HMD-VR, users are fully immersed in the virtual environment and can interact/move in it. Thus, it is expected that these methodologies may be more effective to teach skills to ASD populations and their outcome may be the more generalizable outcome. CAVE-VR/AR was not employed to teach JA skills to individuals with ASD. It may be related to the high cost and complexity of the implementation of this approach.

Only in a very recent study, AR technology was employed (Pérez-Fuster et al. 2022). Perhaps the main reason is that AR is a newer technology than VR. Although AR is more innovative compared to VR, more research is required to show the applicability of AR-based intervention to improve the JA skills of individuals with ASD.

What measures have been employed to assess JA skills in ASD?

JA is commonly quantified using structured assessment procedures that involve specific activities and prompts to evoke behaviours of interest (Roos et al. 2008). Despite the central importance of JA in development, there appears to be a lack of comprehensive measurement tools for JA skills, particularly in later childhood and adolescence (Bean and Eigsti 2012). Modules 1-2 of the gold-standard ADOS-2, ESCS, and CSBS effectively assess JA skills in young children. The ADOS-2 and ESCS measure a child’s response to structured and semi-structured prompts that involve a triadic gaze between the experimenter and an object, or event. The CSBS was developed to evaluate verbal and non-verbal communication in children at risk for communication and language impairments. Two studies in this review have used ADOS-2, ESCS, and CSBS as outcome measures (Fletcher-Watson et al. 2016; Pérez-Fuster et al. 2022). However, ESCS is appropriate for children up to 30 months, ADOS-2 is suitable for those with two-word phrase speech, and CSBS is only normed up to 2 years of age. As a result, these measures are less useful for characterizing deficits in older children.

Some of the measures used, such as SSRS, the social responsiveness scale, second edition (SRS-2) (Constantino and Gruber 2012), and the social communication questionnaire (SCQ) (Rutter 2003) were developed as screens for behaviours associated with ASD. Other measures, such as VABS, are broad in scope and assess either adaptive behaviours as a whole or a range of ASD-related behaviours. Another measure is BOSCC which has been developed based on social-communication behaviours and rated in the ADOS.

In addition to measures used in the reviewed papers, there are other measures in the literature that can be used to assess JA skills. Some of these measures that could be considered for future studies include the childhood joint attention rating scale (C-JARS) (Birkeneder et al. 2023; Mundy et al. 2017) and the social communication assessment for toddlers with autism (SCATA) (Drew et al. 2007). Bean and Eigsti (2012) also developed a JA measure appropriate for ages 7–17 years. Among these measures, C-JARS has been employed in VR studies (Jyoti and Lahiri, 2020, Jyoti and Lahiri 2022).

What are the appropriate autistic individual characteristics (age range and IQ range) in terms of JA development?

Age range

Since RJA begins to develop in typically developing children at around 6 months of age, and deficits in JA are considered by many researchers to be an early predictor of childhood autism (e.g. Osterling and Dawson 1994). Research has shown that difficulties with RJA become less evident in children with ASD as their language or mental age level exceeds what is typically observed in 30-month-old children (Gillespie-Lynch et al. 2013; Mundy et al. 1994). However, differences in IJA do not appear to begin to remit at 30 months. Instead, evidence of differences in IJA in children with autism continues to be observed through the preschool period and even into adolescence (Charman 2003; Dawson et al. 2004). JA is a core deficit in ASD that should be a target of early intervention programs (Mundy and Crowson 1997; Mundy et al. 2009). When examining the studies, only one study emphasized early age (Fletcher-Watson et al. 2016).

A noteworthy point related to the ages of participants is the wide range of the participant’s ages (e.g. 22.17 ± 5.5 years old in Amaral et al. (2018)). The broad age range of participants in JA studies means that variations in the results of JA skills may be influenced by external factors such as maturation and duration and quality of education.

IQ range

IQ scores were reported for the 4 studies examined in this paper. The reported IQ ranges were wide, also. For example, in Amaral et al. (2018) study the IQ scores were 102.53 ± 11.64 whereas those values in Cheng and Huang (2012) study were 61.67 ± 4.16.

Different instruments were employed to measure IQ scores. Wechsler abbreviated scale of intelligence (WASI), Wechsler adult intelligence scale-third edition (WAIS-III), Kaufman brief intelligence test (KBIT), and Leiter-R were administrated by Cheng and Huang (2012), Amaral et al. (2018), Hopkins et al. (2011), and Pérez-Fuster et al. (2022), respectively.

The DSM-5 acknowledges the difficulty of using IQ tests to measure intellectual ability in individuals with autism and cautions that the symptoms of autism itself may complicate the assessment (American Psychiatric Association, 2013). The manual also notes that IQ scores in individuals with ASD may be unstable, particularly in early childhood, meaning that a child’s score may vary widely over time.

Due to the difficulties associated with measuring IQ in individuals with autism, the limited number of participants in the reviewed studies, and discrepancies in the IQ measurement instruments used, it is not possible to explicitly answer the question of how the use of VR/AR therapy works for autistic individuals with different IQ scores. Further studies are needed to explore the effectiveness of VR/AR in improving the JA skills in autistic participants with varying IQ levels.

Does increasing JA skills based on VR/AR intervention have a positive effect on social- communication skills?

JA skills, such as eye gaze and gaze coordination, occur within a social-communication context. Children and adults use these skills to enrich mutual understanding during social communication interactions. As a result, researchers believe that a lack of JA skills may contribute to the social and communication barriers experienced by individuals with ASD (Charman 2003; Sigman et al. 1999; Toth et al. 2006; Warreyn et al. 2014). Teaching individuals with autism to respond to JA is important for increasing their social awareness. In other words, by teaching them to correctly respond to the actions of others in a social-communication context, they learn that other people have social intentions and that a response is expected (Whalen and Schreibman 2003).

Hopkins et al. (2011) reported the effectiveness of using VR-based intervention targeting JA in social interactions. In this study, children with ASD had opportunities to practice eye gaze, expression matching, and face recognition with realistic avatar assistants. Their social skill improvement was measured by SSRS and SSO. Pérez-Fuster et al. (2022) found that participants were able to successfully transfer learned social skills from the virtual environment to real-world situations. However, Fletcher-Watson et al. (2016) did not observe an impact on real-world social communication skills. The rest of studies did not explicitly measure the transfer impact of learned JA skill on social-communication skills in the real-world situation. This is an important aspect that should be considered in future studies.

Limitations of this study

This review study had several limitations that are important to note. First, the search for grey literature was omitted, which may have biased our sample of included studies as traditionally, studies with significant and novel results have a greater chance of being published. Second, the number of eligible studies was small due to the strict inclusion criteria set for this review. Third, the generalization of JA skills could not be examined because all skills were realized in structured settings and the transfer of these skills to natural contexts was neglected in some studies. Forth, it was not possible to conduct a meta-analysis on the effects of interventions due to the small number of eligible studies and the heterogeneity of the outcome measures used.

Conclusions

The use of VR/AR-based interventions for autistic individuals is an innovative approach that has attracted the interest of many researchers. In particular, the number of publications has increased significantly since 2015, coinciding with VR facilities becoming more readily available to developers (Savickaite et al. 2022). Despite this growth, only a limited number of studies have focused on using VR/AR-based interventions to develop JA skills in individuals with ASD. This study is the first to systematically review the current status of the literature on the clinical applications of VR/AR for training JA skills in individuals with autism. While this research field is still in its early stages, VR and AR technologies offer a potential supplement to conventional interventions for developing JA skills. To further advance this unique approach, the results reported in interventional studies were collectively examined and an effort was made to enrich the existing research literature with additional insights.

In conclusion, the results of this study can be summarized as follows: Desktop VR was the most widely used equipment. The age range of individuals with ASD varied between 4.14 and 22.17 years old. The IQ scores of participants with ASD ranged from 61.67 to 102.53. It is noteworthy that only two RCT studies were conducted. RJA was the main focus among JA types in the studies. Humanoid avatars were used as mediators for administering JA agent cues in most studies. Nonverbal communication cues were mostly employed in the JA tasks. There was no consensus on the measures used to assess JA skills. The effect sizes for 50% of outcome measures were small or trivial. More than 50% of the reviewed studies received a weak quality rating. The studies generally reported positive results, and no side effects were reported from using VR/AR technologies.

A bottleneck in the evolution of this field appears to be the lack of appropriate and standardized JA measures for virtual environments. Existing measures are often developed for conventional interventions, but the rapid progress of technology-based interventions such as VR and AR necessitates the design and development of new, appropriate assessment tools.

It has been shown that individuals with autism, particularly children, have quite positive attitudes towards VR/AR technologies when it comes to interacting and developing JA skills (Amat et al. 2021; Cheng and Huang 2012; Hopkins et al. 2011; Pérez-Fuster et al. 2022; Ravindran et al. 2019). Despite these preliminary results on the use of VR/AR to develop JA skills in individuals with ASD, there is not enough evidence to determine the effectiveness of these applications. Although the reviewed articles show positive results, the small number of studies and different assessment tools make it is difficult to determine the effectiveness of this approach with certainty. To address the gaps in the literature, research must be conducted in a focused and targeted manner. It is therefore recommended that new experimental studies be designed and conducted on the use of VR/AR-based interventions to improve JA skills and determine their effectiveness.

Appendix A. Search query

Table A1.

Full search queries used for different databases.

Database	Search Query
Scopus	TITLE-ABS-KEY((VR) OR (‘virtual realit*’) OR (AR) OR (‘augmented realit*’) OR (‘virtual environment*’) OR (MUVE) OR (‘immersive learning’) OR (‘virtual learning*’) OR (‘virtual world*’) OR (‘three dimensional world*’) OR (Avatar) OR (‘virtual character*’) OR (‘virtual human*’) OR (‘Smart Glasses’) OR (‘Head mounted display*’) OR (HMD) OR (CAVE) OR (‘Digital Technolog*’) OR (‘mediated realit*’) OR (‘Computer mediated realit*’) OR (‘Mixed realit*’) OR (‘Multimediated Realit*’) OR (‘Modulated Realit*’) OR (‘artificial realit*’) OR (‘3D interaction*’) OR (‘three dimensional interaction*’) OR (‘computer simulated realit*’) OR (‘computer simulated environment*’) OR (‘Interactive learning environment*’) OR (‘Internet based intervention*’)) AND TITLE-ABS-KEY((Autism*) OR (ASD) OR (Autistic*) OR (Asperger*) OR (‘Pervasive Development Disorder*’) OR (‘pervasive children developmental disorder*’)) AND TITLE-ABS-KEY((JA) OR (‘Joint attention*’) OR (‘Joint engagement*’) OR (Gaze) OR (Attention) OR (‘Eye Tracking*’) OR (‘Eye contact’)) AND (PUBYEAR AFT 1995) AND (LIMIT-TO (LANGUAGE,’English’))
EBSCO	((VR) OR (‘virtual realit*’) OR (AR) OR (‘augmented realit*’) OR (‘virtual environment*’) OR (MUVE) OR (‘immersive learning’) OR (‘virtual learning*’) OR (‘virtual world*’) OR (‘three dimensional world*’) OR (Avatar) OR (‘virtual character*’) OR (‘virtual human*’) OR (‘Smart Glasses’) OR (‘Head mounted display*’) OR (HMD) OR (CAVE) OR (‘Digital Technolog*’) OR (‘mediated realit*’) OR (‘Computer mediated realit*’) OR (‘Mixed realit*’) OR (‘Multimediated Realit*’) OR (‘Modulated Realit*’) OR (‘artificial realit*’) OR (‘3D interaction*’) OR (‘three dimensional interaction*’) OR (‘computer simulated realit*’) OR (‘computer simulated environment*’) OR (‘Interactive learning environment*’) OR (‘Internet based intervention*’)) AND ((Autism*) OR (ASD) OR (Autistic*) OR (Asperger*) OR (‘Pervasive Development Disorder*’) OR (‘pervasive children developmental disorder*’)) AND ((JA) OR (‘Joint attention*’) OR (‘Joint engagement*’) OR (Gaze) OR (Attention) OR (‘Eye Tracking*’) OR (‘Eye contact’))
WoS	TS=((VR) OR (‘virtual realit*’) OR (‘virtual-realit*’) OR (AR) OR (‘augmented realit*’) OR OR (‘augmented-realit*’) OR (‘virtual environment*’) OR (‘virtual-environment*’) OR (MUVE) OR (‘immersive learning’) OR (‘virtual learning*’) OR (‘virtual world*’) OR (‘virtual-world*’) OR (‘three dimensional world*’) OR (‘three-dimensional world*’) OR (Avatar) OR (‘virtual character*’) OR (‘virtual-character*’) OR (‘virtual human*’) OR (‘virtual-human*’) OR (‘Smart Glasses’) OR (‘Smart-Glasses’) OR (‘Head mounted display*’) OR (HMD) OR (CAVE) OR (‘Digital Technolog*’) OR (‘Digital-Technolog*’) OR (‘mediated realit*’) OR (‘mediated-realit*’) OR (‘Computer mediated realit*’) (‘Computer mediated-realit*’) OR (‘Mixed realit*’) OR (‘Mixed-realit*’) OR (‘Multimediated Realit*’) (‘Multimediated-Realit*’) OR (‘Modulated Realit*’) OR (‘Modulated-Realit*’) OR (‘artificial realit*’) OR (‘artificial-realit*’) OR (‘3D interaction*’) OR (‘three dimensional interaction*’) OR (‘three-dimensional interaction*’) OR (‘computer simulated realit*’) OR (‘computer simulated-realit*’) OR (‘computer simulated environment*’) OR (‘Interactive learning environment*’) OR (‘Internet based intervention*’) OR (‘Internet-based intervention*’)) AND TS=((Autism*) OR (ASD) OR (Autistic*) OR (Asperger*) OR (‘pervasive Development Disorder*’) OR (‘pervasive children developmental disorder*’) OR (‘Autism Spectrum Disorder*’)) AND TS=((JA) OR (‘Joint attention*’) OR (‘Joint engagement*’) OR (Gaze) OR (Attention) OR (‘Eye Tracking*’) OR (‘Eye contact’)) AND PY=(1995-2022)
Pubmed	((VR) OR (‘virtual realit*’) OR (AR) OR (‘augmented realit*’) OR (‘virtual environment*’) OR (MUVE) OR (‘immersive learning’) OR (‘virtual learning*’) OR (‘virtual world*’) OR (‘three dimensional world*’) OR (Avatar) OR (‘virtual character*’) OR (‘virtual human*’) OR (‘Smart Glasses’) OR (‘Head mounted display*’) OR (HMD) OR (CAVE) OR (‘Digital Technolog*’) OR (‘mediated realit*’) OR (‘Computer mediated realit*’) OR (‘Mixed realit*’) OR (‘Multimediated Realit*’) OR (‘Modulated Realit*’) OR (‘artificial realit*’) OR (‘3D interaction*’) OR (‘three dimensional interaction*’) OR (‘computer simulated realit*’) OR (‘computer simulated environment*’) OR (‘Interactive learning environment*’) OR (‘Internet based intervention*’) OR (virtual reality[MeSH Terms]) OR (immersive virtual reality[MeSH Terms]) OR (augmented reality[MeSH Terms]) OR (virtual reality exposure therapy[MeSH Terms]) OR (Smart Glasses[MeSH Terms]) OR (Digital technology[MeSH Terms])) AND ((Autism*) OR (ASD) OR (Autistic*) OR (Asperger*) OR (‘Pervasive Development Disorder*’) OR (‘pervasive children developmental disorder*’) OR (Autistic Disorder[MeSH Terms]) OR (Autism Spectrum Disorder[MeSH Terms]) OR (Asperger Syndrome[MeSH Terms]) OR (Pervasive Development Disorders[MeSH Terms])) AND ((JA) OR (‘Joint attention*’) OR (‘Joint engagement*’) OR (Gaze) OR (Attention) OR (‘Eye Tracking*’) OR (‘Eye contact’) OR (Eye-Tracking Technology[MeSH Terms])) AND ((‘1995/01/01’[PDAT]: ‘2022/08/31’[PDAT]))
IEEE	((‘All Metadata’:’virtual realit*’ OR ‘All Metadata’:’augmented realit*’ OR ‘All Metadata’:’virtual environment*’ OR ‘All Metadata’:MUVE OR ‘All Metadata’:’virtual world’ OR ‘All Metadata’:’three dimensional world’ OR ‘All Metadata’:Avatar OR ‘All Metadata’:’Smart Glasses’ OR ‘All Metadata’:HMD OR ‘All Metadata’:CAVE OR ‘All Metadata’:’mediated realit*’ OR ‘All Metadata’:’Mixed realit*’) AND (‘All Metadata’:Autism OR ‘All Metadata’:ASD OR ‘All Metadata’:Autistic* OR ‘All Metadata’:Asperger* OR ‘All Metadata’:’Pervasive Development Disorder’) AND (‘All Metadata’:JA OR ‘All Metadata’:’Joint attention’ OR ‘All Metadata’:’Joint engagement*’ OR ‘All Metadata’:Gaze OR ‘All Metadata’:Attention OR ‘All Metadata’:’Eye Tracking’ OR ‘All Metadata’:’Eye contact’))
ERIC	((VR) OR (‘virtual realit*’) OR (AR) OR (‘augmented realit*’) OR (‘virtual environment*’) OR (MUVE) OR (‘immersive learning’) OR (‘virtual learning*’) OR (‘virtual world*’) OR (‘three dimensional world*’) OR (Avatar) OR (‘virtual character*’) OR (‘virtual human*’) OR (‘Smart Glasses’) OR (‘Head mounted display*’) OR (HMD) OR (CAVE) OR (‘Digital Technolog*’) OR (‘mediated realit*’) OR (‘Computer mediated realit*’) OR (‘Mixed realit*’) OR (‘Multimediated Realit*’) OR (‘Modulated Realit*’) OR (‘artificial realit*’) OR (‘3D interaction*’) OR (‘three dimensional interaction*’) OR (‘computer simulated realit*’) OR (‘computer simulated environment*’) OR (‘Interactive learning environment*’) OR (‘Internet based intervention*’)) AND ((Autism*) OR (ASD) OR (Autistic*) OR (Asperger*) OR (‘Pervasive Development Disorder*’) OR (‘pervasive children developmental disorder*’)) AND ((JA) OR (‘Joint attention*’) OR (‘Joint engagement*’) OR (Gaze) OR (Attention) OR (‘Eye Tracking*’) OR (‘Eye contact’))
Cochrane	#1 ((VR):ti,ab,kw OR (‘virtual realit*’):ti,ab,kw OR (AR):ti,ab,kw OR (‘augmented realit*’):ti,ab,kw OR (‘virtual environment*’):ti,ab,kw OR (MUVE):ti,ab,kw OR (‘immersive learning’):ti,ab,kw OR (‘virtual learning*’):ti,ab,kw OR (‘virtual world*’):ti,ab,kw OR (‘three dimensional world*’):ti,ab,kw OR (Avatar):ti,ab,kw OR (‘virtual character*’):ti,ab,kw OR (‘virtual human*’):ti,ab,kw OR (‘Smart Glasses’):ti,ab,kw OR (‘Head mounted display*’):ti,ab,kw OR (HMD):ti,ab,kw OR (CAVE):ti,ab,kw OR (‘Digital Technolog*’):ti,ab,kw OR (‘mediated realit*’):ti,ab,kw OR (‘Computer mediated realit*’):ti,ab,kw OR (‘Mixed realit*’) OR (‘Multimediated Realit*’):ti,ab,kw OR (‘Modulated Realit*’):ti,ab,kw OR (‘artificial realit*’):ti,ab,kw OR (‘3D interaction*’):ti,ab,kw OR (‘three dimensional interaction*’):ti,ab,kw OR (‘computer simulated realit*’):ti,ab,kw OR (‘computer simulated environment*’):ti,ab,kw OR (‘Interactive learning environment*’):ti,ab,kw OR (‘Internet based intervention*’):ti,ab,kw)9188 #2 MeSH descriptor: [Virtual Reality] explode all trees 528 #3 MeSH descriptor: [Augmented Reality] explode all trees 28 #4 MeSH descriptor: [Virtual Reality Exposure Therapy] explode all trees 229 #5 MeSH descriptor: [Smart Glasses] explode all trees 10 #6 MeSH descriptor: [Digital Technology] explode all trees 9 #7 #1 OR #2 OR #3 OR #4 OR #5 OR #6 9511 #8 ((Autism*):ti,ab,kw OR (ASD):ti,ab,kw OR (Autistic*):ti,ab,kw OR (Asperger*):ti,ab,kw OR (‘Pervasive Development Disorder*’):ti,ab,kw OR (‘pervasive children developmental disorder*’):ti,ab,kw)5224 #9 MeSH descriptor: [Autistic Disorder] explode all trees1185 #10 MeSH descriptor: [Autism Spectrum Disorder] explode all trees1874 #11 MeSH descriptor: [Asperger Syndrome] explode all trees 76 #12 MeSH descriptor: [Child Development Disorders, Pervasive] explode all trees2059 #13 #8 OR #9 OR #10 OR #11 OR #12 5234 #14 ((JA):ti,ab,kw OR (‘Joint attention*’):ti,ab,kw OR (‘Joint engagement*’):ti,ab,kw OR (Gaze):ti,ab,kw OR (Attention):ti,ab,kw OR (‘Eye Tracking*’):ti,ab,kw OR (‘Eye contact’):ti,ab,kw)37390 #15 MeSH descriptor: [Eye-Tracking Technology] explode all trees 19 #16 #14 OR #15 37390 #17 #7 AND #13 AND #16 11
Google Scholar	allintitle:(virtual reality AND joint attention) allintitle:(VR AND joint attention) allintitle:(Avatar AND joint attention) allintitle:(HMD AND joint attention) allintitle:(Digital Technology AND joint attention) allintitle:(AR AND ‘Joint attention’) allintitle:(‘virtual reality’ AND ‘autism’ AND GAZE) allintitle:(AR AND ‘autism’ AND ‘Joint attention’) allintitle:(‘virtual reality’ AND ‘autism’ AND attention) allintitle:(VR AND ‘autism’ AND GAZE)

Appendix B. Research report rigour and strength

Table B1.

Research report rigour rating for SSEDs.

Study	Rigor Rating
	Primary Quality Indicators						Secondary Quality Indicators
	PART	DV	IV	BSLN	VIS ANAL	EXP CON	IOA	KAP	BR	FID	G/M	SV
(Cheng and Huang 2012)	A	U	A	H	H	H	Y	N	N	N	Y	Y
(Amaral et al. 2018)	H	A	H	U	U	A	N	N	N	N	Y	N
(Ravindran et al. 2019)	A	U	A	U	U	A	N	N	N	N	N	Y
(Pérez-Fuster et al. 2022)	H	A	H	H	H	H	Y	Y	Y	N	Y	Y

PART: participant characteristics; IV: independent variable; DV: dependent variable; BSLN: baseline condition; VIS AN: visual analysis; EXP CON: experimental control; IOA: interobserver agreement; KAP: Kappa; FID: fidelity; BR: blind raters; G/M: generalization and/or maintenance; SV: social validity; H: high quality; A: acceptable quality; U: unacceptable quality; Y: there is evidence; N: there is no evidence. Rating form adapted from (Reichow 2011) (p. 38).

Table B2.

Research report rigour rating for group research designs.

Study	Rigor Rating
	Primary Quality Indicators						Secondary Quality Indicators
	PART	IV	CC	DV	LRQ	STAT	RA	IOA	BR	FID	ATR	G/M	ES	SV
(Hopkins et al. 2011)	H	A	H	H	H	H	Y	Y	Y	N	Y	N	N	Y
(Fletcher-Watson et al. 2016)	H	A	A	H	H	H	Y	Y	Y	N	Y	Y	N	Y
(Amat et al. 2021)	U	A/	H	H	H	A	N	N	N	N	Y	N	N	Y

PART: participant characteristics; IV: independent variable; CC: comparison condition; DV: dependent variable; LRQ: link between research question and data analysis; STAT: statistical analysis; RA: random assignment; IOA: interobserver agreement; BR: blind raters; FID: fidelity; ATR: attrition; G/M: generalization and/or maintenance; ES: effect size; SV: social validity; H: high quality; A: acceptable quality; U: unacceptable quality; Y: there is evidence; N: there is no evidence. Rating form adapted from (Reichow 2011) (p. 38).

Disclosure statement

No potential conflict of interest was reported by the authors.