Virtual Reality Air Travel Training Using Apple iPhone X and Google Cardboard: A Feasibility Report with Autistic Adolescents and Adults

Abstract

Rapid rises in autism diagnoses are increasing the demand for effective services and straining service providers. When individuals on the autism spectrum turn 18, they are faced with even greater barriers to services, since many services are delivered in school settings. Thus, there is a need for more accessible tools that teach daily life and communication skills to autistic adolescents and young adults. The current project reports findings from a pilot study using virtual reality (VR) to teach air travel skills to autistic young adults. The authors repurposed a virtual airport environment previously used to treat fear of flying for this study. Seven participants on the autism spectrum viewed a 5-minute virtual airport simulation with an overlaid narrative script using an iPhone X^® and Google Cardboard^® device once per week for 3 weeks. Researchers collected measures of attentiveness, language function, activity comprehension, and clinical observations on how participants interacted with the technology. Analyses revealed improvements in attentiveness, certain language functions such as labeling vocabulary, and activity comprehension in most participants. Clinical observations revealed acceptability of this technology and its capability to serve as an appealing media to deliver interventions. Thus, it is feasible to apply mobile VR trainings with autistic adolescents and young adults. We discuss ways to improve the pedagogical approach of VR-enhanced interventions in light of these findings. In the future, we plan to develop and test more virtual environments that address the needs of young adults on the autism spectrum, such as interview training and independent living skills.

Lay summary

Why was this virtual reality air travel training program developed?

Each year, more and more people travel via airplane. Long lines, unexpected changes and other stressors can make air travel overwhelming and difficult, particularly for autistic people. Our goal was to develop and pilot test a program to help autistic adults learn air travel skills without needing to physically enter an airport.

What does the virtual reality air travel training program do?

Our virtual reality air travel training (VR-ATT) program presents a virtual simulation of the steps that travelers go through in an airport. It guides users through entering, checking in, navigating security, waiting at the departure gate, and boarding. VR-ATT also contains a narrative to guide users by highlighting important information such as: “Look, there's the ticket counter. Let's check in with the attendant.” This script was based on social stories, which are often used to help individuals on the autism spectrum learn communication and social interaction strategies. Seven autistic adolescents and adults participated in our program. They watched the VR-ATT simulation on an iPhone X^® and Google Cardboard^® two to three times over the course of 3 weeks. Google Cardboard is an inexpensive virtual reality (VR) headset.

How did the researchers evaluate the VR-ATT program?

We evaluated the program in three ways. First, we asked each participant to retell the sequence of events of the simulation. In other research studies, participants who retold what they watched showed a greater ability to apply the skill to the real world. Second, we tracked how and what types of language participants used. Third, we measured if participants could independently view the VR-ATT simulation without side effects such as dizziness or cybersickness (common side effects of VR viewing). This helps us understand the feasibility and acceptability of using VR with autistic individuals.

What were the early findings?

Early findings suggest that autistic individuals are accepting of the iPhone and Google Cardboard VR viewing method. They also provide preliminary support for the ability of the program to promote improvements in functional language skills such as labeling vocabulary, which helps individuals interact and navigate busy environments, such as an airport. Finally, the findings suggest that participants can accurately retell the sequence of events in the virtual simulation.

What were the weaknesses of this project?

One weakness was that researchers could not test how participants transferred the skills they learned in the virtual training to a real-world airport. Another weakness is the small sample of participants. Finally, not every participant completed all three training sessions, making it difficult to draw precise conclusions about the program's acceptability and success.

What are the next steps?

Future studies should include a larger number of participants, a real-world test, changes to the virtual simulation to encourage more language use and interaction with virtual characters, and use more accurate measurement materials.

How will this work help autistic adults now or in the future?

This air travel program may someday help autistic people learn what to expect and how to communicate in a busy airport setting. This program also highlights that new technology, such as VR, could potentially improve access to services and help more people in need in the future.

Introduction

Greater awareness of autism and changes to diagnostic criteria are contributing to rapid increases in diagnoses across the United States (from 1 in 150 in 2000 to 1 in 49 in 2016). However, limited funding and other resources strain access to services that contribute to independent and fulfilling lives.^1–6 Researchers estimate that 500,000 autistic individuals will turn 18 in the coming decade and age out of school-based services, cutting access to therapies, case management, vocational preparation, and other daily living services by nearly 60%.⁷

Mobile technologies (i.e., smartphones, tablets, and other devices) help mitigate service shortages by providing remote trainings and services on demand.^8–10 Take video modeling (VM), for example, viewed easily on mobile phones or tablets, these short videos model behavior, communication, and other processes that can help autistic individuals learn how to navigate complex social environments. Researchers at Queen's University Belfast report that 20 autistic participants aged 3–16 years successfully acquired and generalized air travel skills after four VM sessions at home with their parents.¹¹ In fact, VM improves skill acquisition faster than in vivo (real life) modeling and promotes generalization in autistic populations for three primary reasons: (1) videos can emulate naturalistic settings that are difficult to create in vivo; (2) VM allows for control and customization of training environments; and (3) videos can be conveniently and repeatedly replayed.^12,13

While VM offers advantages over in vivo learning, technological developments and emerging research suggest that virtual reality (VR) provides even greater skill acquisition and generalization effects. Today, researchers investigate applications of VR to help autistic people learn various daily living skills.^14–16 Like VM, VR delivers repeatable, controllable, and safe practice environments that help autistic individuals practice new skills safely and efficiently.

A key distinction, however, is that VR displays a digitally constructed environment, allowing for greater control over the processes and presence of stimuli at any given time. Self et al., whose between-group comparison of eight children on the autism spectrum reveals advantages of skill acquisition using VR to teach safety skills,¹⁷ referenced this advantage. While participants of VM and VR groups displayed generalization and maintenance of the targeted safety skills, recipients of VR training required half of the training time to acquire skills compared with their peers in the VM group.¹⁷ The authors suggest that the ability of children to view multiple viewpoints of a situation in VR as opposed to a single viewpoint using static video produced this effect.¹⁷

Other researchers corroborate the malleability of VR and its success in skill training applications with autistic populations, too.¹⁸ Two studies in particular demonstrate how VR facilitates safe practice environments and improve autistic adults' pedestrian safety skills.^19,20 Smith et al. cited the ability to individualize trainings as a key variable in the success of VR interview trainings for autistic adults.²¹ Simões et al. elucidated this notion of individualization by applying wearable sensors to monitor the physiology of 10 autistic people while practicing bus riding skills in VR.²² In this study, as physiology stabilizes (i.e., lower heart rate and electrodermal activity), new stimuli (i.e., people, sounds, colors) are added to the practice environment. This “VR-graded exposure” technique—gradually adding stimuli as physiology stabilizes—significantly improved bus riding skills.²² It is for these reasons, according to Maskey et al., VR-graded exposure is now applied in the treatment of specific phobias experienced by autistic individuals.²³

Each of the selected studies above supports VR as a tool to mitigate increasing autism service shortages. For instance, Self et al. observed accelerated skill acquisition in VR over VM and decreased production time for training materials.¹⁷ Others underscore the convenience and effectiveness of real-time modification of virtual environments.^21,22 All these authors, in fact, agree that VR is efficacious because appearances, events, and other stimuli in a training environment can be manipulated to fit the needs of each participant.^17,19–23 This is particularly intriguing as it relates to the unique and varied ways that autistic people receive, process, and compartmentalize contextual stimuli in the brain.

Executive functions such as decoding and grouping information into relevant schemas are important for navigating social environments and interactions but can be challenging for autistic people. Sinha et al. suggested that while processing centers in the brain receive all the environmental inputs, autistic individuals have difficulty compartmentalizing that information into comprehensive schemas.²⁴ Difficulties discerning salient contextual information increase the likelihood that environmental stimuli “flood” sensory processing centers and elicit stress or anxiety.²⁴ In light of these processing challenges, VR practice environments may be beneficial for autistic people as they provide safer, more controlled, and often more effective environments to learn new skills than learning skills in vivo or using VM.²⁸ Moreover, navigating complex social situations in overwhelming sensory environments, such as those encountered in airports or health care settings, may be particularly stressful or anxiety provoking for autistic people. During these situations, autistic individuals often exhibit difficulties in adaptive verbal and nonverbal communication, and states of stress and anxiety may be communicated behaviorally. Research suggests that functional communication training can help mitigate these behavioral responses.^24–27 Thus, in this study, we investigate the feasibility of delivering VR-based functional communication training to prepare autistic individuals for a highly stimulating environment—an airport.

The number of passengers traveling by air has increased 137% annually since 2004; it was expected that 2020 would see 4.72 billion air travelers worldwide, although the situation surrounding COVID-19 has altered those projections.^29,30 While air travel is increasing in popularity, it can be difficult for autistic people who find communication and social interaction challenging, experience sensory sensitivities or aversions to change, or have difficulty in navigating dynamic environments.^25,31 Every year, agencies and organizations partner to host events for autistic individuals and their families to experience a “dry-run” air travel experience—taking them through check-in to boarding the airplane.²⁵ These events are in high demand across the United States, but limited in their availability—a reflection of services for autistic individuals in general^2–7—heightening calls for new ways to teach air travel skills to autistic people. Thus, we applied VR-based functional communication training in this study to help autistic participants learn how to communicate in and navigate an airport setting. Functional communication training is based on work by Prizant and Wetherby^26,27 and stipulates that learning verbal and/or nonverbal communication strategies, such as asking for help, labeling actions, or identifying objects can help individuals process and navigate social environments.^24,32 Our previous reports further elucidate how basic communication strategies such as asking for help and labeling actions improve an autistic individual's ability to contextualize and navigate a social environment, such as an airport.^32–35 However, these works reported findings from autistic children; at the time of this study, there are no VR-based air travel training programs available for adolescents or adults. Thus, the current report investigates the feasibility of this application with autistic adolescents and young adults. Also this emerging practice differs from our previous reports as it is designed to specifically meet the needs of adolescents and young adults in airport settings, rather than children who are often accompanied by a guardian.

The Virtual Reality Air Travel Training Program

Description of the virtual reality air travel training program

We developed a virtual reality air travel training (VR-ATT) program for use by autistic adolescents and young adults. The program aims to help autistic individuals learn air travel and related functional communication skills by presenting a narrated virtual tour of an airport. The program intends to facilitate verbal and nonverbal communication strategies that help autistic individuals navigate a busy airport environment. The VR-ATT program included one 20-minute session per week for 3 weeks.

Equipment

There are various VR modalities available as cited in the literature.¹⁸ First, there are computer-linked head-mounted displays—high-end computer systems linked to large goggles.^15,17 Next, there are stand-alone VR headsets, such as the Oculus Quest, which negate the need for computers and house all VR-related hardware and software inside a head-mounted display.¹⁴ Finally, there are mobile VR options that utilize smartphones and small lightweight goggles to display virtual environments.¹⁶ Mobile VR, such as the Google Cardboard, is much cheaper than more complex computer-linked systems.¹⁶ Thus, to address increasing demand for accessible tools and services, the current intervention utilizes an iPhone X^® and Google Cardboard^® VR viewer to display the virtual airport simulation.^32,33,35,36 The current study adopts the VR materials applied in preceding reports, which outline the acceptability of mobile VR in younger autistic populations.^32,33,35,36

Virtual environment design

We used a preexisting virtual airport simulation designed, developed, and clinically validated to treat fear of flying in children, adolescents, and adults by Wiederhold and colleagues.³⁷ Figure 1 depicts the display devices, style of viewing, and training environment used in this study. We made a number of modifications to the environment according to recommendations set forth in previous pilot study reports with autistic children^32,33,35,36 First, a narrated script was pre-recorded and overlaid onto the virtual training scenario to guide participants through four “checkpoints”: the ticket counter, security, waiting at the gate, and boarding the airplane. This script was based on social stories,³⁴ which are often used to help individuals on the autism spectrum learn communication and social interaction strategies. The second modification was the inclusion of prompted questions from avatars to elicit more engagement and interaction at the check-in and security checkpoints. Finally, replicating the skill acquisition measure of Simões et al.,²² we added a post-activity comprehension test. This test was omitted from our previous VR-ATT reports in autistic children (aged 5–9 years) because they exhibited limited verbal and nonverbal communication strategies.^35,36 Our speech–language therapist deemed participants of the current study of the appropriate age to answer such questions.

FIG. 1.

(A) Equipment and materials, including Google Cardboard^®, Communication Board, and Apple iPad^® for voice recording. (B) Participant viewing VR-ATT in Google Cardboard. (C, D) Screenshots of the VR-ATT simulation. VR-ATT, virtual reality air travel training.

VR-ATT sessions

All sessions began with a brief (∼2 minutes) warm-up period where a speech–language therapist introduced the activity using props and pictures. She said things such as “we're going to watch a video about going to the airport today,” “have you ever been to the airport?” and outlined the steps of the session (viewing the video, answering questions, and giving feedback).

Next, participants were asked to hold the VR headset up to their eyes to view the simulation. Scaffolding was in place to ensure that the participant viewed the entire duration of the 5-minute simulation, within recommended guidelines.^38,39 Each session was audio-recorded to collect communication samples and feedback on the feasibility of the intervention. During each session, participants watched the fixed-path airport simulation in VR as the research staff observed and collected measures. At the end of each session, researchers administered the comprehension test, prompting participants to answer questions related to the air travel processes they just viewed.

Evaluation Methods

We conducted a pilot study of our VR-ATT with seven autistic adolescents and young adults. Research staff visited the therapy center multiple times per week for 4 weeks. Each session was attended by two research staff, a participant, and an aide from the therapy center. This study was approved by an Independent Review Board.

Participants

Convenience sampling from a therapy center in Southern California yielded seven individuals (six males and one female, mean age 18.28 years, range 10–22 years) diagnosed with autism spectrum disorder (as corroborated by clinical records). Six were between the chronological ages of 16–22 years and one was aged 10 years (Table 1). The exclusion criteria were history of seizures or coexisting serious health conditions such as epilepsy, cancer, or the presence of a pacemaker. References to gender have been removed from any case descriptions to provide anonymity. All individuals participated in this trial with full informed consent from legal guardians and under their own assent.

Table 1.

Participant Age and Number of Training Sessions Received

Participant	Age, years	Sessions
a	22	2
b	21	2
c	22	3
d	16	3
e	10	3
f	19	3
g	18	3

Our research objectives were:

• RO1: To establish feasibility of delivering interventions using inexpensive, accessible, and easy to use VR platforms.

• RO2: To assess air travel skill acquisition following VR-ATT.

• RO3: To assess changes in communication skills following VR-ATT.

Each evaluation method is listed in accordance with the three research objectives stated above.

RO1: establish feasibility of Google Cardboard VR delivery

We observed whether or not participants attended to the VR simulation to measure acceptance of this technology by autistic adolescents and adults. We determined attention to the VR simulation as holding the headset up to their eyes for the full video duration with less than three reminders. Also, halfway through each session, we conducted a “check in” with participants where we asked them if they felt dizzy, sick, or uncomfortable (adverse reactions to VR). We also conducted a “check in” at the end of the simulation.

RO2: assess air travel skill acquisition

Due to logistical constraints, we did not include a final real-world generalization test at the local airport, a departure from previous reports.^33–35 Instead, skill acquisition was measured via an activity comprehension post-test, asking participants to retell the series of checkpoints they had just viewed in VR. This test was based on a similar approach used by Simões et al.²² Participants were asked to retell the sequence of events they just watched in the VR headset. In this comprehension test, researchers prompted participants with five questions asking about the sequence of checkpoints (check-in, security, gate, boarding) and types of actions associated with each checkpoint (get ticket, put backpack on security belt, wait at gate, board airplane, and find a seat). Answers were coded either 2 or 0. An answer was scored 2 if a participant answered correctly. An answer was scored 0 if a participant answered incorrectly or did not answer. The style in which each question was answered (i.e., verbally, nonverbally) was recorded and evaluated as part of RO3. Simões et al.²² demonstrated that improved comprehension leads to greater skill transfer in the real world. To achieve a preliminary understanding of the relationship between vocabulary use and skill acquisition—and in the absence of a real-world transfer test—we calculated the Pearson correlation coefficient between vocabulary use and comprehension. This calculation helps elucidate any interaction between the two variables for further investigation.

RO3: assess changes in communication skills

Communication skills were assessed two ways. First by tracking changes in activity-specific vocabulary, which was broken down into three language functions: labeling, action, and drawing attention. These are both functional and socially appropriate to the context. A sample of the activity-specific vocabulary is shown in Table 2. A simple tally tracked the participants' language use throughout each session. Nonverbal participants used communication boards to produce communication targets. The second approach coded participants' verbal and nonverbal responses to the comprehension test administered for RO2. Researchers coded answers to each question for level of communicative response (level 0, 1, or 2). Participants scored 2 when they independently generated the answers (verbally or via the use of a communication board). Alternatively, the clinician used a carrier phrase to scaffold the participant's answer if the participant was delayed in generating a response. Verbal and nonverbal answers generated via carrier phrase were scored 1 point. No answer scored 0. Here, the type of communication style was independent of the correctness of the answer to the comprehension test (as measured in RO2). That is, a participant could score high on communication measures while scoring low on the comprehension test.

Table 2.

Targeted Language Function Examples

Labels	Action	Draw attention
Ticket	Let's go	Goodbye
Airplane	Walk	Hello
Backpack	Wait	Help
Worker	Sit down	Look
Security	Listen	Where

Preliminary Results

Five of the seven total participants received three training sessions. The remaining two participated in only two sessions for scheduling reasons. Table 1 breaks down the number of training sessions by participant and age.

RO1

Behavioral observations revealed all participants fully attended to the virtual simulation—a key metric of acceptability.^39,42 Participants independently held the Google Cardboard device and reported no adverse effects during the check-in point or after the intervention. Clinicians noted participants verbally responded to bids for communication from the avatars in the virtual environment. Participants also responded to the social narrative script or imitated its direction. For example, once seated in the airplane, the script says, “I can watch a movie, listen to music, or practice my relaxed breathing” to which one participant said, “That's a good idea!” Some participants attempted to physically reach out and touch an item in the virtual world, speak to avatars, and describe what they were seeing.

RO2

Most participants displayed improvements in the comprehension test. Figure 2 displays scores from the first and last sessions for all but one participant. Participant “b” did not complete the comprehension test after either of their two sessions. Cumulative change scores indicate a 10% overall improvement from the first session to the last session. Two participants scored lower on the final test than the first and four participants exhibited improvements. A moderate positive correlation between vocabulary and comprehension scores was observed (r = 0.38), although nonsignificant (p > 0.05).

FIG. 2.

Changes in comprehension test scores for each participant from the first session to the last session. *Participant completed two of the three training sessions. Participant “b” did not complete the comprehension test.

RO3

Of the three language functions, labeling was the most utilized and the most improved throughout the intervention. Action and drawing attention vocabulary were rarely used and saw minimal change. Figure 3 presents changes in labeling, action, and drawing attention vocabulary for each participant from the first session to the last session. Labeling vocabulary increased 38% on average from the first session to the last session across all participants. All but one participant experienced a 2- to 4-word improvement. The number of action vocabulary words used was low across participants pre- and post-intervention, and vocabulary used to draw attention was used only once by two participants across sessions. While participants showed improvements in activity comprehension (as measured by RO2), their communicative responses to those questions (RO3) showed little improvement. That is, while participants could retell the sequence of events, their response styles (i.e., independent, scaffolded, or no response) did not change over time.

FIG. 3.

Changes in language function (labels, actions, draw attention) from the first session to the last session for each participant. Note: Missing columns indicate zero value.

Discussion

Overall, the data presented in this report are encouraging for the feasibility of delivering Google Cardboard VR-based training to autistic adolescents and adults. There is also preliminary support for this specific module to teach air travel and communication skills. Participants' interactions with avatars suggest that inexpensive VR, such as Google Cardboard, can elicit feelings of being in a virtual world, or presence. According to Bailey and Bailenson,⁴⁰ underdeveloped executive functions in children or developmentally delayed individuals inhibit the distinctions between real and virtual, making VR experiences more vivid and real than things such as pictures or TV screens. This can make children and developmentally delayed individuals more likely to be influenced in positive ways (e.g., prosocial education) by VR.⁴⁰ While Newbutt et al.³⁹ suggested autistic students prefer high-end VR headsets (i.e., HTC Vive or Oculus Quest) over less advanced versions (i.e., Google Cardboard), and most of the current research utilizes high-end systems, the current study highlights that some skill acquisition and communication can be achieved with low-cost materials.^15–23 Devices such as Google Cardboard are also intriguing as they are less cumbersome and obtrusive, allowing for more natural and functional therapist–client interactions.^32,33,36,39 Moreover, the application of inexpensive and low-tech VR tools in therapeutic and home settings is more feasible and directly impactful than higher cost computer-linked head-mounted displays, and this inexpensive technology is “just good enough to imprint a new paradigm.”⁴⁰

Results from the comprehension test are also encouraging. Four of the six participants who completed the test displayed at least a one-point improvement in retell. Although the current study did not test the real-world skill transfer of participants, it is expected that improvements in comprehension of the VR activity translate to improvements in real-world activity completion, as evidenced by Simões et al.²² and previous VR-ATT reports.^33,35,36

Evaluation data revealed modest improvements in activity-specific labels and nearly no changes in action or drawing attention vocabulary. Participants who received all three training sessions showed more improvement in activity-specific labels than the participants who received only two trainings. While labeling vocabulary increased, functional communication and generalization of skills to the real world require more complex communication behaviors, such as actions (Let's go!) and drawing attention (look, help me). It is important to adapt the narrative script to elicit language function such as action and drawing attention, both during the VR simulation and comprehension activity. To accomplish this, we may adapt the virtual training scenario to provide more interaction verbally (i.e., more questions from avatars) and physically (i.e., manipulating virtual objects with controllers or other inputs). Moreover, while participants completed the retell test, minimal changes were observed in their response types (i.e., independent, scaffolded by therapist, or no response). These nuanced communication styles often take longer than 3 weeks (intervention length) to change.⁴¹

Limitations

While the current findings are encouraging for the feasibility of inexpensive VR interventions with autistic individuals, limitations remain. First, the conclusions drawn from current data are limited, as this is not a study of efficacy, but feasibility. Second is the absence of a real-world training transfer test. This is integral to demonstrating effective skill transfer as previous reports of this training method have accomplished.^33,35,36 The study is also constrained by its sample size (n = 7), a common limitation of current research investigating VR applications with autistic populations.¹⁴ Multiple reviews touch on this, noting that in order for VR to reach its full potential more robust empirical research is needed.^14,28,42 Future trials should also include validated research measures, although the nascence of this specific field makes it difficult to find validated VR-specific measurement materials, especially those related to communication and autism-specific behaviors.

Conclusions

The current report highlights the feasibility of inexpensive VR headsets and helps pave the way for future research. For example, research that previously examined high-end head mounted displays (HMDs) can be revisited and duplicated with an arm of the study applying inexpensive VR. This is useful in understanding whether Google Cardboard-like devices offer the same advantages over traditional methods that more expensive systems do, such as faster training times¹⁷ and control over environmental stimuli.^21–23 Studies comparing VM with inexpensive VR, then, are also an important next step. While current literature displays advantages of high-end VR headsets over non-VR groups,^15–23 applications of Google Cardboard or similar products are absent. These types of duplication studies and revisions might help parse out the differences between high-tech (e.g., HMD) and low-tech (e.g., Google Cardboard) systems.

Overall, it is encouraging to see that participants were positively engaged and willing to wear the Google Cardboard, indicating feasibility for more rigorous data collection in future studies. Mobile VR, such as Google Cardboard, is feasible, inexpensive, easy to use, and shows promise as a therapeutic tool to improve access to a host of training services. As autism diagnosis rates steadily increase, it is important that effective and accessible tools permeate into the homes of autistic individuals and their families. This report suggests that such systems are capable of delivering specific skill trainings (i.e., walking through the airport) and facilitating social communication, although more robust research is required.

Authorship Confirmation Statement

This research and compilation of this article were completed with contributions from all authors. As first author, I.T.M. conducted research trials, analyzed data conducted literature searches, and prepared the first and final drafts of the article. C.S.M. also conducted research trials, analyzed data, and contributed to the methods and conclusions sections of this article. M.D.W. contributed to the methodology, results, and discussion sections, while also overseeing the completion of the article from the beginning to the end. As final author, B.K.W. contributed to the study design, literature searches, introduction and discussion sections, while also editing of the article throughout all stages. All co-authors have reviewed and approved the article before submission, and the article has been submitted solely to this journal and is not published, in press, or submitted elsewhere.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

Funding for this research was generously provided by the Northeast Arc ‘Arc Tank 2.0’ competition.