An Evolution of Virtual Reality Training Designs for Children With Autism and Fetal Alcohol Spectrum Disorders

Abstract

This article describes an evolution of training programs to use first-person interaction in virtual reality (VR) situations to teach safety skills to children with autism spectrum disorder (ASD) and fetal alcohol spectrum disorder (FASD). Multiple VR programs for children aged 2 to 9 were built and tested between 1992 and 2007. Based on these results, a learning design evolved that uses practice in virtual space with guidance and correction by an animated character, strategic limitations on allowed actions to force correct patterning, and customization of worlds and responses to simplify user controls. This article describes program evolution by comparing design details and results as variations in behavioral responses between disorders, differences in skill set complexity between different safety skills being taught, and improved technology required changes in the virtual training methodology. A series of research projects are summarized in which the VR programs proved effective for teaching children with ASD and FASD new skills in the virtual space and, where measured, most children generalized the actions to the real world.

Parents and educators often have concerns about teaching children with special needs how to respond appropriately when in new situations. For critical safety dangers, practice can be problematic. Both autism spectrum disorder (ASD) and fetal alcohol spectrum disorder (FASD) are defined as disorders that include multiple subcategories with varying deficits and severities. This article describes our research at Do2Learn in partnership with North Carolina State University, the University of North Carolina at Chapel Hill, and Emory University using virtual reality (VR) to train children with ASD and FASD to handle two safety dangers: crossing a street in traffic and exiting a home when fire is present. Our latest research, which involves an ongoing design to teach children to recognize safe boundaries, is also described.

SAFETY CONCERNS FOR CHILDREN WITH ASD AND FASD

In the United States, unintentional injuries are the leading cause of death for children (Runyan et al., 2005). Such accidents also result in disabling injuries of many kinds, and for these reasons are a major public heath concern. In addition to traffic accidents, injuries as the result of residential fires and burns or street accidents involving pedestrians are among the most common (Istre, McCoy, Carlin, & McClain, 2002; Runyan et al. 2005). Fires and burns in the home are the leading cause of unintentional death among children aged 1 to 14 years (Runyan et al., 2005). Approximately one fifth of the almost 5,000 individuals reported killed in motor vehicle crashes involving pedestrians in the United States in 2003 were in the 0–14 age group (National Highway Traffic Safety Administration [NHTSA], 2004). In addition to recommendations to parents, school personnel, and other adults designed to avoid preventable accidents, agencies such as the NHTSA recommend that children be taught safety skills consistent with their developmental level (Garbarino, 1988). Preschool- and school-aged children should learn, at home and at school, age-appropriate methods to prevent or avoid injuries (Jones, Kazdin, & Haney, 1981).

Those children with preexisting disabilities and behavior problems are at even greater risk for injuries (Injury Prevention for Children with Special Health Care Needs Work Group, 1999; Leland, Garrard, & Smith, 1994; Sherrard, Ozanne-Smith, & Staines, 2004). Children with cognitive limitations and behavioral problems are more than twice as likely to die in a fire or other accident than typical children (Injury Prevention for Children with Special Healthcare Needs Work Group & United States Fire Administration [USFA], 2006).

Unfortunately, children with disabilities may also be less able to understand and benefit from prevention information. Thus, it is important to develop prevention and intervention methods that are effective with this vulnerable group of children. Children with ASD and FASD both experience learning problems that make prevention training more difficult (American Psychiatric Association, 2000) and put them at greater danger. ASD consist of pervasive developmental disorders characterized by severe impairment in cognitive, communicative, social, and behavioral functioning (Courchesne, 1989; Koegel, Koegel, & O’Neill, 1989; Mesibov, Schopler, & Kearsey, 1994; National Institutes of Mental Health, 2006). FASD involves a series of irreversible birth defects that can include mental retardation, growth deficiencies, central nervous system dysfunction, craniofacial abnormalities, and behavioral maladjustments (Coles et al., 1991, 1997; Mattson et al., 1996; Stratton, Howe, & Battaglia, 1996). ASD and FASD are irreversible and persist throughout the lifespan.

Although there are inconsistent profiles across individuals diagnosed with ASD and FASD, commonly shared traits involved in both disorders include abnormal response to input stimuli, difficulty with social engagement, and the inability to generalize between environments. Communication weaknesses identified in both disorders can interfere with verbal attempts by parents and teachers to explain dangers or proper actions and may limit the ability to seek help. Behavioral and sensory distortions can result in a child approaching danger, hiding, or taking other dangerous actions in response to auditory alarms or visual warnings. Social interaction and judgment deficits can lead a child to miss normal warning cues and to react inappropriately to people attempting to assist. Rigidity and compulsive behavior can make it difficult to respond spontaneously to new situations and to generalize safety concepts across different danger situations. On the basis of these shared traits, in the research trials, we worked with children across disorders and severities within the two spectrums to determine whether using a practice space offered by VR technology would be of benefit when learning new safety skills by either or both groups.

Virtual reality is a method for creating the illusion of presence in a computer-generated environment. The environment can be represented on a desktop computer screen or a projection screen. In the classic example, the real world may be occluded with an immersive headset that contains two computer scenes that combine to create a stereo image. As the individual moves with either body motion or a more standard input, such as a mouse, his or her position is tracked and the computer scenes are changed to give the illusion of interaction with the imaginary world.

Researchers from a variety of fields, including psychology, sociology, medicine, engineering, computer science, and education, have verified the efficacy of VR systems for psychoneurological assessment, rehabilitation, retraining, and adaptation to a range of mental and physical impairments (McComas, MacKay, & Pivik, 2002; Rizzo, 2007; Strickland, 1997). An Institute of Defense Analysis report on the effectiveness of educational uses of VR technology determined that VR instruction was educationally effective and highly motivating (Youngblut, 1998). Virtual worlds used with children with special needs include Active Worlds, Digitalspace, and Brigadoon, a world for children with Asperger’s syndrome (Damer, Gold, & de Bruin, 1999; Farkin et al., 2005). This control and interaction has been documented to improve learning skills among individuals with mental and physical disorders (Foreman, Wilson, & Stanton, 1997). Single-user virtual practice environments have improved learning skills among individuals with mental and physical disorders (Coles, Strickland, Padgett, & Belmoff, in press; Foreman et al., 1997). Adolescents with autism used virtual environments in research studies for social actions on par with matched nonautistic counterparts, and understood virtual environments as a representation of reality (Parsons, Mitchell, & Leonard, 2004; Rutten et al., 2003).

Virtual reality’s benefits in addressing ASD and FASD common learning problems included repeatable, consistent practice with predictable responses and clear, on-screen guidance. Less traumatic and safer learning situations, control of input stimuli and complexity to help with appropriate task focus, and strong engagement value were effective as a teaching format within the virtual scenes we used. In our VR lessons, major issues involved the design of the VR environment, including training procedures, provision of adequate user controls, and appropriate user feedback. Some solutions were highly user dependent and required customization for the difference between learning styles of children with ASD and FASD, but others were effective in all our studies for children with either disorder.

AN EVOLUTION OF STUDY METHODS AND RESULTS OF VR LESSONS

Since 1992, we have experimented with different VR systems to evaluate the effectiveness of the technology as a teaching tool for children with ASD and FASD. This article discusses five elements in the changing design of our program for teaching safety: (1) a customized headset-based VR hardware system for street crossing with children with ASD; (2) a flat-screen, personal computer (PC)-based fire safety program for children with ASD; (3) several flat-screen, PC-based fire and street safety programs for children with FASD; (4) several Web-delivered fire and street safety programs for general public use; and (5) CD-based commercial game software to explain a safe yard boundary concept. Each evolutionary step occurred as either technology changed significantly or we attempted to extend the lessons from children with one disorder, ASD, to another disorder, FASD. In this overview, differences between the VR programs are organized for comparison of the design, trial setting, instruction, user control, customization, and results with progressions in both time and technology.

Immersive street safety VR for children with ASD

Design

The original study was an attempt to determine whether children with ASD would use VR equipment and whether they could learn when in a virtual space (Strickland, Marcus, Mesibov, & Hogan, 1996). It consisted of an immersive ProVision 100 fully integrated VR system with head-mounted display, body tracker, and three-dimensional hand controls (Figure 1). Participants were two moderate-level functioning children, one girl and one boy, aged 7 and 9, who met the classification for autism, scoring 36.5 and 34, respectively, on the Childhood Autism Rating Scale (Schopler, Reichler, & Renner, 1985). These two children were placed in a street crossing virtual scene to learn two basic steps of stopping at a stop sign and tracking moving cars before crossing a street. The two children came in separately for approximately 30-min to 1-hr sessions over a 5-week period. VR training for each child was limited to approximately twenty 3- to 5-min periods during the 5 weeks, with familiar play and work routine intermixed between VR exposures during each session.

Figure 1.

Immersive street crossing with child with autism spectrum disorder.

Trial setting

Because both children displayed low tolerance for new situations, we attempted to mirror classroom settings in the play portions of the study by including activities borrowed from their schools, such as coloring books and puzzles. We also incorporated the picture card scheduling and visual communications each child was familiar with in home and in school. Details included dividing rooms into separate areas that duplicated classroom centers, including centers labeled for “play,” “computer,” and “work.” When PC-based virtual systems became available in later trials, tests were moved to settings that were already familiar to each child. Anticipating that these children with ASD might have difficulty adapting to unfamiliar people and activities, we included older siblings as guides to demonstrate the headset first. Duplicating a known pattern of space and people that would place the VR module within a context of familiarity was an underlying testing parameter that we have included in all research trials.

Instruction

Because neither child had a speaking vocabulary of more than a few words, communication about how to use the technology became an immediate problem. Giving verbal instructions to the children while they were using the headset was not particularly successful. We immediately started using physical manipulation to show actions, such as having a parent initially turn the child’s head to find a car on the street. For the child with more severe difficulty, we had to hold him at the stop sign multiple times to convey the action of stopping. This child showed confusion adjusting to the interface of effects on the virtual world visual overlay created by moving in the physical world; his limited vocabulary precluded our ability to explain the concept of a virtual space. He always started his lesson by searching for his feet in his virtual body, and when asked to point to the virtual stop sign, would point with his finger to the sign in his headset. The other child appeared more comfortable and pointed within the virtual space. We limited headset exposure to no more than 5 min at a time because of the inability of the children to provide verbal feedback of any discomfort and the difficulty of providing verbal instruction when the children were in the headset.

User controls

Headset tracking monitored the child’s walking and head turning motions in the virtual space. A three-dimensional joystick allowed more sophisticated actions, such as moving all the way across the street by pressing a button rather than taking the steps required to cross a street in real space. Neither child used the three-dimensional joystick comfortably, although the child who grasped the concept of pointing in virtual space also learned to press a joystick button to walk across the street. User control in the virtual space consisted primarily of the physical actions—walking, stopping, head turning, and pointing. The child who adjusted to the virtual space most comfortably created safety concerns by moving quickly around in the virtual world, oblivious to the cables tethering the headset to the computer. Based on these use issues, coupled with the hardware complexity of this interface, user controls evolved in all other studies to consist of a standard computer equipment such as a monitor, mouse, and keyboard.

Customization

Scene complexity details such as car colors were individually adjusted on the basis of the limited color palette known by each child. In later trials, we continued to increase options to customize scenes details (e.g., number of objects in space; smoke accompanying fire), instructional guides (e.g., child, dog, or rabbit to demonstrate action; text tips on screen), sounds (e.g., cars have sounds; background music), controls (e.g., option to do alone or have demonstration; game pad, mouse, or keyboard input accepted), and other program parameters (e.g., only correct actions allowed; play repeated or difficulty advanced). The number of options reached the largest variation in our first FASD trials.

Headsets presented issues that are unique as a user interface. In this first trial, we made subtle adjustments in areas that we felt were critical for the children’s acceptance of the headset, including placing each child at his or her normal real-world height level when in the virtual world and adjusting headset focus to the eye separation distance of the smaller faces of the children.

Results

Before exposure, neither child displayed awareness of street boundaries or demonstrated normal safety actions such as tracking cars moving on a street while standing at a street curb. While in the virtual environments, both children repeatedly immersed themselves in the three different street scenes to a degree that they verbally labeled objects and colors of objects, moved their bodies in response to actions within the worlds, such as tracking cars, and located a stop sign in a virtual scene and walked to it and stopped (Strickland et al., 1996). The recognition of actions associated with these limited street safety skills were new behaviors for both children. No attempt was made to measure generalization of these new actions to the real world, although the parents of the girl indicated that she did track moving cars with her head after VR training.

PC-based fire safety VR for children with ASD

The next transition occurred when the availability of PC-based VR system in 1997 provided the opportunity to determine whether the street safety lesson used with the $100,000 VR equipment in the first trial could be duplicated on a low-cost home computer.

Design

This PC-based study investigated whether VR could be used to teach home fire safety skills to children with pervasive developmental delays (PDD), a term that is inclusive of ASD (American Psychiatric Association, 2000), and if the skills learned on the computer would generalize to the real world. Fourteen children aged 3 to 6 years took part in either a home-based or school-based research trial in which each child was shown the USFA recommended safety actions. These include exiting from a fire danger zone by the shortest safe route and waiting at a predetermined meeting place outside of the home (USFA, 2005).

The school-based part of the study was set up at the rear of the room in a public school prekindergarten program for children identified with PDD in a midsize US metropolitan city. Testing was conducted over several months during regular classroom daytime activities. The home-based part of the study took place in the children’s private residences where researchers set up a computer in an area suggested by the parent, usually in the kitchen or in the breakfast or living room. Testing and training took place in a 1-hr session for children in the home-based environment and in multiple 10-min sessions at school. Home-trained children were tested for generalization to a real meeting place at their home. Because children exiting a fire in a school normally follow a different procedure, generalization of home exit steps was not tested in the school-based trials. However, students at school were asked to “show and tell” how to respond to a fire at home at the conclusion of the computer-based probe. The school-trained children were tested for retention of skills a week later.

A single-subject changing criterion design was used to evaluate effects of the training program individually for each child. Students were introduced individually to a series of fire safety skills arranged in a sequential order based on task analysis, beginning with the basic terminology of recognizing fire, smoke, and meeting place, and progressing to crawling low under smoke, exiting the house to a safe meeting place without assistance, and waiting at the meeting place. Following a baseline for each skill, students were trained to a level of mastery. All children were given a pretest to assess their knowledge level before beginning. Training about any objects and actions unknown in the pretest, such as the term “meeting place,” consisted of teaching with a computer program in a multiple-choice game. Navigation within the virtual space was taught by practice moving around a six-room virtual house. After demonstrating user-control competency, the USFA recommended home fire safety steps of recognizing a fire danger, leaving a house by the shortest route when fire or smoke was present, and waiting in a predetermined waiting place were practiced to mastery using the VR program.

Correct responses were reinforced visually and verbally by the computer on the basis of the child’s actions in the virtual world. An animated character (avatar) named Buddy demonstrated proper actions and continually interacted with the child (Figure 2). Unlike the initial headset study, assistance was never provided by the testers, only by Buddy. The program monitored the child’s motion in the virtual space at all times, and Buddy responded to all player actions by speaking and gesturing to the player. The game was designed to be practiced in levels, starting with the simplest, where the player became comfortable in exploring the house, and moving to advanced levels, where random multiple fires and smoke tested varying fire dangers. The skill could be practiced with Buddy initially, and later could be practiced alone as the child became more confident.

Figure 2.

Buddy teaching home fire safety to child with autism spectrum disorder.

Trial setting

The low cost and portability of the PC hardware allowed us to conduct all trials in each child’s school or home. Although initial pilot tests involved a headset, the final trials used 12-, 15-, or 17-in. flat-screen monitors. Monitor size did not affect results, but details such as overhead lighting glare, computer location in the room, and the presence of other children affected results by creating distractions. The differences in testing designs between the home-based and school-based measurements were dictated by limited sequential time availability of children within the normal school-day setting and travel logistics and availability of children in the home. Despite this testing difference, learning success between the two groups appeared similar, and failures were a factor of user interface issues that different exposure methods would not have resolved.

Instruction

The training format in this study was modified to avoid the verbal communication that was problematic in the headset trials. The avatar, Buddy, did all training and correction in a first-/second-person (I/you) interactive mode. Buddy showed a correct action and then provided reinforcements like jumping and saying “Good job” if the child repeated the action. If the child attempted a dangerous motion, such as walking into a fire, the screen went black, the danger was explained and the child was told to “Try again,” and then placed back at the beginning of the game. The young children engaged with Buddy and referred to him afterwards as they would a real friend. Although a range of animal and human-like animated characters was provided in later trials to allow children to choose the avatar, choice of the teaching character did not appear to change the overall program effectiveness in our trials. Buddy remained the most popular character.

Another significant change in the instructional format was to remove user control during certain training dialogues. The child was turned to face Buddy when Buddy was speaking or toward the scene detail, such as a fire, when the detail was being explained. If the discussion were short enough (i.e., under a minute), the child appeared to accept the loss of control. Without taking away controls, we found the children preferred to keep moving in the virtual space and did not attend to instructions.

Before trials began, a pilot phase tried variations of forced attention to instruction and free play to determine how much loss of control would be tolerated by children. We typically had Buddy always moving and talking to the child and the child was allowed free action except when initially encountering a new situation. Instructions were broken into multiple actions, each under 10 s, with the child demonstrating each observed action before regaining free motion control.

A variety of punishments for dangerous actions were tried, but any response that was even slightly engaging, such as having the fire flame up, became a rewarding action. If the dangerous action were ignored, such as blocking children from walking into a fire but allowing them to move elsewhere, the children did not avoid the action in the future. A black screen with user control inhibited for a few seconds was the most effective deterrent. When we began placing the child back to a previous play position and forcing a repeat of actions already completed, the negative actions decreased quickly. If the penalty was too harsh, such as having a black screen for too long (>5 s) or moving back more than one step in the previously completed play, children indicated irritation and sometimes stopped playing.

User controls

User control was a difficult issue, partially because of the age and dexterity limitations of the children with special needs. Building virtual environments and navigation controls that permitted children with ASD and FASD in a later study to move without becoming frustrated was a major challenge. Removal of the headset eliminated the problems with the tethered walking space, but the young children who were pretested for control options in this trial were unable to handle the complexity of 6 degrees of freedom in virtual three-dimensional space with a three-dimensional joystick. Because the lessons involved only normal walking motion in a two-dimensional space, control was restricted to moving forward and backward and turning right and left, with motion adjusted to normal walking speeds. When first testing this sequence, children repeatedly ran into corners and got stuck on sides of objects. In our studies, we found that once frustrated, some children would not try again. Three children failed to learn in the second trials primarily because of navigation issues. To address this recurring problem, we created a bumper-car game-type world where the program would allow participants to slide off corners of objects if they ran into them.

In the fire safety environment, enough children became frustrated while moving within the closed walls of the virtual house to require us to provide additional navigation training. We added a Treasure Hunt game with an open-yard navigation space and five objects to catch. Although the yard practice significantly helped navigation, all virtual worlds used in safety lessons were designed to allow easy movement. As an example, the dining room table and chairs were moved to the side of the room, rather than being placed in the center of the room when children kept running into a chair close to the entry door. Joystick, game pad, keyboard, and mouse controllers were available for these trials, but the two preferred controls were the four direction keys on the keyboard and the mouse. In the next study, we eliminated the mouse because children had difficulty understanding the concept of lifting the mouse to recenter when it ran off the edge of the pad. Bright red, blue, yellow, and green tapes were placed on keyboard direction keys for visual reinforcement.

The use of the arrow keys limited user controls to only left/right or forward/backward direction buttons and prohibited players from looking up or down, or turning their heads independently of their bodies. Compensations in the play were designed to make this restriction acceptable, and no child expressed problems with this limited motion control. As an example, in the Treasure Hunt game, the objects were placed at eye level, rather than on the ground where they could disappear from vision when approached. The eye direction control stayed at straight ahead for these ASD trials, but the eye direction design was modified in the Stay-in-Yard (“safe boundaries”)games, which are currently in development. The goal is to allow children to move automatically from looking forward to looking down, depending on where the player would be expected to look (see details below).

Initially, children with ASD had problems visually separating the background from the treasure objects, so the objects were made large, colorful, and spinning. When the child came close to a treasure, it would explode in shiny dust, disappear, and a voice would say, “Good job,” along with the score bar recording a point. The reward score bar was added to the Treasure Hunt and learning lessons to measure progress toward completion in addition to delivering reinforcement. This was particularly important for children with ASD who wanted to know when an activity was considered completed.

Customization

The worlds were not customized in these trials for each child as they were in the initial headset study, but the learning sequence was expanded to allow learning in four selectable steps—explore house; leave house and go to a meeting place; respond to fire (in selectable places within the house) and go to a meeting place; and respond to smoke and go to a meeting place. All steps could be completed with Buddy leading, by the child alone, or in repeated patterns in which Buddy demonstrated first and then the child did the steps alone. On the basis of these and later trials, the sequence that provided the best retention for the children we tested was to have Buddy demonstrate twice while the child followed, and then have the child repeat twice alone. If the child missed any steps, Buddy would repeat the demonstration.

Results

Eleven of 14 children in this ASD study demonstrated 100% accuracy in the virtual world in performing the USFA fire safety steps, both with and without flames and smoke. Of the three children who were not successful in learning the new skills, one was unable to complete the pretest and use direction keys, one was unable to move about the virtual environment, and one, who had no verbal abilities, could not demonstrate any transfer of knowledge or skills to the real-world prompt at the immediate conclusion of the computer probe or at the 1-week follow-up. All children in the home-based trial were successful in generalizing to the real world after training. All school-based students who reached mastery of the computer skills were able to demonstrate knowledge of skills immediately following the final computer-based probe in response to questions about what to do in a home fire. With no further reinforcement, more than half of the school-based children retained the skills in follow-up tests a week later where they were asked in the real world to repeat the steps learned the week previously in the virtual world.

PC-based fire and street safety VR for children with FASD

Using the results of the first two ASD studies and incorporating the similar learning styles of children with ASD and FASD, the third study extended the lessons to two new VR programs for fire and street safety for children with FASD.

Design

Trials between 2001 and 2005 included a pilot with only the fire safety program (Padgett, Coles, & Strickland, 2005) and a larger study measuring both fire and street safety programs with children with FASD. The larger study is discussed here because results from the pilot were included in all design and testing considerations for the later programs.

Participants were 32 children, aged 4 to 9 years, who had a diagnosis of FAS or partial FAS, based on standardized criteria (Coles et al., in press). We compared two groups of FASD children, each group receiving exposure to either a fire safety or a street safety game. Two methods were used to assess the effects of the intervention: pre-and posttest, within-subject evaluation and a between-groups comparison, with each child acting as both a control and an experimental subject. All children received a pretest on both skill sets. They were then randomized to one condition and exposed to one of the computer games. Following mastery of this game, all children were tested again about both skills in a posttest procedure. This allowed us both to measure learning of the individual games and have a control group for the game to which they were not exposed. To examine retention of the information learned in this manner, children’s knowledge of both conditions was reassessed at a 1-week follow-up. Finally, to evaluate the extent to which knowledge could be generalized from the computer experience, all children were asked to demonstrate their learning behaviorally immediately after the computer session and the posttest and at a 1-week follow-up.

Trial setting

All children were tested at an FAS clinic that they had been attending previously for diagnosis and treatment. Although the setting was familiar to them, as for children in the ASD trials, the testing space was different. It involved a closed room with only the computer and a tester they had never met before. This arrangement allowed tighter control of the test setting than the previous ASD trials, but results indicated that the less-stimulating environment did not significantly change the learning results for the children measured. The cartoon avatar’s continual interaction with the child appeared to be key to holding attention, and physical isolation was unnecessary in our trials for effective learning.

Instruction

Because of distractibility issues inherent in FASD (and not common for ASD), the programs required redesign to become more dynamic, verbal, and “flashy.” Background sounds, such as those of cars and birds, and background songs common to commercial games were added, along with more detailed verbal instructions and feedback. Reinforcement cues, such as yellow guiding arrows on the floor, were added to compensate for the expected memory differences (Figure 3). Tests were redesigned to make each new learning change a smaller transition. Results from the first trial indicated that the children with FASD did not need many of these modifications, although the stronger verbal reinforcements were helpful.

Figure 3.

Example fire safety modifications for children with fetal alcohol spectrum disorder.

A noteworthy change in instruction occurred during the second trial for children with FASD, which involved both street and fire safety programs. Although the fire safety program had basically the same learning statistics as previous ASD studies, children in the FASD street crossing program initially failed to learn the target skills. Despite containing the same number of actions as fire safety, the street crossing skill appeared more confusing. Some children could not remember the direction to look first (left or right). If they looked the wrong way initially, in follow-up practice, they repeated the identical wrong pattern. That is, they would look right, left, right, left each time, rather than just left, right, left. (The program used a right-side road traffic driving pattern.) After one child turned around 360° instead of stopping when looking only the distance necessary to check for a car, in future crossings, he spun around first before looking correctly. The reward for the correct action became confused with a mismatch of correct and incorrect steps.

Thus, it became obvious that providing a virtual practice space with dynamic interactive teaching and a reward/punishment feedback was not sufficient in these trials for children with FASD. The children appeared to be learning the wrong safety actions as they reinforced mistakes by repeating them. Behavior problems were also observed, perhaps linked to frustrations accompanying these learning problems. In one example, a child emotionally confronted Buddy during play by saying, “I know what to do but you can’t make me.” Based on the obvious failures at this point, the trial was halted and the design modified to address these issues using different user controls.

User controls

In earlier designs, we had implemented limited computer adjustments to the player’s free motion, such as freezing the child to look at Buddy when Buddy was speaking, or positioning the player correctly when near a target stop point. Instructions were again modified, keeping the previous free motion limitations, but adding code to force the correct sequences for the more complex steps. For example, in the left, right, left looking sequence as a child crossed the street, if the child hit the right key before left key, it was ignored. When the left key was touched, it immediately turned the player left. Once the turn distance was enough to see the oncoming car, the left motion stopped and the program gave the “Good job” reward, even if the player continued to press the left button. The right key was the only key recognized next and the distance was frozen at the maximum right turn distance needed to see cars. No acknowledgement of the wrong action, which could make it stand out as a remembered pattern, was given. Initially, we had concerns that this would not create the same learning success, but both the computer and generalization tests indicated that the value of the program may not be just as a practice environment, but as a chance to pattern the correct behavior by disallowing wrong actions. No children we tested seemed aware of this loss of control or commented on it.

Thus, we extended this to other actions such as turning around in the crosswalk in the middle of the street and attempting to walk back to the starting side. Because the child is allowed to vary from a straight line as long as he or she stays in the crosswalk, the removal of turning control while in the middle of a crosswalk did confuse children. Because this is a critical safety lesson, to not walk back to the starting side in a crosswalk, we handled this control loss in a different way. When a child reached a 90° turn in the crosswalk, the directional key would be disabled and Buddy would say, “Don’t turn around in the street.” The other direction and forward keys would still be enabled, so the only direction left for the child to move was forward. If a child stepped outside the crosswalk, it was treated as stepping into the street as before and the screen went black and Buddy would say, “Stay in the crosswalk.” The lessons in the final code contained combinations of free player motions (the most common), limited motions (looking for cars at the curb, trying to turn around in the crosswalk), and frozen motion (when Buddy was speaking, when stepping into the street).

Furthermore, one additional navigation modification was made. Despite use of the same Treasure Hunt navigation practice as in earlier trials, children with FASD had problems moving in the virtual space. Training worlds were modified again for further buffering, such as putting invisible rounding lines around corners, trees, or objects so that the player would continue forward motion even if running into an object or corner. Anything that could obstruct the path or trap the user’s forward motion was removed. The children we tested with FASD were less proficient or patient in using the controls, tended to run into more parts of the world, and displayed less tolerance for delays. The player speed was adjusted to allow fast walking but no running, doors were widened and the buffering angles increased to slide off objects faster, if encountered. After initial adjustments for playability, the trials were restarted and results discussed below are based on this redesign.

Results

All children with FASD were able to learn all steps correctly in the virtual practice world (Coles et al., in press). At the first real-world generalization test for the trained skill, 72% demonstrated all four steps correctly. At follow-up a week later, 69% of both groups performed all four steps correctly again in the real world, indicating good generalization and retention of information practiced in the virtual space with no follow-up training. As a control, it was important that skill knowledge for the safety actions not trained on did not improve for either group.

Web-delivered fire and street safety VR games

When Web-delivered VR platforms became available in 2001, the fire and street safety VR programs were made available for free play from the www.do2learn.com Web site (Figure 4). Although no formal trials were carried out with users, several thousand individuals used the fire and street safety lessons between fall 2001 and spring 2005.

Figure 4.

Web-delivered street safety program.

Design

These programs provided learning modules based on the previous fire and street safety techniques. Lessons included choices of teaching avatars and adjustable levels of play difficulty. While the fire safety replicated the final design from the previous study, the street crossing was expanded to menu-selectable activities of “stay in yard,” “cross at crosswalk,” “cross with crossing sign,” and “cross at stoplight with crossing sign.” Within each activity, two worlds were selectable—a quiet suburban setting with two-lane narrow streets and a busy urban setting with four-lane streets and more world activity. Lesson modes for all games included instruction by avatar with correction, play alone, or a repeatable combination of instruction and alone.

Technical issues

Feedback from users did not indicate play issues, but there were significant technical issues related to the platform and Web playability (Rizzo, Strickland, & Bouchard, 2004). These issues included slow download for the game platform and graphics. We tried to minimize this by optimizing all scenes and characters and having selectable resolution settings, but Web speed was a problem for most users and latency often made play frustrating. Maintaining software that utilized so many outside, ever-changing programs (browsers, the Web game platform, Java language) proved to be difficult, time-consuming, and expensive. When spyware was introduced into the game engine provided by a third party, we removed these games from our Web site.

Recognizing safe boundaries VR games

The latest evolution occurred with the increased quality and affordability of good game engines that can create not just controlled lessons in virtual space but also engaging and high-quality game animation, visuals, and play (Figure 5). Using this improved technology and the lessons learned from earlier VR programs, we have built an adventure where the child protects a toy circus from being stolen by pixies while recognizing the boundaries of a yard. As the circus toys and pixies move in and out of the boundaries, children must recognize transition spaces while engaging in recovering or chasing characters. Because most pedestrian accidents involving children occur when a child darts into the path of a vehicle (NHTSA, 2004), this is an important safety issue as even children who know how to cross a street can loose track of safe boundaries when doing actions such as chasing a ball.

Figure 5.

Recognizing safe boundaries.

Design

An issue not assessed in the earlier trials that testers noted was that children quickly tired of playing the limited lessons. However, if a child followed Buddy only once and then did the lesson once alone, the steps were not patterned sufficiently to guarantee generalization. The learning sequence used in our trials for success involved four plays, twice with Buddy leading and twice practicing alone. Children at times tired of doing the same steps before the game was repeated four times, particularly after playing the Treasure Hunt twice.

To be effective for general use, voluntary reinforcement play is the goal of the present program. Lessons have been moved from an education VR platform to a commercial and fun game play using the Epic Unreal Tournament game engine. Despite the score bar used for point keeping in all previous lessons, it was impossible to lose because you had no opponent. In the new design, there are villains and challenges built into the game.

Instructions

Pilot trials indicated a broader range of choices require more on-screen text and reminders if the desired action is not performed within an expected time. Visual displays, such as arrows, which proved to be unnecessary in the first FASD research where motion inside a house was already limited, have been added to show the boundary of a large yard when the player does not identify it within a predetermined period (Figure 5). Buddy acts as an ever-present helper, who demonstrates the correct action when approached by the player.

Controls and customizations

Although formal trials have not begun, changes have been made in the previous designs on the basis of pilot tests of this game using children with a range of developmental delays. User controls were modified to project player intent and compensate for difficulties players experienced when attempting to remember several rules at once while competing in fast action between good (player) and bad (pixie) forces. One example modification is that eye gaze angle now incorporates player intent. Player eye level is modified to look up when walking forward and down when performing a function that requires lowered vision, like marking a yard edge. Lessons learned from previous trials, including limited new skill step sequences, having Buddy demonstrate first, automatic correction, player motion control variations, object buffering, and continual rewards and nonengaging punishments, have been incorporated into this game. Changes include a different, self-correcting action play and standard commercial game features such as play resume and automatic difficulty adjustments.

DISCUSSION

A series of VR programs that taught real-world actions to children with both ASD and FASD have been developed over the past 15 years. The methods evolved on the basis of lesson complexity and disability. Pilot evaluations were conducted in all studies to work out recurring problems including user controls and play motivators. The design success of many of our parameters may be related to the age of the subjects. An engaging animated teaching avatar and the particular game play used for these younger children was not effective in later studies with older teens with ASD, who were motivated more by peer interaction in a virtual shared space (Strickland, 2004).

The headset was unnecessary in our trials, but children with more severe ASD and FASD disability levels might benefit from the isolation, controlled focus, and user feedback that headsets provide. We conducted an object learning study with headsets earlier in our research that indicated although isolation and novelty value were useful in promoting learning for children diagnosed with ASD (Strickland, Osborne, & Evans, 2000), it did not give significantly different results from similar lessons learned on a flat-screen monitors in our trials (Evans, Osborne, & Strickland, 2001) to justify the exponentially greater cost.

User control was a continual challenge. The fact that children were required to manipulate a standard input device required us to select participants who demonstrated this prerequisite ability. In a limited early study (Strickland & Chartier, 1996), we attempted to measure brainwave activity within a headset while in a virtual setting to see whether that technique might be expanded to provide a communication channel for children who could not use more standard input devices. We have not pursued this concept, and all children who completed our safety studies, though sometimes diagnosed as clinically low functioning, were able to perform all steps in our trials without assistance. Programs using eye tracking or headset brainwave monitoring as virtual world control could extend the learning value to groups of children unable to use our systems.

Thus, our program designs required that target skills be taught in short, predictable periods to accommodate testing restrictions. Training in one session of approximately 30 min was statistically successful for children in our later studies, with the user control and play modifications discussed earlier. Because children at this young age as well as children with attention disorders such as FASD often exhibit limited attention spans, they are not likely to play long enough in a simulated environment to learn new skills through trial and error, as we tried with our initial headset study. The controlled learning with a teaching avatar in restricted time periods resulted in transference with fewer exposures and should allow easier adoption by parents and teachers in broader applications.

In general, all features from testing environment to teaching methods evolved significantly from our first trials to the present. We initially overestimated the difficulty in using the technology and underestimated the player interaction component. Despite these limitations, we found that in general this technology involving first-person practice with demonstration guided by animated characters, combined with strategically placed controls on what is allowed within the worlds, was highly effective for patterning the desired actions and teaching children new skills. More surprising, the lessons learned in VR in the short exposures generalized at more than 50% to similar real-world situations. For children who had demonstrated severe learning problems previously, the quick adaptation of new skills and extension to real situations was encouraging. The technology evolved over the years encompassing our studies and is now a widely available tool that holds promise for many applications where interactive practice provides learning value for children with ASD and FASD.