Usability and Feasibility of an Internet-Based Virtual Pedestrian Environment to Teach Children to Cross Streets Safely

Abstract

Child pedestrian injury is a preventable global health challenge. Successful training efforts focused on child behavior, including individualized streetside training and training in large virtual pedestrian environments, are laborious and expensive. This study considers the usability and feasibility of a virtual pedestrian environment “game” application to teach children safe street-crossing behavior via the internet, a medium that could be broadly disseminated at low cost. Ten 7- and 8-year-old children participated. They engaged in an internet-based virtual pedestrian environment and completed a brief assessment survey. Researchers rated children's behavior while engaged in the game. Both self-report and researcher observations indicated the internet-based system was readily used by the children without adult support. The youth understood how to engage in the system and used it independently and attentively. The program also was feasible. It provided multiple measures of pedestrian safety that could be used for research or training purposes. Finally, the program was rated by children as engaging and educational. Researcher ratings suggested children used the program with minimal fidgeting or boredom. The pilot test suggests an internet-based virtual pedestrian environment offers a usable, feasible, engaging, and educational environment for child pedestrian safety training. If future research finds children learn the cognitive and perceptual skills needed to cross streets safely within it, internet-based training may provide a low-cost medium to broadly disseminate child pedestrian safety training. The concept may be generalized to other domains of health-related functioning such as teen driving safety, adolescent sexual risk-taking, and adolescent substance use.

Between 2005 and 2010, over 3000 American children died as a result of pedestrian injuries (NCIPC 2012). Over 300,000 others were treated in hospital emergency departments but survived (NCIPC 2012). Despite its role as a leading cause of pediatric death (NCIPC 2012; WHO 2008), pedestrian injury represents a preventable pediatric health problem that, unlike many global child health problems, is not decreasing in scope (Ameratunga et al 2006; WHO 2008).

A range of factors increase pedestrian injury risk. Environmental factors such as road and sidewalk engineering contribute, and can be ameliorated through traffic engineering interventions. Driver factors such as speeding and distraction also play a role, and can be ameliorated through institution and enforcement of traffic laws and behavior change interventions targeting drivers.

A third major factor that contributes to child pedestrian injury risk, and the current focus, is child behavior. Safe street-crossing requires the ability to control impulses and attend to relevant stimuli; perceptual ability to detect and interpret multiple moving vehicles; and cognitive ability to process perceived information about the road environment and, in the span of milliseconds, decide whether it is safe to initiate motoric movement across the street. Developmental psychology research suggests several functions required for safe pedestrian ability develop through early and middle childhood (Barton and Morrongiello 2011; Demetre 1997; Plumert et al 2007; Thomson 2007; Whitebread and Nielson 2000).

The logical question behaviorists ask is whether a behavioral intervention might teach children relevant cognitive, perceptual, and attentional/impulse control skills at an earlier age than they would otherwise develop those skills. Early attempts used classroom training strategies and achieved mixed success (Duperrex et al 2002; Schwebel et al 2012). Others used one-on-one training methods whereby a skilled pedestrian teaches a child to cross directly at street environments (Ampofo-Boateng and Thomson 1993; Barton et al 2007; Demetre et al 1993; Thomson et al 1992; van Schagen 1988). Such programs tend to achieve success, with children as young as age 5 learning and retaining some pedestrian skills after on-site training (Thomson et al 1992; van Schagen 1988). The major limitation to such training programs is the time and labor involved, often beyond the budget of schools or community centers to institute broadly.

More recently, virtual reality (VR) has been explored as a strategy to teach children to cross streets safely. VR offers the advantage of repeated practice tailored to the child's ability level without intensive adult involvement. With a proper virtual environment application validated to reflect real-world activity and behavior (Schwebel et al 2008), one can offer children repeated practice to improve pedestrian skills. Early small-scale trials (Bart et al 2008; McComas et al 2002; Thomson et al 2005) and a recent randomized controlled trial (Schwebel et al in press; Schwebel and McClure 2010) suggest children can learn and retain pedestrian safety skills after training in virtual pedestrian environments (see also Babu et al 2009, 2011 for work along a similar pathway with bicycling simulation, focused also on peer influences in training and safety).

Although VR technology has evolved rapidly (and costs decreased) the past decade, it remains beyond the budget of many schools and community centers, thus prohibiting wide dissemination efforts. An effective intervention is not useful if it cannot be widely disseminated for broad use at a reasonable cost. Thus, even if VR training is an effective way to teach children to cross streets safely – and preliminary evidence offers data to suggest it is – it is not an effective intervention until society can broadly disseminate it at a reasonable cost.

With the ultimate goal of broad, cost-efficient dissemination of an effective behavioral intervention for pedestrian safety training that might be generalized to other domains of child and adolescent health, the present study was designed as a pilot study to offer feasibility and usability data for an alternative to large, bulky, and rather expensive virtual pedestrian environment systems. Specifically, we sought to test the feasibility and usability of an internet-based virtual pedestrian environment to train 7- and 8-year-old children in pedestrian safety. We addressed three questions:

1. is an internet-based virtual pedestrian environment usable for 7- and 8-year-old children – that is, for independent engagement without adult supervision;

2. is an internet-based virtual pedestrian environment feasible for pedestrian safety training – that is, does it produce data that could be used for research and training purposes; and

3. is an internet-based virtual pedestrian environment engaging and entertaining for 7- and 8-year-old children – that is, do they find it enjoyable and do they stay engaged in it for a training session of reasonable length?

Methods

Participants

Ten 7- and 8-year-old children (M = 7.76 years, SD = 0.52; 40% female; 50% African-American) were recruited from community sources. Parents provided informed consent and children informed assent. The protocol was approved by the Institutional Review Board at University of Alabama at Birmingham.

Internet-based Virtual Reality System

The internet-based system was adapted from a bulkier, full-room virtual pedestrian environment described in previous work (e.g., Schwebel et al 2008). Presented as a “game” to children, the program was developed using Unity 3D software and packaged as a Unity Web Player Application. It runs over a standard internet connection and functions on any internet-connected desktop or laptop computer and could be adapted for mobile devices.

Users view the center of a street environment simulating an actual crosswalk near a local elementary school (See Figures 1-3 for screenshots). The computer offers brief verbal directions, instructing users to move the mouse left to see the traffic/environment to the left and right to see the traffic/environment to the right. Mouse movements are continuous, displaying 75° of vision horizontally and 60° vertically, with horizontal vision moving progressively in conjunction with mouse movements. Directions also inform users to push the space bar when they believe it safe to cross. Doing so triggers an avatar to cross the street, altering the environment from first-person immersive to third-person observer. This switch, which happens quite seamlessly, replicates the bulkier laboratory VR environment and allows users to witness and learn from the success of the crossing.

Figure 1.

Screenshot of internet-based virtual pedestrian environment. Straight-on view.

Figure 3.

Screenshot of internet-based virtual pedestrian environment. View to right side.

When users cross the street safely in the internet-based system, a cartoon character appears and offers positive reinforcement with one of two positive messages. When the crossing is risky (a “close call”, defined as the avatar being within one second of collision with a vehicle), the cartoon character appears and offers a cautionary message. When the avatar is struck by a vehicle, the screen freezes just before “impact” and the cartoon character recommends to the user that he/she “try to be more careful next time”.

Avatar walking speed in the internet-based virtual pedestrian environment is based on the walking speed of 299 participants ages 7-8 in previous research (Avis et al 2012; Schwebel and McClure 2010; Schwebel et al 2008, in press). Prior to beginning a session, users choose from three walking speed options (slow, 5.07 feet/second; medium, 4.44 feet/second; fast, 4.01 feet/second). The medium speed represents the average walking speed of the 299 previous participants; the slow and fast represent one quartile above and below the mean. Users also choose a preferred traffic density from 3 levels (light, 8 vehicles/minute; middle, 12 vehicles/minute; heavy, 16 vehicles/minute). These were determined based on average traffic density of 11.29 vehicles/minute (SD = 2.46) at the actual crosswalk site during before-school hours. Traffic travels at 30 MPH (posted speed limit at simulated site) and the types of vehicles (e.g., sedans, vans, trucks) on the road mimic counts observed during pre-school hours.

Protocol

After informed consent processes, parents completed a brief demographics questionnaire. Children were taken to the internet-based VR system in a small room. Experimenters instructed children to begin engaging in the “game” and stayed to assist if needed. Experimenters also guided children back to the website if they attempted to navigate to alternative websites or programs on the computer. After 15 “warm-up” trials at a comparatively easy level of pedestrian crossing (fast avatar walking speed and light traffic density), children were given a 5-minute break with parents and then returned to the virtual pedestrian environment and completed 15 experimental trials with the environment using the medium avatar walking speed and medium traffic density. Two participants started but did not complete the full second data collection session, one due to technical error and one who requested to discontinue that portion of the experiment for unspecified reasons. Data were dropped for the second session of those two participants; all other data were available. Following engagement in the internet-based VR, the experimenter completed the child questionnaire orally with all participants. Families were compensated for their time.

Pedestrian Safety Measures

The following six variables were recorded electronically by the VR application and considered for validity and crossing safety:

• Hits: when avatar was struck by a vehicle during crossing

• Close calls: when avatar was within 1 second of being struck by a vehicle during crossing

• Missed opportunities: when participant could have safely crossed but chose not to; safe gaps were those 1.5 times the time required to cross

• Start delay: time between safe gap appearing and participant initiating movement into the street

• Time to contact (TTC): shortest temporal distance between avatar and oncoming vehicle during crossing

• Attention to traffic: looks left plus looks right while waiting to cross, divided by wait time

Self-Report Measures

Parents completed a brief demographic questionnaire. Following engagement in the internet-based virtual environment, children completed a brief questionnaire orally with a researcher. Four measures were derived from that questionnaire:

Ease of engaging in game, created by averaging responses to two items. One evaluated confusion using the game and the other asked whether the child needed help from an adult to figure out how to use the game. Both were answered dichotomously, with 2 indicating difficulty engaging in the game and 1 indicating no difficulty.

Ease of crossing street, focused specifically on ease of using the computer hardware to cross the virtual street. Three items, each answered on a 4-point scale, were averaged. Items addressed difficulty looking for cars from the left, difficulty looking for cars from the right, and difficulty crossing the street. Higher scores indicated greater difficulty.

Enjoyment of game. Children responded to three items regarding enjoyment of the game and desire to play it further. One asked if the game was fun and children answered on a three-point scale (yes, a lot vs. yes, a little bit vs. no, not really). A second asked if the child would like to play this game again in the future. A three-point response scale was used (yes vs. maybe vs. no). A third asked if the child thought other kids the same age would like to play, again on a three-point scale (yes vs. maybe vs. no). The three scales were averaged, with higher scores indicating greater enjoyment from the game.

Perceived educational value. Children responded to two items regarding perceived education from the game. The first asked children whether they believed they learned anything about crossing streets safely from playing the game; they responded on a three-point scale (yes vs. maybe vs. no). The second asked children if they believed other children their age might learn from the game; again, a three-point response scale was provided (yes vs. maybe vs. no). The two responses were averaged, with higher scores indicating higher perceived educational value from the game.

Researcher-Report Measures

Experimenters tallied two measures during the experimental sessions: (a) instances when children requested or showed obvious need for help from an adult to use the internet-based virtual pedestrian environment and (b) instances when the child tried to veer off the website to engage in other games or computer-based activities. Each was analyzed as a count variable.

Observational Measures

Experimental sessions were videotaped through a one-way mirror and later coded for two measures of child engagement, restlessness and attention. Restlessness incorporated fidgeting, wiggling, and movements while playing the game and was evaluated on a 6-point scale ranging from 1 (sits still, doesn't move much in chair beyond what is required for game play) to 6 (gets up, usually more than once, and moves around room away from chair area). Attention incorporated distraction, focus, and attention while playing the game and was evaluated on a 6-point scale ranging from 1 (extremely focused on the stimuli for the full session) to 6 (unfocused; rarely attending to the relevant stimulus; mind and eyes wandering; attention is lacking for most or all of the session). All sessions were independently coded by two trained research assistants. Ratings were almost identical (kappa = .93) and the single difference resolved by averaging.

Results

We evaluated usability through two constructs from the instrument completed by children after engaging in the internet-based virtual pedestrian environment (See Table 1). Children felt the program was easy to engage in (2-item aggregated construct on 2-point scale, mean = 1.25, SD = 0.35). At the item-level, 80% of children reported the game was not confusing and 80% reported they did not need adult help to use the game.

Table 1.

Descriptive Data (N = 10)

Variable	Mean (SD)
Demographics
Sex (% male)	60%
Race/Ethnicity (% White)	50%
Race/Ethnicity (% African American)	50%
Age (years)	7.76 (0.52)
Usability
Ease of Engagement (2-point scale)	1.25 (0.35)
Ease of Crossing (4-point scale)	1.37 (0.82)
Feasibility - Pedestrian Measures
Hits (count of 15 crossings)	1.38 (1.77)
Close calls (count over 15 crossings)	2.25 (2.49)
Time to contact (seconds)	1.98 (0.30)
Missed opportunities (count over 15 crossings)	3.50 (4.11)
Start delay (seconds)	0.71 (0.33)
Looks by wait time in seconds	1.06 (0.53)
Engagement & Education
Enjoyment of game (3-point scale)	2.63 (0.37)
Perceived educational value (3-point scale)	2.45 (0.64)
Fidgeting (6-point scale)	2.44 (0.73)
Restlessness (6-point scale)	2.00 (0.00)
Requests for help from adult (count)	0.00 (0.00)
Attempts to navigate elsewhere on computer (count)	0.00 (0.00)

Children also felt it was easy to cross the street within the program (3-item aggregated construct on 4-point scale, mean = 1.37, SD = 0.82). At the item-level, 80% of children found it “not at all” hard to figure out how to look for cars coming from the left and 90% “not at all” hard to figure out how to look for cars from the right. Similarly, 80% of children felt it “not at all” hard to figure out how to cross the street.

Feasibility was assessed by evaluating whether we could successfully cull data concerning children's pedestrian behavior from the internet-based VR system. Such data could be used both for research and training purposes. Of particular interest, such data could be used to tailor training, such that children elevate to more advanced virtual pedestrian situations as their crossing skills improve.

We obtained all six pedestrian safety outcomes successfully from the system (See Table 1). Children were hit by virtual vehicles an average of 1.38 times (SD = 1.77) and experienced an average of 2.25 close calls (SD = 2.49) across the 15 trials. They generally chose safe gaps to cross within (average time to contact during crossing = 1.98 seconds; SD = 0.30) and, perhaps reflecting their age and inexperience as pedestrians, missed an average of 3.50 safe opportunities to cross the street over the 15 trials (SD = 4.11). The children had an average delay of 0.71 seconds (SD = 0.33) prior to entering the street to cross within a safe gap and looked left or right an average of 1.06 times per second (SD = 0.53). Together, the pedestrian data indicate poor pedestrian skill among this sample of 7- and 8-year-olds, but that finding is not novel or unexpected among this small and young sample. More critical is evidence that we could collect valid and realistic data about pedestrian behavior via an internet-based VR system.

Children's engagement and perceived entertainment from the system was evaluated via both self-report and researcher observations. Children's self-report of enjoyment on the aggregated scale was high (M = 2.63, SD = 0.37 on 3-point scale). At the item level, 90% of the children reported the program was “fun” to play, all answered either “yes” (60%) or “maybe” (40%) to whether they would like to use the game again, and 70% indicated that they would recommend it to other children their age (the remaining 30% responded “maybe”).

Researcher tallies of children requiring help to use the environment and times when children tried to veer off the website to other computer-based activities were both null for all participants; that is, researchers noted no times when children requested help from the researcher and no instances when any participating child tried to connect to alternative websites. Researcher ratings of fidgeting during the experimental session averaged 2.44 (SD = 0.73) on the 6-point scale. A 2 reflected sitting in the chair for the entire session and “a bit” of wiggling, while a 3 reflected sitting in the chair for the entire session and some movement and wiggling. Researcher ratings of attention during the experimental session were all 2.00 (SD = 0.00), reflecting a child who was “quite focused” for the session, with attention on the game for essentially the entire session.

We also considered children's perceived educational value of the game. On the two-item aggregated measure of perceived educational value, children scored an average of 2.45 (SD = 0.64). At the item level, 70% of children answered “yes” (60%) or “maybe” (10%) to whether they believed they learned something from the game and 100% of children (60% yes, 40% maybe) believed other children their age would or might learn from the game.

Discussion

Results of this pilot study suggest an internet-based virtual pedestrian environment offers a usable, feasible environment for pedestrian safety training. Children used the environment without assistance and judged it educational and entertaining. They remained engaged and attentive in the environment.

This study was designed to be preliminary, and the sample too small to offer useful data on child pedestrian safety (which is available elsewhere; see Schwebel et al 2012 for review), but the present results offer initial evidence that an internet delivery system might be considered for broad pedestrian safety training, and beyond that for other purposes in training children and adolescents to stay safe in potentially risky situations. Existing pedestrian safety training programs shown to be efficacious are either highly expensive and laborious (e.g., individualized streetside training with skilled older pedestrians) or expensive, bulky, and infeasible for broad dissemination (e.g, laboratory-based VR environments). If an internet-based virtual pedestrian environment application proves efficacious for training – and these data offer the initial step toward that possible outcome – then we as scientists and interventionists may have a novel, low-cost, and powerful means to train young children to cross streets. With broad dissemination, an internet-based pedestrian training system could have wide-ranging impact on pediatric safety.

This work is preliminary and conclusions must be drawn cautiously. We foresee three next steps as foremost. First, a more intensive feasibility/evaluation study might be conducted with a larger sample to demonstrate that children can use an internet-based VR environment and children can learn from an internet-based VR environment. This study might include children across a wider age range than used in the present study, and a larger and more diverse sample to establish usability, feasibility, and efficacy more definitively.

Second, alternative VR environments might be developed. The VR application presented in this study focuses on mid-block crossings on bi-directional streets. Although this comprises a high-risk environment for children in middle childhood (Warsh et al 2009), it does not offer targeted training to children for crossing signaled intersections, streets with unidirectional traffic, or streets with multiple lanes in each direction. Similarly, because the avatar walks at a constant speed, the VR presented in this study does not allow children to engage in “aversive action” to escape a dangerous situation if they recognize risk mid-crossing. Alternative VR applications might include means for children to speed or slow their walking speed mid-crossing.

Third, efficacy trials should be conducted to evaluate whether children's learning in an internet VR environment is comparable to learning using other efficacious pedestrian safety training strategies such as individualized streetside training or training in a traditional VR environment. There are disadvantages to an internet VR environment. By using the internet, one sacrifices realism and immersion. What remains unclear and in need of empirical validation is whether sacrificed realism or immersion influences training. An interventionist's ultimate goal is to teach the cognitive and perceptual skills needed for a task, such as to cross a street safely. Research to date suggests such skills can be learned in a semi-immersive laboratory-based pedestrian VR (Schwebel et al 2013). Can we take the next step, to move from semi-immersion in a laboratory-based VR to a two-dimensional immersive environment presented via internet? Will it be as effective for children to practice and learn the cognitive/perceptual skills of judging traffic, deciding when a safe gap appears, and “entering” the road? Empirical data are required, but one might deduce the cognitive and perceptual skills required for safe pedestrian ability do not require full immersion. It may be adequate – perhaps even equivalent – to learn those skills in a less immersive, less realistic simulated environment.

Beyond pedestrian safety, virtual reality – and especially internet-based virtual reality that can be broadly disseminated – might offer great potential to reduce risk to children and adolescents in a wide range of settings. It seems a small leap, for example, to extend from a virtual pedestrian environment to virtual bicycling (see Babu et al 2009, 2011; Plumert et al 2004 2007) or driving (see Fisher et al 2002, 2006) environments to train child cyclists and teen drivers in safe practices. Similarly, virtual environments might be used to teach youth about the risks of ATV operation. A greater step, but still feasible, may be virtual environment applications that train adolescents in the risks of unsafe sex, drug and alcohol use, and high-risk adventure sports, or that treat individuals with wide-ranging problems such as difficulty with stress management (Serino et al 2013), sexual deviance (Renaud et al 2013), or treatment of mental illnesses like anxiety and schizophrenia (Gutiérrez-Maldonado et al 2013; Spanolli et al 2013). As today's society, and especially today's youth, use the internet with increasing frequency, educational applications and games they can access on computers, tablets, and phones offer a unique opportunity to provide health-related training to children and adolescents.

Figure 2.

Screenshot of internet-based virtual pedestrian environment. View to left side.