Virtual reality by mobile smartphone: improving child pedestrian safety

Abstract

Background

Pedestrian injuries are a leading cause of paediatric injury. Effective, practical and cost-efficient behavioural interventions to teach young children street crossing skills are needed. They must be empirically supported and theoretically based. Virtual reality (VR) offers promise to fill this need and teach child pedestrian safety skills for several reasons, including: (A) repeated unsupervised practice without risk of injury, (B) automated feedback on crossing success or failure, (C) tailoring to child skill levels: (D) appealing and fun training environment, and (E) most recently given technological advances, potential for broad dissemination using mobile smartphone technology.

Objectives and methods

Extending previous work, we will evaluate delivery of an immersive pedestrian VR using mobile smartphones and the Google Cardboard platform, technology enabling standard smartphones to function as immersive VR delivery systems. We will overcome limitations of previous research suggesting children learnt some pedestrian skills after six VR training sessions but did not master adult-level pedestrian skills by implementing a randomised non-inferiority trial with two equal-sized groups of children ages 7–8 years (total N=498). All children will complete baseline, postintervention and 6-month follow-up assessments of pedestrian safety and up to 25 30-min pedestrian safety training trials until they reach adult levels of functioning. Half the children will be randomly assigned to train in Google Cardboard and the other half in a semi-immersive kiosk VR. Analysis of Covariance (ANCOVA) models will assess primary outcomes.

Discussion

If results are as hypothesised, mobile smartphones offer substantial potential to overcome barriers of dissemination and implementation and deliver pedestrian safety training to children worldwide.

Injury is the leading cause of paediatric mortality in the USA and much of the world, killing more American children aged 1–18 years than all other causes combined.¹ Morbidity rates far exceed mortality and impact our public health system tremendously. This proposal considers pedestrian injury, which is among the leading causes of paediatric unintentional injury.¹ In a single year, over 4900 American pedestrians are killed and 207 000 others injured; about a fourth of injured pedestrians are children.¹ The estimated financial burden of pedestrian injuries in the USA exceeds $12 billion a year.

Several studies document that young children regularly negotiate street environments alone when going to and from school.^2–4 In middle childhood (age 5–9 years), about 60% of pedestrian injuries and mortalities occur when the child is crossing a road at or between intersections^5–7—with mid-block crossings particularly dangerous near schools.⁸

INTRODUCTION

Previous pedestrian safety interventions

A recent systematic review of randomised controlled trials (RCTs) to teach children pedestrian safety uncovered 19 publications reporting 25 studies.⁹ Together, they indicate children’s capacity to learn pedestrian skills. Among the studies designed to teach children mid-block crossing skills (eg, judging safety of traffic gaps amidst moving vehicles), the most effective training strategies were those that involved repeated practice either in vivo at streetside locations or in simulation using virtual reality (VR) environments.⁹

VR, the strategy proposed for use in the present study, has been advocated as a strategy to teach children to cross streets for several reasons. First, VR permits training and practice that cannot be ethically accomplished in the actual environment due to danger; this is why it is used for training in military missions,¹⁰ piloting high-speed aircraft¹¹ and high-risk surgery.¹² Second, VR offers capacity for systematic delivery and control of stimuli. It permits customised training to individual skill levels and progressive escalation in challenge as skills increase. Finally, VR is engaging. This is particularly important to train children, who in today’s world are accustomed to interactive and entertaining technology-based learning. Most children enjoy VR engagement, and that enjoyment may translate to better attention and ultimately better education.

VR technology has been used for over a decade now to understand children’s pedestrian safety. Early research was conducted in non-interacting and non-immersive VRs,^13,14 but more recently, immersive or semi-immersive interactive VR systems have been developed.^15,16 There is ample evidence to indicate feasibility and usability of VR to study child pedestrian safety, and our own research demonstrates VR’s validity compared with real world pedestrian behaviour among both children and adults.¹⁶

In previous work, we used semi-immersive virtual environments to train children in pedestrian safety.^17,18 In one study, 240 children completed baseline assessments and then were randomly assigned to four groups: VR training, one-on-one streetside training with an adult, training via commonly used video/internet programmes and a no-contact control group.¹⁷ All three training groups received six 30 min training sessions. Immediate post-training and 6-month follow-up assessments evaluated pedestrian safety learning and retention. Children in the VR and streetside training groups performed more safely postintervention than those in the video/internet or control groups. In a second trial, a more mobile ‘kiosk’ VR was placed in schools and community centres.¹⁸ Using pragmatic trial strategies, children were exposed to six 15 min VR training sessions. Replicating findings from the first trial, children demonstrated a decrease in time to enter safe traffic gaps post training. The number of unsafe crossings did not change significantly, however.

As we considered results across the two previous trials, two sets of results piqued our interest: (A) greater effect sizes in learning occurred in the first study, which included six 30 min training sessions versus the second, which included six training sessions of just 15 min (half the training time); and (B) decreases in start delays—temporal delays in entering safe traffic gaps, which are hypothesised to represent inefficient and immature decision-making processes¹⁴—were consistently strong, offering evidence of greater efficiency in children deciding when to enter traffic gaps following training that did not always yield significantly safer crossings, especially in the second study.

These two observations highlight the fact that the process of how children learn to cross streets is largely undocumented. We know judgement of moving objects is a challenging cognitive-perceptual task, and that skill develops with age.^19–23 We also know that coordinating those cognitive-perceptual judgements with motoric aspects of initiating and regulating one’s own movement across the street is complicated, as it involves judgement and decision-making that leads to motor initiation.²⁴ An in-depth analysis of children’s pedestrian learning across training trials suggests children develop more efficient and accurate pedestrian decision-making with training, but six 30 min training sessions are insufficient for most 7–8-year-olds to achieve adult levels of pedestrian functioning.²⁵ Thus, the present trial will offer up to 25 training sessions to children and evaluate the extent of training required for children to progress to adult levels of pedestrian safety functioning.

Previous research also suffers from another challenge: if VR is an effective strategy to teach children to cross streets, how does one broadly disseminate the strategy? Both technical challenges and financial cost of large-scale VRs are decreasing rapidly, but VRs remain beyond the resource capacity of most schools, community centres and other entities who might promote pedestrian safety training broadly to children. Delivery of a VR environment to a mobile smartphone platform could overcome that barrier.

GOOGLE CARDBOARD AND SMARTPHONE VRs

Google Cardboard is a VR platform released in 2014 that offers capacity to transform a mobile Android or iOS smartphone into a VR device. Using Google Cardboard is surprisingly simple. Users download a free app and place their mobile phone into what is literally a piece of cardboard folded up and fitted with focal lenses. The system costs less than $5; it is so easy and practical that McDonald’s restaurants in Sweden are piloting delivery of VR-based games to customer’s smartphones, with children’s Happy Meal boxes folded up to create the Cardboard hardware apparatus. The phone’s gyroscopes and accelerometers facilitate display of interactive scenes that offer a feeling of immersion. The image is sharp, bright and veridical. Stereo sound is delivered through the phone’s speakers.

Our pedestrian VR programme is built upon the Unity game engine developed by Unity Technologies and can be delivered immediately and at minimal cost to almost any mobile smartphone in the world. Feasibility testing suggests the platform is usable and feasible, results in minimal motion sickness, and has convergent validity with behaviour in a semi-immersive VR (DC Schwebel, J Severson, Y He. Using smartphone technology to deliver a virtual pedestrian environment: Usability and validation. Manuscript under review 2016).

AIMS AND HYPOTHESES

Using a non-inferiority clinical trial design, this research has two goals: (A) demonstrate that training children in pedestrian safety using mobile phones and Google Cardboard achieves pedestrian safety learning equivalent to that of a semi-immersive VR environment, and (B) document the extent of training needed for 7–8-year-old children to achieve adult levels of pedestrian safety. We have three primary objectives:

1. Demonstrate children trained in pedestrian safety amidst a Google Cardboard VR achieve equivalent levels of pedestrian safety to children trained in a full semi-immersive VR pedestrian environment after 6 training trials of 30 min each, the level of training used in previous work.¹⁷

2. Demonstrate that: (Aim 2a) most (>80%) 7–8-year-old children achieve adult levels of pedestrian safety after completing 25 30 min sessions, and (Aim 2b) determine how many training sessions are sufficient for children to achieve adult levels of pedestrian safety.

3. Using non-inferiority trial methods, demonstrate that children trained in pedestrian safety amidst a Google Cardboard VR achieve equivalent levels of pedestrian safety to children trained in a semi-immersive VR pedestrian environment after 25 training trials.

METHODS

We will conduct a randomised non-inferiority trial to test whether pedestrian safety training in Google Cardboard is non-inferior to the VR system previously shown efficacious and to evaluate the extent of training required for most 7–8-year-old children to achieve adult levels of pedestrian functioning. We will accomplish these goals using a trial with two equal-sized groups of children ages 7 years and 8 years (total N=498). All participants will complete a thorough battery of field-based and laboratory-based measures of street crossing and pedestrian skills during baseline and postintervention visits, as well as during a 6-month follow-up. They also will complete up to 25 pedestrian safety training trials until they reach adult levels of functioning, with half the children randomly assigned to complete training in Google Cardboard and the other half in a semi-immersive kiosk VR.

Non-inferiority trials

Non-inferiority trials are a form of RCT used to test whether a new intervention achieves outcomes no worse (and potentially better) than an existing intervention.^26–28 Non-inferiority trials are especially valuable when the existing intervention achieves desired outcomes but the new intervention is easier or cheaper to implement, has fewer side effects, or is otherwise more desirable than the existing effective intervention. In such situations, it is unethical to place some participants randomly in placebo or no-treatment control groups,²⁹ as would be done in ‘traditional’ superiority-based RCTs. Instead, a non-inferiority trial compares the existing, known-to-be-effective intervention with a novel one that might offer equal or advantageous treatment outcomes with fewer adverse influences.

Non-inferiority trials are used most often in pharmaceutical trials, where a newly developed medication is compared with a currently preferred one. The newly developed medication might offer lower cost, fewer side effects or other advantages and the researcher seeks to demonstrate that it functions as well as (or better than) the currently preferred medication. Comparing the new medication to a placebo is unethical, as it leaves research participants untreated for a treatable condition. The leap to our proposed research is surprisingly natural. Semi-immersive VR systems have demonstrated efficacy to teach children pedestrian skills but are bulky, expensive and impractical to implement broadly. Mobile smartphones are widely available and therefore Cardboard can be delivered to schoolchildren worldwide broadly and cheaply. Enrolling participants and randomly assigning them to a no-treatment control seems ethically troubling, as we know training in VR benefits children. If our proposed hypotheses prove true, we can move to broad dissemination of an effective, low-cost, easily used intervention.

From a statistical theory perspective, non-inferiority trials are initially non-intuitive. As Popper’s classic treatise³⁰ outlined, null hypotheses cannot be proven but only disproven. Therefore, non-inferiority trials propose seemingly backward null hypotheses, that the existing intervention is superior to the new intervention. A prestated margin of non-inferiority (δ) is proposed based on previous research and the null hypothesis rejected if that margin is not met via consideration of CIs with a predetermined α level. The alternative to the null is that the new intervention is non-inferior to the existing one.

Participants

Four hundred and ninety-eight 7-year-old and 8-year-old children will be recruited from the local community in Birmingham, Alabama. They will represent local racial and socioeconomic diversity.

VR: technical specifications and user experience

Two VRs will be used, the ‘kiosk’ VR in a laboratory setting and the Google Cardboard VR delivered via smartphone. In the kiosk system, children stand on a plywood curb and step down onto a pressure plate to trigger the system for crossing. The system runs on a single Windows 7 PC with an Intel Core i5-3330 3.0 GHz Quad-Core desktop processor and GeForce GT 640 video card. The virtual environment is displayed on three vertically mounted Samsung MD55C 55″ Direct-lit LED displays and is moved vertically on the displays to match the participant’s eye level. In the Google Cardboard system, children use a standard Android or iOS mobile smartphone inserted into a durable View-Master VR viewer. The street crossing simulator software on both systems are built on top of the Unity game engine with identical virtual environments, game play logic and data collection mechanism; they only differ in platform-specific software adaptation.

In both VR platforms, prior to children entering the VR, the researcher enters a walking speed (based either on the child’s previously measured walking speed in a separate location or age-based averages), gender and skin tone (to select a sex-matched and race-matched avatar; only kiosk uses avatars). Children view a bidirectional roadway modelled after an actual street environment near a local school. Traffic density, traffic speed and vehicle types are adjusted to researcher preferences and child skill level. Ambient and Doppler-accurate traffic noise are delivered. Over a dozen types of vehicles (including cars, SUVs, pick-ups, and also ambulances, school buses, etc) appear in random order at researcher-determined frequencies. When children deem it safe, they step off the simulated curb to trigger a pressure plate (semi-immersion) or push a button (Cardboard). In the kiosk VR, the virtual world view then changes from first to third person, permitting children to view themselves crossing. This switch from first to third person happens seamlessly and most users do not even notice the switch when asked about it later. In the Cardboard VR, the virtual world remains at first-person view, as the viewpoint follows the avatar across the street.

Following the avatar’s crossing, the child is informed about the safety of crossing. Extensive data, including the precise positions of the avatar, vehicles and users’ head movements, are collected.

General protocol

Children will participate in up to 31 sessions: pretraining lab session (1.5 hours), pretraining field session (20 min), up to 25 training sessions (30 min each), post-training lab session (1.25 hours), post-training field session (20 min), 6-month follow-up lab session (1.25 hour) and 6-month follow-up field session (20 min).

During pretraining sessions, baseline pedestrian safety measures will be collected in both virtual and real (field) environments. We also will assess demographics, anthropometry, temperament, verbal functioning and pedestrian behaviour history. Following pretraining assessment (but not before to reduce researcher or participant bias arising from knowledge of group assignment), children will be randomly assigned to one of two groups, Cardboard or kiosk VR. Random assignment will occur by children choosing a piece of paper that assigns them to the group at the end of the baseline assessment. Six training sessions will be held for all children, and the training sessions will continue until the child reaches adult levels of pedestrian safety functioning. Following training, post-training pedestrian behaviour will be collected during lab and field visits with assessment protocols similar to those collected pretraining. Finally, two 6-month follow-up sessions, one lab and one field, will assess skill retention. Details of all sessions appear below.

Protocol: pretraining assessment

Two sessions, one lab-based and the other field-based, will assess pretraining baseline pedestrian abilities. We also will collect demographic and individual difference data during those visits. The longer visit will be in the laboratory, during which children will complete 40 VR street crossings, 20 in the kiosk VR and 20 in the Cardboard VR. Order of VRs and traffic volume will be randomised. All trials will be conducted using standard protocol and instructions, preceded by orientation trials.

The second pretraining session will occur in the field, at the actual site of the simulated environment. Children will complete eight crossings using the ‘shout’ technique,³¹ whereby they stand immediately adjacent to the road and shout ‘now’ when they deem it safe to cross. Children will also complete eight crossings using the ‘two-step’ technique,³¹ whereby they stand two steps off the curb, and take two steps towards the road to indicate they deem it safe. Traffic patterns will be those that naturally occur. Outcome measures from pedestrian tasks are detailed below.

Protocol: training sessions

Children will complete up to 25 training sessions. Training sessions will be scheduled twice a week and will be identical in both groups except for the VR used. Children will engage in the VR for three sets of 15 crossings, or 45 crossings at each training session (expected duration ~30 min). The first two sets of crossings will be tailored to children’s ability level, with traffic density set just beyond the level they previously succeeded crossing at. The third set of crossings will be standardised for all visits, at 30 MPH and 10 vehicles/min/lane. Along with serving as a training trial, this third set will typically be a little easier for children, providing positive feedback and motivation for training. It also will offer a standardised assessment of ability.

All children will complete six visits to the laboratory for training. Following the sixth and all subsequent visits, we will monitor each child’s number of unsafe crossings in the final standardised assessment. When children complete two consecutive visits at a level equal to or safer than the average adult performance, they will be considered ‘competent’ pedestrians functioning like an adult and training will be discontinued. We expect most children will reach this level but in rare cases that continue to 25 trials without reaching adult thresholds, training will be discontinued. Adult performance will be defined based on previous research with >300 adults.

Protocol: post-training assessment

Post-training assessment will occur 3.5 months following baseline and will mirror pretraining. Two sessions will be held, one lab and one field. Lab assessments will include 20 crossings in each VR and field assessments of eight crossings each using the shout and two-step techniques. We also will administer self-reported evaluations of the training experience to children and parents.

Protocol: 6-month follow-up assessments

Follow-up will occur 6 months post baseline and will match the pretraining and post-training assessments to assess long-term skill retention. Along with VR and field measures, we will gather descriptive data from parents and children on perceived efficacy of the interventions and learning.

Pedestrian measures

In all pedestrian simulations (both virtual and streetside), several measures will be collected. Two constructs will serve as primary outcomes, start delay and unsafe crossings. Start delay is defined as the temporal lapse between a safe gap between vehicles emerging and children entering that gap to cross the street. Previous research indicates it is an excellent proxy for the efficiency of pedestrian decision-making.¹⁴ Unsafe crossings will be the sum of ‘hits’ with simulated vehicles, plus ‘close call’ instances when the child was within 1 s of being hit by an oncoming vehicle. We also will assess several secondary measures, including time to contact (shortest distance between children and an oncoming virtual vehicle), attention to traffic (looks left and right divided by wait time), missed opportunities (rejected gaps ≥1.5 times participant’s crossing time), wait time (average time waiting to cross, divided by cars that pass while waiting) and gap size (temporal gap crossed within).

All VR measures will be collected electronically. Field-based measures will be assessed through video coding, as completed previously.^17,32 Two coders, masked to condition, will independently rate behaviour using established objective written criteria. κ will be computed on 20% of the sample to demonstrate inter-rater reliability.

Other measures

We will collect other measures as potential covariates. Parents will report basic demographic data (eg, child gender, race/ethnicity, birthdate; family socioeconomic status (SES)), child temperament, and children’s pedestrian behaviour and habits. Children will complete a brief intelligence screen.

Data analysis plan

Descriptive statistics for participants randomised to each intervention will be summarised for each outcome using standard measures of central tendency and variability. Several covariates may impact the relation between the intervention and the outcome measures of interest, including age, gender, temperament, attention during intervention, pedestrian experience and verbal intelligence. Because children will be randomised to the interventions, we expect these covariates to be balanced across intervention groups. To test this, we will assess the relation between each covariate and intervention group. If differences emerge, those variables will be included as covariates in primary analyses.

Primary inferential data analyses will address the study’s three objectives. Specific Aim 1 is to test whether immersive training in Cardboard improves children’s street crossing skills after six training trials at a rate not inferior to improvement seen in the semi-immersive kiosk VR. Given its relevance especially during the early stages of learning pedestrian safety, start delay will serve as the primary dependent measure and the hypothesis of interest for testing change, assuming that an improvement is reflected by a negative value (ie, a lower post-training value is better), is:

$${\mathrm{H}}_{\mathrm{o}}:{\mathrm{\Delta }}_{\text{LVR}}-{\mathrm{\Delta }}_{\text{IVR}}\ge \delta \text{vs.}{\mathrm{H}}_{\mathrm{A}}:{\mathrm{\Delta }}_{\text{LVR}}-{\mathrm{\Delta }}_{\text{IVR}}<\delta$$

where δ is the non-inferiority margin. We will perform ANCOVA to determine if the difference in change from postintervention to preintervention visits in the two groups falls below the non-inferiority margin, after adjustment for the baseline measure. If so, we will reject the null hypothesis and conclude training in Cardboard is not inferior to training in the kiosk VR. We will use similar methodology also to examine secondary outcome measures such as unsafe crossings and attention to traffic.

Specific Aim 2 is to (2a) demonstrate that most children achieve adult-level pedestrian functioning after 25 training sessions and (2b) demonstrate how many training sessions are sufficient to achieve this functioning. Because the safety of crossings is ultimately of greatest interest from a public health perspective, unsafe crossings will serve as the primary outcome for this aim; start delay will serve as a critical secondary outcome. Adult functioning will be estimated based on the performance of 311 healthy adults in four previous studies.^16,33–35 Aim 2a will be evaluated via simple descriptive statistics to determine what per cent of children achieve an unsafe crossings rate equivalent to adults in multiple simulated crossings. Aim 2b will also be evaluated descriptively, in this case by examining the average number of trials needed to achieve adult functioning across the sample, as well as the trajectory and distribution of that variable.

Specific Aim 3 is to test whether training in Cardboard improves children’s street crossing skills after 25 training trials, or fewer if adult functioning is achieved earlier, at a rate not inferior to improvement seen in the semi-immersive kiosk VR. Two dependent variables will be primary, start delay and unsafe crossings (other measures will be considered secondarily), and we will test the aim with the same hypothesis, analyses and assumptions as in Specific Aim 1. Two sets of analyses will be conducted, one to test change from baseline to postintervention and the second to test change from baseline to follow-up visits 6 months later.

CONCLUSIONS AND IMPLICATIONS

The planned research addresses a critical public health problem, uses novel technology and applies innovative research methodology. It epitomises the movement to translate theory and basic research findings into practice^36,37 through health behaviour change mechanism that can be broadly disseminated at minimal cost. If our hypotheses prove true, we will take steps to disseminate the programme widely to schools and communities.

Although we are evaluating this programme in the USA, it has potential for domestic dissemination and for use globally. We have conducted parallel research to train children in pedestrian safety in China³⁸ and envision a future where contextually relevant pedestrian environments are simulated and training is distributed using mobile smartphones to schoolchildren worldwide.