Effects of smartphone texting on the visual perception and dynamic walking stability

Abstract

Mobile phone use while walking can cause dual-task interference and increases safety risks by increasing attentional and cognitive demands. While the interference effect on cognitive function has been examined extensively, how perception of the environment and walking dynamics are affected by mobile phone use while walking is not well understood. The amount of visual information loss and its consequent impact on dynamic walking stability was examined in this study. Young adults (mean, 20.3 years) volunteered and walked on a treadmill while texting and attending to visual tasks simultaneously. Performance of visual task, field of regard loss, and margin of stability under dual-task conditions were compared with those of single-task conditions (i.e., visual task only). The results revealed that the size of visual field and visual acuity demand were varied across the visual task conditions. Approximately half of the visual cues provided during texting while walking were not perceived as compared to the visual task only condition. The field of regard loss also increased with increased dual-task cost of mobile phone use. Dynamic walking stability, however, showed no significant differences between the conditions. Taken together, the results demonstrate that the loss of situational awareness is unavoidable and occurs simultaneously with decrements in concurrent task performance. The study indicates the importance of considering the nature of attentional resources for the studies in dual-task paradigm and may provide practical information to improve the safe use of mobile phones while walking.

INTRODUCTION

Advances in smart phone technology have allowed pedestrians to carry on telephone conversations and multimedia activities, including listening music, watching movies, and playing games while walking. Unfortunately, a number of studies have pointed out the safety issues of pedestrian such as distracted walking incidences related to mobile devices (Lamberg and Muratori, 2012; Stavrinos et al., 2011). For instance, 17% adults in of all United States have reported bumping into some object while texting and walking (Madden and Rainie, 2010). A dual-task interference caused by operating a mobile phone while walking was associated with increased cognitive demands placed on working memory and physical demands associated with manipulation of the phone, and reduced situational awareness (Horberry et al., 2006; Rubinstein et al., 2001; Schabrun et al., 2014).

While decreased availability of visual information of surroundings is one of the contributing factors threatening the safety of pedestrians using mobile phones (Hatfield and Murphy, 2007), what aspects of the visual information influence the attentional capacity to navigate through a complex and changing environment is not well understood. Situational awareness is defined as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” (Endsley, 1988). In particular, information on the situational awareness while walking and texting has been limited. Studies have provided only fragmentary evidence by measuring visually noteworthy objects in the environment (Hyman et al., 2009), the number of times the person looked up from the phone (Plummer et al., 2015), and quantified unsafe behavior when mobile phone users cross the street (Nasar et al., 2008; Stavrinos et al., 2011). These investigations are considered relatively subjective, and do not reflect the various nature of visual information that pedestrians interact with in the real world.

During walking, the body is in a continuous state of imbalance, so that the center of mass (COM) and momentum of the body need to be controlled in each subsequent step in order to prevent a fall or gait imbalance (Lugade et al., 2011). This is why dynamic stability of walking is known to be an indicator assessing the risk of falls in the elderly as well as gait imbalance in complex environment. Studies have reported decreased walking speed and increased sway, stride length and width as a negative impact of mobile phone use while walking. It is because perceptual and cognitive distraction caused by mobile phone use while walking may result in walking errors. The impact of dual-task on dynamic stability in walking was greater in medial-lateral direction, and pronounced more with physically demanding tasks compared to the cognitive tasks (Marone et al., 2014). As yet, no studies have attempted to evaluate dynamic walking stability in dual-task situation where visual attentional demands to the environment are required.

Another issue is the head movement control while walking and texting. Locomotion is a complex motor skill that has to deal with dynamic changes in the environment by integrating sensory-motor information efficiently. The head, a natural frame of reference for movement control that contains the visual and vestibular systems, needs to be modulated to maintain the stability in space (Paillard, 1988; Pozzo et al., 1990). Head stabilization could ensure the clear vision through gaze stability (Bril and Ledebt, 1998). If head movements were random during a locomotor task, the sensory information transmitted would be difficult to interpret and information issued from the locomotor movement itself would be difficult to disentangle from noise information (Berthoz and Pozzo, 1988). Head movement control with divided attention, therefore, would negatively impact not only the situational awareness but also the dynamic stability of walking.

More studies need to provide evidence on the effects of mobile phone use while walking on the situational awareness and walking stability and how these effects are associated with the nature of visual information. The purposes of this study were: [1] to examine the effect of mobile phone use on situational awareness and [2] to quantify dynamic stability while walking and texting. Visual cue detection rate, field of regard loss, and dynamic stability of walking between single and dual task conditions were compared to identify the effects of mobile phone use while walking.

MATERIALS AND METHODS

Participants

Twenty subjects (10 males and 10 female; 20.3±1.2 years; 1.7±0.1 m; 69.8±13.7 kg) were recruited for this study. Individuals were excluded who did not use a touch screen mobile phone or less than a year of experience using a typical QWERT keyboard phone. Participants were required to have normal or corrected-to-normal vision and be healthy and without any known neurological or musculoskeletal abnormalities. Participants provided informed consent and all procedures were approved by the Institutional Review Board of University of Massachusetts Amherst (approval number: 014-1928).

Apparatus

An eight-camera ProReflex Motion Capture system (Qualisys, Gothenburg, Sweden) sampling at 120 Hz was used to gater spatial and temporal kinematic data. The cameras were positioned around the treadmill (TrakMaster, JAS Fitness Systems, Newton, KS, USA) and captured reflective markers attached to the participants. To evaluate mobile phone use, all participants used the same phone for the experiment (Galaxy S2, Samsung, Seoul, Korea) and texting performance was measured using a mobile application software (Fast Type ver. 1.55; Android, Vauréal, France). The application software provided a random sequence of words and measured speed and accuracy of texting in a 60-sec fixed time duration. To evaluate the visual task, three LCD monitors (1704FPV, Dell, Round Rock, TX, USA) were located in front of the treadmill (Fig. 1). A center monitor was positioned 2 m away from the treadmill center at eye level (1.6 m from the treadmill) and two side monitors (one top-left and one bottom-right) were positioned 45° horizontally and 22.5° vertically from the center monitor. The root mean square distance from the estimated eye position to each monitor screen was equal to 2 m. A white fabric backdrop (4.5 m×3.5 m) was located behind the monitors so that the monotone background enhanced the contrast of image displayed in the monitor to the background. A wireless microphone (DKW-1 GT, Nady Systems Inc., Emeryville, CA, USA) was used to measure the participants’ verbal response. Precise control of visual cue and acquisition of verbal response were programmed in Matlab (The MathWorks, Natick, MA, USA) in the PsychToolbox environment (Brainard, 1997; Pelli, 1997).

Fig. 1.

Experimental setup. Three black circles indicate the projection on the floor of the location of monitors in the transverse plane. A center monitor was positioned 2 m away from the treadmill center at eye level (1.6 m from the treadmill) and two side monitors were positioned 45° horizontally (Hor) and 22.5° vertically (Ver) above (top-left) and below (bottom-right) from the center monitor. The root mean square distance from the estimated eye position to each monitor screen was equal to 2 m.

Procedures

Participants performed a minimum of 5 warm-up texting trials while seated until they felt comfortable about the mobile phone and texting application software. After instrumenting participants with markers to the anatomical landmarks, they were asked to perform baseline measurement in two different conditions: a continuous straight-line walking (walk only) and a walking while texting (walk+text) for 60 sec. For walk+text, the priority was given to the texting task. The treadmill speed was set at 2 mph, measured and maintained throughout the experiment.

Upon the completion of the baseline measures, each participant completed four conditions of a visual task (visual task) while walking and texting. Two different size of visual fields (visual field) were established by displaying visual cues either on the center monitor only (narrow) or across three monitors (wide). Also, two types of visual task (task type) were used to assess the ability to detect the visual cues in the environment. A simple color change-detection task was used for the general detection and the screen color changed from white to gray for a fixed period of time (i.e., 1 sec) (peripheral). Dynamic visual acuity was assessed by presenting Landolt C optotypes (right orientation default) on the screen, followed by the change of the orientation of optotype to one of three configurations (up, down, and left) (foveal). One optotype size representing 20/125 Snellen ratios was used. Participants were instructed to verbally state either “gray” or “C (si:)” when the detection of color change or the identification of orientation change of the optotype was made. A response after the end of the change was allowed until the next visual cue appeared. The 9 visual cues (i.e., color or optotype orientation change) were presented for 60 sec in each trial and the time interval between visual cues were randomized.

The participants were asked to complete as many words as accurately as possible while simultaneously identifying any changes in the visual environment as accurately and as quickly as possible. No priority was given regarding texting, walking, and visual task, but they were instructed to use their normal method of texting while naturally walking. Baseline measurement of the visual task was not given to the participants. However, pilot testing (n=3) of the visual task, performing the visual task with the instruction to focus on visual task only while standing and walking at 2 mph at a 2-m distance from the monitor, indicated that nearly all visual cues were detected and identified in both visual cue types (standing >98%; walking >96%). Therefore, the baseline to compare the outcome of visual task while walking and texting was set at 100%. Each condition was repeated three times and the order of visual task condition was randomized across participants.

Data analysis

One subject data was removed from the analysis due to the marker dropout and therefore a total of 19 participants’ data were analyzed. For the visual task performance, the total percentage of visual cues detected was calculated in each visual task (total 27 = 9 visual cue×3 trials). Response time, time elapsed between visual cue presentation and verbal response, was calculated in millisecond. Field of regard loss (FORL, see below) and margin of stability (MOS, see below) were determined based on 30 strides occurred in the middle of the 60-sec data collection period.

Field of regard loss

The level of perception on the environment was assessed by the FORL (Palmer et al., 2012). FORL was developed to coarsely measure the changes in the field of regard relative to the environment. FORL was calculated as the sum of the products of the absolute value of sagittal head orientation and the inverse of the percentage of maximum vertical head height at each point across the gait cycle (Equation 1). Given that the FORL in upright stance is zero, the relative change in FORL was calculated, and each trial was normalized to 100% of the event.

$$FORL\hspace{0.17em}(°)=\sum _i^n\left[\mid Head\hspace{0.17em}Orientation\mid \times \left(\frac{1}{\%\hspace{0.17em}Max\hspace{0.17em}Head\hspace{0.17em}Height}\right)\right]$$ (1)

with Head Orientation in pitch angle and Max Head Height (100%) measured in upright stance. The greater FORL indicates the reduction of visual attention to the information-rich area compared with the upright stance position (i.e., zero FORL).

Dynamic stability

Dynamic stability was quantified by the MOS (Equation 2), which was calculated as the distance between the extrapolated COM (xCOM; Equation 3) and the lateral border of the base of support (BOS; Equation 4) (Rosenblatt and Grabiner, 2010). We calculated the MOS from heel strike to contralateral toe off and selected the minimum value. The positive MOS indicates that xCOM is located within the BOS and dynamically stable. Analysis of MOS in this study was limited in medial-lateral direction (Marone et al., 2014).

$$MOS=BO{S}_{lateral}-xCOM$$ (2)

$$xCOM=P_x+\frac{V_x}{W_{\mathit{0}}}$$ (3)

where P_x = COM position, V_x = COM velocity, $W_{\mathit{0}}=\sqrt{\frac{gravity}{vertical\hspace{0.17em}COM\hspace{0.17em}position}}$

$$BO{S}_{lateral}=hee{l}_{ML}\pm (\%\hspace{0.17em}\text{contact})(0.152BH)cos\theta \pm (0.0275BH)sin\theta$$ (4)

where BH=body height, heel_ML=frontal plane position of the heel marker, θ=angle between horizontal line and line connecting the heel and metatarsal marker

Statistical analysis

All outcome variables were averaged across the three trials within each task condition. All variables were compared between conditions with a 2 visual field×2 task type analysis of variance (ANOVA) with repeated measures (RM-ANOVA). A one-sample t-test was used to determine whether visual task performance across all visual task conditions was different from the baseline, which was set at 100 % by definition. For FORL and MOS, the baseline performance (walk only) was compared to the average performance across dual task conditions (walk+text, visual task) using a one-way ANOVA with repeated measures. In the case of violations of the sphericity assumption, F values were adjusted with the Greenhouse-Geisser procedure. Significant main and interaction effects were further analyzed using paired t-tests at each level. Statistics were performed in SPSS ver. 16.0 (SPSS Inc., Chicago, IL, USA). The level of significance for all statistical tests was set at P<0.05.

RESULTS

Visual task performance

There was a significant difference between baseline measures (100%) and visual task conditions (52.6%±4.0%) for the response rate [t(18)=13.43, P<0.01], indicating a significant effect of texting on visual information pickup. There was a significant interaction between visual field and task type for the response rate [F(1,18)=12.48, P<0.01] and response time [F(1,18)=5.85, P<0.05] (Table 1). Post hoc comparisons showed that both response rate and time did not differ in narrow condition, but decreased response rate [t(18)=5.70, P<0.01] and increased response time [t(18)=2.61, P=0.017] were observed in foveal condition.

Table 1.

Comparison of visual task performance between visual task conditions

Variable	Visual field
	Narrow, task type		Wide, task type
	Peripheral	Foveal	Peripheral	Foveal
Response rate (%)	62.18±4.63	58.67±3.91	56.14±3.43	33.72±4.38
Response time (ms)	1,102±29	1,117±27	1,170±34	1,327±45

Values are presented as mean±standard deviation.

Field of regard loss

FORL compared across conditions (walk only, walk+text, visual task) showed that the greatest FORL was observed in walk+text condition, in which the priority was given to the texting [F(2,36)= 148.14, P<0.01] (Fig. 2 left). Analysis for the head pitch angle also showed the greatest downward head orientation in the walk+text condition, indicating the dominant contribution of head pitch angle to the FORL [F(2,36)=175.22, P<0.01] (Fig. 2 right). Two-way RM-ANOVA (visual field×task type) revealed a significant main effect of task type on the FORL [F(1,18)=5.87, P= 0.026] (Fig. 3 left). Head pitch angle was greatest in the wide-foveal condition [F(1,18)=6.40, P=0.021] (Fig. 3 right).

Fig. 2.

Comparisons of field of regard loss between baseline and dual-task conditions Values are presented as mean±standard error of the mean.

Fig. 3.

Main effect of task type on field of regard loss (left) and visual field × task type interaction effect on head orientation (right). Values are presented as mean±standard error of the mean.

Dynamic stability

Dynamic walking stability measured by MOS was compared between single (walk only) and dual task conditions (walk+text, visual task) (Fig. 4). MOS was greatest in the walk+text (43.6 ±6.88 mm) and smallest in the walk only condition (35.8± 7.15 mm). However, the differences were not statistically significant [F(2,36)=1.41, P=0.255]. Neither the visual field nor the task type had an influence on the MOS (P>0.05).

Fig. 4.

Comparisons of margin of stability between baseline and dual-task conditions. Values are presented as mean±standard error of the mean.

DISCUSSION

This study demonstrated that texting while walking results in a significant dual-task interference on the ability to identify changes in the environment as well as dynamic field of view that pedestrian’s visual attention is available. Findings regarding the specificity of loss of visual information associated with the nature of attentional resources provide meaningful information to improve the safe use of mobile phone while walking.

The results from the visual task performance quantified the impact of mobile phone use while walking on the visual perception in the environment. Nearly half (45.3%) of the visual information provided during walking and texting was not detected. This study also demonstrated that the loss of visual information is associated with the specific aspects of visual information. Decreased rate of visual cue detection and increased time to detect the visual cue were augmented with increases in visual field size and demand on the foveal vision. Rapid gaze shifts are achieved by two separate functions: they first bring the image of an object, detected in the retinal periphery, to the fovea; and then they reorient the head and eye in space to view a new part of the visual scene using ocular saccades (Laurutis and Robinson, 1986). The lower visual cue detection rate and longer response time in the Wide-Foveal condition, therefore, might be associated with neuro-muscular delays resulted from the frequent head rotations. The greater head pitch angle in Wide-Foveal condition (Fig. 3 right) may indicate the strategy employed by the participants, facilitating the visual cue search on the two side monitors. These findings suggest the importance of understanding what aspects of the visual information influence the attentional capacity to navigate through a complex and changing environment. Therefore, future study should consider that the nature of visual information that pedestrians interact with in the real world varies significantly in space as well as time such as orientation in the visual field, duration available, or visual acuity required to discriminate details.

Dual task interference was also significant in the FORL, and walk+text condition exhibited the greater FORL compared to the visual task conditions (Fig. 2 left). Given that the FORL in dynamic movement situation includes the information for the prospective control of the oncoming movement (Palmer et al., 2012), this result should be understood that FORL is not only the loss of information on the visual cue in the environment but also the attentional loss on the area that contains the information critical for the control of stable walking. Moreover, the results revealed that the changes in head pitch angle were the major contributing factor in the FORL. It is assumed that increased engagement to the texting in the walk+text condition would have influenced the degree of downward head pitch angle as well as the duration of gazing the smart phone screen. Indeed, the height of the head was maintained constantly throughout the walk (i.e., −2 cm; 98.8% of the standing height). Moreover, because the maximum height difference from double-support phase to single-support phase during normal walking is approximately 9.5 cm (Perry, 1992), the FORL associated with changes in head height would have been negligible.

The present study failed to demonstrate the dual task interference during walking and texting on the dynamic stability in walking. The MOS did not differ across the conditions but even increase in the dual task conditions. Hallemans et al. (2010) suggested that the loss of visual information associated with visual acuity and visual field could impact the locomotion in route planning, orientation and the maintenance of stability. For instance, the reduced visual information during walking could result in the more cautious walking patterns. Unaffected dynamic walking stability in this study, however, should not be interpreted that texting while walking is safe in terms of walking stability. Studies have reported that young adults were able to selectively prioritize their attention between texting and walking, according to specific instruction (Bloem et al., 2001; Plummer et al., 2015). Therefore, the possibility that the equivalent level of walking stability might be compromised by the loss of visual cue detection and FORL should be considered.

In conclusion, the study demonstrated that the loss of visual information pickup during walking and texting is unavoidable and occurs simultaneously with decrements in concurrent task performance. The study indicates the importance of considering the nature of attentional resources for the studies in dual-task paradigm and may provide practical information to improve the safe use of mobile phones while walking.