Slow-motion smartphone video improves interobserver reliability of gait assessment in ambulatory cerebral palsy

Abstract

Purpose:

Structured visual gait assessment is essential for the evaluation of pediatric patients with neuromuscular conditions. The purpose of this study was to evaluate the benefit of slow-motion video recorded on a standard smartphone to augment visual gait assessment.

Methods:

Coronal and sagittal plane videos of the gait of five pediatric subjects were recorded on a smartphone, including four subjects with ambulatory cerebral palsy and one subject without gait pathology. Twenty-one video scorers were recruited and randomized to evaluate slow-motion or normal-speed videos utilizing the Edinburgh Visual Gait Score. The slow-motion group (N = 11) evaluated the videos at one-eighth speed, and the normal-speed group (N = 10) evaluated the same videos at normal speed. Interrater reliabilities were determined by calculating intraclass correlation coefficients for each group as a whole, for each Edinburgh Visual Gait Score item, and after stratification by evaluator experience level.

Results:

The slow-motion group exhibited an intraclass correlation coefficient of 0.65 (95% confidence interval: 0.58–0.73), whereas the normal-speed group exhibited an intraclass correlation coefficient of 0.57 (95% confidence interval: 0.49–0.65). For less-experienced scorers, intraclass correlation coefficients of 0.62 (95% confidence interval: 0.53–0.71) and 0.50 (95% confidence interval: 0.40–0.59) were calculated for slow motion and normal speed, respectively. For more-experienced scorers, intraclass correlation coefficients of 0.69 (95% confidence interval: 0.61–0.76) and 0.67 (95% confidence interval: 0.58–0.75) were calculated for slow motion and normal speed, respectively.

Conclusions:

Visual gait assessment is enhanced by the use of slow-motion smartphone video, a tool widely available throughout the world with no marginal cost.

Level of evidence:

level I, randomized study.

Introduction

Structured assessment of gait is an essential component of the evaluation of pediatric patients with neuromuscular disorders. ¹ As these patients may present with complex gait disturbances, identifying the specific component(s) of gait that are contributing to pathologic gait patterns is a prerequisite for planning targeted, effective interventions, including orthotics and surgical correction. The gold standard for gait assessment is instrumented gait analysis (IGA) or three-dimensional gait analysis (3DGA), whereby trained observers utilize laboratory equipment to dynamically measure every component of gait, including kinematics, kinetics, electromyography, and pedobarograpy.^1,2 Given the limited availability and expense of this method, there has been increasing interest in structured visual assessment of gait using decision support tools. ¹ A variety of visual gait assessment tools have been developed, with the Edinburgh Visual Gait Score ³ (EVGS) exhibiting better reliability and validity than other instruments. ¹ The EVGS exhibits a high level of interobserver agreement when used to assess coronal and sagittal plane video of children with cerebral palsy (CP). Nonetheless, experienced gait analysts perform better than less-experienced observers utilizing this tool. ⁴

The purpose of this study was to validate a widely available, inexpensive adjunct to the use of EVGS for visual gait analysis—slow-motion video recorded on a smartphone—to facilitate and refine visual analysis. Modern smartphones have built-in functions to record video with high frame rates that can be replayed at a speed 25% or even 12.5% of normal. Although standard video can potentially be slowed down artificially, these high frame rates (upward of 240 frames per second) may be necessary to capture important gait deviations, as meaningful gait differences can happen on time scales as small as 20 ms (1/50 s), ⁵ and standard video would potentially miss these changes. This readily available “slow-motion” function may therefore facilitate more precise observation of gait parameters than is possible with normal-speed video or the naked eye. The hypothesis of this study was that visual gait assessment performed with the use of slow-motion coronal and sagittal plane video recorded on a smartphone would exhibit improved interobserver reliability relative to visual gait assessment with normal-speed smartphone video.

Materials and methods

Ethical approval

Ethical approval for this study was obtained from the study institution’s institutional review board (IRB #20-000895).

Video recording

Videos of the gait of five children were presented to video scorers (the study subjects) to compare slow-motion to normal-speed video. The children in the videos (video subjects) were aged 8 to 17 years. Four had a diagnosis of ambulatory CP (Gross Motor Function Classification System (GMFCS) I or II) with known gait differences, and one had a typical gait pattern. After informed consent was provided, coronal and sagittal plane videos were obtained using an iPhone X (Apple, Cupertino, CA), with each video subject walking down and back along the clinic hallway. Videos were recorded by study authors with the phone held by hand at a height of approximately 3 feet from the ground. The smartphone’s built-in slow-motion video function was used to record the videos. These videos were deidentified with the use of cropping and blurring functions and edited to show four full gait cycles (two gait cycles in one direction and two gait cycles in the other direction). The videos were embedded in a web page adjacent to a digital version of the EVGS survey (Figure 1). Two versions of the web page were created: one utilized slow motion videos at the default slow-motion viewing speed (12.5% speed, 240 frames per second), and the other included identical videos displayed at normal speed (100% speed, 30 frames per second).

Figure 1.

Online portal presenting gait videos to video scorers.

Video scoring

Medical students, resident physicians, attending physicians, and other gait specialists were recruited as video scorers (the study subjects). A sample size of 10 video scorers per group was estimated to achieve 95% confidence intervals (CIs) with bounds of +/–0.20 for an intraclass correlation coefficient (ICC) of 0.70. Twenty-one video scorers were ultimately recruited, which included three medical students, nine junior residents, three senior residents, two attending physicians, two physical therapists, and two non-physician gait specialists. Video scorers were stratified into less-experienced (medical students and Postgraduate Year (PGY) 1–3 resident physicians) and more-experienced (PGY 4–5 resident physicians, attending physicians, physical therapists, and other gait specialists) categories. Video scorers visited a web survey, were asked for their level of training, and were then randomly assigned by a software-based randomizer to view the web page with slow-motion videos or the web page with normal-speed videos. The web interface allowed scorers to pause, restart, and replay videos but not to change the play speed from its default (12.5% of normal for slow-motion videos and 100% of normal for normal-speed videos). Block randomization was utilized to generate groups with approximately equal numbers of scorers at each training level. There were five more-experienced and six less-experienced scorers assigned to the slow-motion group, and four more-experienced and six less-experienced scorers assigned to the normal-speed group.

Analysis

Responses to the EVGS were converted to a numeric ordinal scale. ICCs were calculated for the slow-motion and normal-speed groups in their entirety, as well as with stratification by experience level into less-experienced and more-experienced categories. ICC values were also calculated for each individual EVGS survey question for slow-motion and normal-speed groups separately. Analysis was performed in the R statistical software environment (The R Foundation, Vienna, Austria).

Results

The slow-motion group exhibited an ICC of 0.65 (95% CI: 0.58–0.73), whereas the normal-speed group exhibited an ICC of 0.57 (95% CI: 0.49–0.65; Figure 2). For less-experienced scorers (medical students and PGY 1–3 resident physicians), ICCs of 0.63 (95% CI: 0.54–0.71) and 0.51 (95% CI: 0.41–0.60) were calculated for slow motion and normal speed, respectively. For more-experienced scorers (PGY 4–5 resident physicians, attending physicians, physical therapists, and other gait specialists), ICCs of 0.68 (95% CI: 0.60–0.76) and 0.67 (95% CI: 0.58–0.75) were calculated for slow motion and normal speed, respectively.

Figure 2.

Intraclass correlation coefficients for gait assessment utilizing slow-motion and normal-speed video, stratified by experience level. Vertical lines indicate 95% confidence intervals.

ICC values for individual EVGS items ranged from –0.03 to 0.81 in the slow-motion group and –0.03 to 0.86 in the normal-speed group (Table 1). Utilizing the conventional cutoff of >0.60 for good agreement, ⁶ five items in the normal-speed group and seven items in the slow-motion group showed good agreement. Ten items exhibited qualitatively higher ICC values in the normal-speed group than the slow-motion group, with differences in ICCs ranging from 0.01 to 0.17. Seven items exhibited qualitatively higher ICC values in the slow-motion group than the normal-speed group, with differences in ICCs ranging from 0.09 to 0.42. Four items exhibited ICCs with differences greater than 0.20 between groups: maximum dorsiflexion in stance, maximum dorsiflexion in swing, terminal knee position in swing, and pelvic rotation in mid-stance. These items all demonstrated higher ICC values in the slow-motion group than in the normal-speed group.

Table 1.

Intraclass correlation coefficients for gait assessment with slow-motion and normal-speed video.

Condition	Normal speed	Slow motion	Difference
All subjects, all questions	0.57 (0.49 to 0.66)	0.65 (0.58 to 0.73)	0.08
Less experienced, all questions	0.51 (0.41 to 0.60)	0.63 (0.54 to 0.71)	0.12
More experienced, all questions	0.67 (0.58 to 0.75)	0.68 (0.60 to 0.76)	0.01
1. Initial contact	0.86 (0.65 to 0.98)	0.81 (0.57 to 0.97)	–0.05
2. Heel lift	0.70 (0.41 to 0.95)	0.70 (0.40 to 0.95)	–0.01
3. Max dorsiflexion in stance	0.33 (0.08 to 0.83)	0.75 (0.47 to 0.96)	0.42
4. Hindfoot varus/valgus	0.50 (0.17 to 0.94)	0.40 (0.11 to 0.91)	–0.10
5. Foot rotation in stance	0.45 (0.16 to 0.89)	0.40 (0.13 to 0.86)	–0.05
6. Foot clearance in swing	0.55 (0.24 to 0.92)	0.65 (0.34 to 0.94)	0.09
7. Max dorsiflexion in swing	0.39 (0.12 to 0.86)	0.74 (0.46 to 0.96)	0.35
8. Knee progression angle in stance	0.65 (0.35 to 0.94)	0.58 (0.27 to 0.92)	–0.08
9. Peak knee extension in stance	0.60 (0.29 to 0.93)	0.43 (0.15 to 0.88)	–0.17
10. Terminal knee position in swing	0.47 (0.17 to 0.89)	0.68 (0.39 to 0.95)	0.22
11. Peak knee flexion in swing	0.57 (0.26 to 0.92)	0.47 (0.18 to 0.89)	–0.11
12. Peak hip extension in stance	0.24 (0.03 to 0.77)	0.17 (0.00 to 0.71)	–0.07
13. Peak hip flexion in swing	0.08 (–0.04 to 0.60)	–0.01 (–0.07 to 0.39)	–0.09
14. Pelvic obliquity mid-stance	–0.03 (–0.08 to 0.33)	–0.03 (–0.08 to 0.30)	0.00
15. Pelvic rotation mid-stance	0.01 (–0.07 to 0.45)	0.25 (0.04 to 0.78)	0.24
16. Peak trunk sagittal position	0.59 (0.28 to 0.93)	0.54 (0.24 to 0.91)	–0.06
17. Maximum lateral shift	0.62 (0.31 to 0.94)	0.74 (0.46 to 0.96)	0.12

95% confidence interval bounds are provided in parentheses.

Discussion

This randomized experiment demonstrated that slow-motion smartphone video increases the interobserver reliability of visual gait analysis for pediatric patients with CP relative to normal-speed video. The ICC value associated with slow-motion video was 0.65, which is within a range typically interpreted as “good” correlation, whereas the ICC for normal-speed video was 0.57, in the “fair” range. ⁶ Subset analyses demonstrated that this benefit was greatest for less-experienced observers and was minimal for more-experienced observers. For four items in the gait assessment survey, the ICC value improved more than 0.20 points with slow-motion video: maximum dorsiflexion in stance, maximum dorsiflexion in swing, terminal knee position in swing, and pelvic rotation in mid-stance.

The ability to make reliable, structured gait observations is paramount to the successful treatment of ambulatory patients with CP. ⁷ While gait abnormalities can be broadly grouped into categories such as jump gait, crouch gait, and stiff gait, among others, an effective treatment approach requires accurately differentiating primary from compensatory joint movements and targeting primary problems without inadvertently worsening compensatory deviations. To differentiate primary from compensatory problems, at the very least, observers must make detailed observations of the position of each joint at each phase in the gait cycle. When resources permit, high-speed cameras in multiple planes, machine-readable markers, force plates, and pedobarographic plates can be utilized to quantify gait disturbances (i.e. 3DGA and IGA). However, this equipment and the expertise required to complete the assessment are not readily available in all clinical settings, nor is this level of specificity necessary in all circumstances. As such, decision support tools such as the EVGS have emerged to allow for a structured visual assessment of the relevant parameters needed to make appropriate clinical decisions when an arduous IGA is either not indicated or not available.

The findings from this study suggest that slow-motion smartphone video may improve the interobserver reliability of structured visual gait assessment relative to standard video played at a normal speed. As a widely available tool with no marginal cost, no upfront time burden such as an app download, and almost no learning curve, slow-motion smartphone video has the potential to be immediately and easily adopted by clinicians treating ambulatory patients with CP. Widespread adoption of this accessible tool may result in more reliable recognition of gait patterns and dependable differentiation between primary and compensatory gait disturbances, potentially improving ambulatory CP treatment globally, with the greatest impact on resource-limited locations.

The value added from slow-motion video in this study was more apparent in less-experienced observers. Less-experienced observers showed an improvement in ICC from 0.51 (fair) to 0.63 (good) when slow-motion video was used instead of normal-speed video, whereas the ICC values for more-experienced observers were 0.67 and 0.68 for normal speed and slow motion, respectively. This is in line with previous research demonstrating that less-experienced observers tend to have lower reliability in gait assessment. Ong et al. ⁸ had six medical students review the gait videos from the original EVGS validation study and showed that inexperienced observers (medical students) had less interobserver reliability for individual items relative to more-experienced observers. Del Pilar Duque Orozco et al. ⁴ asked seven raters to score 30 gait videos of children with spastic CP with the EVGS; the three most-experienced observers (10+ years) had a higher percentage of complete agreement for each of the EVGS items than the less-experienced (2–5 years) and inexperienced observers. Brown et al. ⁹ showed a similar reliability increase with greater experience in observers utilizing the Visual Gait Assessment Scale. This is the first study we are aware of, however, that demonstrates the capability of readily available technology to improve the reliability in less-experienced observers. This suggests that slow-motion smartphone video may be particularly useful in training environments or when observations need to be made by individuals who are new to gait assessment.

The reliabilities of four EVGS items seemed to be particularly enhanced by the slow-motion video: maximum dorsiflexion in stance, maximum dorsiflexion in swing, terminal knee position in swing, and pelvic rotation in mid-stance. All four of these items rely on sagittal plane observations, and three of the four focus on the distal segments (foot, ankle, knee). In general, distal segment observations had higher ICC scores across both video types than proximal segment observations (hip, pelvis, trunk). Previous researchers reported similar findings, demonstrating that reliability is higher for distal segments in visual gait assessment compared to proximal segments.^4,9 This study pushes this observation further, showing that not only are distal segment observations more reliable in general, but that slow-motion video preferentially enhances distal segment observations. This supports the clinical utility of utilizing slow-motion smartphone-aided assessment as sagittal plane observations of the ankle and knee are key to classifying CP gait pathology and to guiding treatment. ⁷

Aroojis et al. previously published the results of a similar study utilizing the EVGS in conjunction with slow-motion video, including two gait observers and utilizing an angle measurement app for assessment. ¹⁰ These authors found that inter-observer reliabilities were moderate or better (>0.40) for 11 items and slight/fair (≤0.40) for 7 items. In contrast to this previous work, this study included a comparison group (normal-speed video) and did not utilize an angle measurement app. Although such a tool may be helpful, it may increase both the time burden associated with gait evaluation and the learning curve of smartphone-aided assessment without commensurate benefit. Moreover, this study included multiple observers with variable levels of experience, allowing for a more robust analysis. Including novice gait evaluators demonstrated the increased utility of this tool for clinicians new to gait evaluation.

One potential drawback of using slow-motion video for visual gait analysis relative to normal-speed video or the naked eye is that more time is necessarily required to view and analyze video at one-eighth speed relative to normal speed. Because of this, slow-motion smartphone video will not be useful in every gait evaluation, but it still has the benefit of requiring fewer resources and less set-up than most if not all other methods of augmenting naked-eye assessment.

While this study demonstrates the benefits of utilizing slow motion video for in-clinic gait analysis, it is not without limitation. Principally, because 3DGA was not completed for each participant, we were unable to determine the impact of slow-motion smartphone video on evaluator accuracy (i.e. in accordance with a gold standard). Further research is needed to evaluate any potential gains in accuracy with slow-motion smartphone video analysis. Another potential limitation was that the reliability measure chosen, the ICC, does not have a validated method for hypothesis testing, precluding the opportunity to report that the demonstrated increase in reliability utilizing slow-motion video was or was not “statistically significant.” We nevertheless used the ICC as it is a common, readily interpretable measure of reliability. The 95% CIs were provided to permit the interpretation of statistical confidence. Furthermore, there is precedent for the qualitative comparison of ICC and kappa values between groups in visual gait assessment.^4,8,9 Finally, our results may not be generalizable to gait assessment for patients with gait pathologies not related to CP; however, the requisite components of gait are constant regardless of the context. It requires few jumps of logic to assume that the technology would be potentially useful for other etiologies of gait disturbance. Furthermore, we posit that the cost and risks associated with using slow-motion smartphone video are sufficiently low such that the threshold of evidence required to generalize the use of this technology should also be relatively low.

Slow-motion smartphone-aided gait assessment may be a useful adjunct to the evaluation of pediatric patients with ambulatory CP. This simple, low-resource, easy-to-use, and intuitive tool improved the reliability of gait observations relevant to treatment planning for patients with CP-associated gait pathology. Four gait observations, in particular, improved most with the slow-motion video: maximum dorsiflexion in stance, maximum dorsiflexion in swing, terminal knee position in swing, and pelvic rotation in mid-stance. As less-experienced observers saw the most benefit, this tool may be most effective in training medical students, residents, and other learners participating in the care of patients with CP. Future studies might evaluate whether smartphone-aided assessment is sensitive to treatment-induced changes in gait.