Examination of the gait evaluation index using shoes with built-in motion sensors: intra-rater reliability during comfortable walking and fastest walking

Abstract

[Purpose] To determine the intra-rater reliability of gait evaluation data obtained using shoes equipped with built-in motion sensors. [Participants and Methods] Thirteen healthy adults were recruited and gait evaluation data (stride length, stride speed, stride duration, stance phase duration, and swing phase duration) were obtained during comfortable and fastest possible walking. Gait evaluation was repeated after a one-week interval. The intra-class correlation coefficients and systematic errors of the gait evaluation data during comfortable walking and fastest walking were determined. [Results] The test-retest gait assessment data had good intra-rater reliability. No systematic errors were observed in any of the gait evaluation data. [Conclusion] Intra-rater reliability of gait evaluation using shoes with in-built motion sensors was verified, indicating that these data can be used for clinical applications.

INTRODUCTION

Two main methods exist that physical therapists use to evaluate gait patterns. One is qualitative analysis based on the observations of the physical therapist, and the other is quantitative analysis using equipment such as three-dimensional motion analyzer^{1, 2)}. In clinical practice, the former type of qualitative analysis is most common. This is because quantitative analysis using equipment such as three-dimensional movement analysis equipment is difficult to apply in medical and welfare facilities because of the need for environmental and condition control, as well as the high economic and human costs. By contrast, qualitative analysis based on the observations of the physiotherapist has the advantages of a wide range of applications, immediacy, and low costs. However, it is often inadequate as an objective evaluation method, and its low reliability is a disadvantage³⁾. Therefore, a combination of qualitative and quantitative gait analysis using simple and low-cost instruments is ideal and practical for gait analysis.

Recently, the use of small wearable sensors has attracted attention as simple and low-cost devices for measuring posture and gait^4,5,6). Among these, shoes with built-in motion sensors have been developed and put into practical use⁷⁾. Shoes with built-in motion sensors are noninvasive and easy to wear; therefore, they can be easily used in clinical practice. Measure the gait data for each walking cycle is possible by wearing shoes with built-in motion sensors on both feet. The motion sensor acquires acceleration and angular rate data from an accelerometer and angular rate sensor, respectively. Time integration of the angular rate data provides the angular information. The velocity is calculated by integrating the acceleration data over time and the position data is obtained by further integrating the acceleration data over time. Therefore, combining qualitative gait analysis based on the observations of the physical therapist with quantitative analysis using motion sensor shoes is expected to become a useful evaluation tool. However, the reliability of shoes with built-in motion sensors has not been investigated sufficiently. Currently, only one reported study has demonstrated that shoes with built-in motion sensors have validity relative to that of three-dimensional motion analysis systems⁷⁾. This system has been validated in a previous study using the same sensor-embedded shoes as in the present study. In that study, the relative validity was tested in nine healthy adult participants by comparing the measurements obtained from the sensor-embedded shoes and the three-dimensional motion analysis system used in the current research. The results showed excellent relative validity for eight gait parameters: stride length, stride duration, stride frequency, stride speed, vertical height, stance phase duration, swing phase duration, and sagittal angle at initial contact. The intra-rater reliability of test-retest for shoes with built-in motion sensors has not yet been examined. Additionally, previous studies have performed measurements on a treadmill, whereas in clinical practice, gait is often assessed on level ground. Therefore, the accumulation of basic data is urgently required for clinical use. Although several previous studies have examined the intra-rater reliability of other sensor-embedded gait-assessment devices, the reliability of the specific shoe-type motion sensor device used in this study has not yet been evaluated^{8, 9)}. Understanding the existence of systematic errors in the measured values is necessary to verify the reliability of shoes with built-in motion sensors and judge the effectiveness of rehabilitation. In addition, in clinical practice, we often focus on improving gait speed^{10, 11)}. Therefore, verifying whether shoes with built-in motion sensors can accurately reflect the characteristics of walking at different speeds is important. In clinical settings, walking patterns are evaluated under two conditions: comfortable walking and fastest walking. Comfortable walking reflects mobility in daily life. In contrast, maximum walking reflects preliminary motor abilities and muscle strength^{12, 13)}. Because each walking speed condition reflects different abilities, the evaluation was performed under two conditions. Therefore, to utilize shoes with built-in motion sensors in clinical settings, we believe that verifying their reliability under two different conditions is necessary: comfortable walking and fastest walking.

Therefore, this study examined the intra-rater reliability and existence of systematic errors in shoes with built-in motion sensors using relative and absolute reliability and Bland–Altman plot analysis. If the intra-rater reliability of the sensor-embedded shoes can be confirmed, they can be used as a tool to evaluate changes in walking ability over time during inpatient rehabilitation. Future studies should target the elderly and clinical populations; however, this time, as a preliminary study, healthy young people were targeted.

PARTICIPANTS AND METHODS

The participants were 13 healthy young adults (7 men, 6 women). Their mean age and height were 19.2 ± 1.0 years and 165.6 ± 8.2 cm, respectively. Participants were defined as those who did not have any functional impairment of the trunk or limbs that would interfere with walking at the time of measurement. This study was approved by the ethics committee of Sendai Seiyo Gakuin College and was performed in accordance with the Declaration of Helsinki revised in October 2013 (approval no. 0537). The participants were recruited through announcements at my university. The purpose and outline of the study were explained and participation was voluntary. Those who expressed a desire to participate were asked to sign their names on a consent form, and consent to participate in the study was obtained. It was also fully explained verbally and in writing (request and explanation regarding the study) that this study had no bearing on the grade.

Walking was measured using shoes (Fig. 1) with a built-in motion sensor (ORPHE Inc., Tokyo, Japan). For the gait measurements, we used an indoor walking path (total length: 11 m) with 3 m each for acceleration and deceleration. The first walking task consisted of two conditions: comfortable walking and walking at the fastest speed. The order of measurements was as follows: comfortable walking was performed first, followed by the fastest walking¹⁴⁾. The second measurement was performed one week after the first measurement. In the comfortable walking test, the participants were instructed to walk at their usual speed, and in the fastest walking test, they were instructed to walk as fast as possible. The motion sensor built into the shoes is a wearable device with dimensions of 45 mm × 29 mm × 14 mm and a weight of 20 g. The motion sensor acquired data at 200 Hz. The acceleration and angular velocity data in the three axial directions acquired by the motion sensor at 200 Hz were analyzed in real time to calculate the walking state for each step and were stored in the built-in flash memory. Acceleration and angular velocity data recorded using motion sensors were processed using a proprietary gait analysis software (ORPHE Inc., Tokyo, Japan) to calculate the gait parameters. This software automatically detects the timing of initial contact (IC), foot flat (FF), and toe-off (TO) from the acceleration and angular velocity obtained by the motion sensor and calculates gait evaluation parameters based on the detected timing of each of the consecutive FF data. No manual confirmation or correction was required during the gait-event detection process. The speed, relative position, and relative angle were calculated using an inertial navigation system based on zero-velocity updates, which correct drift errors by assuming zero velocity during the stance phase¹⁵⁾. The following five gait evaluation parameters were calculated: stride length (anteroposterior displacement of the foot during one gait cycle), stride speed (average speed of the foot during one gait cycle), stride duration (time required for one gait cycle), stance phase duration (time the foot is in contact with the ground during one gait cycle), and swing phase duration (time the foot is in the air during one gait cycle)⁷⁾. The average values of the four walking cycles from the start of walking and from the fourth walking cycle¹⁶⁾ to the seventh walking cycle, were calculated. In this study, we examined the intra-rater reliability and systematic error of gait evaluation data obtained from shoes equipped with built-in motion sensors.

Fig. 1.

Shoes with built-in motion sensors.

Special shoes with a sensor space in the sole of the shoe were used for the measurements (SHIBUYA 2.0, ORPHE Inc., Tokyo, Japan). Built-in under the insole at about 33% of the total length from the heel. Since the size of the internal space and the motion sensor are the same, there is little risk of displacement. Before walking, the participants were required to stand for at least 3 sec to perform calibration.

For the statistical analysis, the normality of the gait evaluation data (stride length, stride speed, stride duration, stance phase duration, and swing phase duration) obtained from the motion sensors was checked. In this study, we calculated the intraclass correlation coefficients (ICC) for the intra-rater reliability. The data were measured for two days and re-measured one week after the first measurement date, and the mean values of each were examined using ICC (1, 1). A 95% confidence interval (95% CI) was calculated for each patient. For absolute reliability, Bland–Altman plot analysis was used to check for systematic errors, and the 95% CI (MDC₉₅) of the minimal detectable change (MDC) was used to determine the measurement error. For the Bland–Altman plot analysis, the 95% CI of the difference between the two measurements was calculated to determine whether an additive error existed between the two measurements. When the interval contained zero, the measured values were distributed in a certain direction, and the presence of an additive error was judged. Regression equations were calculated to determine the presence or absence of proportional errors, and the significance of the regression was evaluated. If the regression was found to be significant, it was judged that a proportional error existed. MDC₉₅ was calculated by using the standard error of measurement (SEM) and the standard deviation of the difference between two means (SDd), where SEM was calculated using SEM = SDd ÷ Ö2, and MDC₉₅ was obtained using MDC95 = SEM × 1.96 × Ö2. The significance level was set at p<0.05. All data were analyzed and evaluated using SPSS Statistics for Windows (version 24.0; IBM Corp., Armonk, NY, USA).

RESULTS

The results of gait evaluation data obtained from a motion sensor during comfortable walking and fastest walking are shown in Tables 1 and 2. The ICC (1,1) for stride length during comfortable walking was 0.90 (95% CI: 0.74–0.97), with an SEM of 0.04 m and an MDC₉₅ of 0.11 m. Similarly, stride speed showed an ICC (1,1) of 0.90 (95% CI: 0.71–0.97), an SEM of 0.06 m/sec, and an MDC₉₅ of 0.15 m/sec. Stride duration had an ICC (1,1) of 0.90 (95% CI: 0.73–0.97), with an SEM of 0.02 sec and an MDC₉₅ of 0.05 sec. The ICC (1,1) for stance phase duration was 0.83 (95% CI: 0.54–0.94), with an SEM of 0.02 sec and an MDC₉₅ of 0.05 sec. Swing phase duration showed an ICC (1,1) of 0.89 (95% CI: 0.68–0.96), with an SEM of 0.01 sec and an MDC₉₅ of 0.02 sec. During fastest walking, stride length showed an ICC (1,1) of 0.92 (95% CI: 0.76–0.97), with an SEM of 0.05 m and an MDC₉₅ of 0.14 m. Stride speed had an ICC (1,1) of 0.90 (95% CI: 0.70–0.97), an SEM of 0.06 m/sec, and an MDC₉₅ of 0.16 m/sec. Stride duration showed an ICC (1,1) of 0.85 (95% CI: 0.59–0.95), with an SEM of 0.04 sec and an MDC₉₅ of 0.11 sec. The ICC (1,1) for stance phase duration was 0.90 (95% CI: 0.71–0.97), with an SEM of 0.03 sec and an MDC₉₅ of 0.09 sec. Swing phase duration had an ICC (1,1) of 0.82 (95% CI: 0.52–0.94), with an SEM of 0.01 sec and an MDC₉₅ of 0.04 sec.

Table 1. Reliability of gait evaluation data during comfortable walking.

	Mean ± SD		ICC (95% Cl)	SEM	MDC95
	First session	Second session
Stride length (m)	1.58 ± 0.13	1.60 ± 0.15	0.90 (0.74–0.97)	0.04	0.11
Stride speed (m/sec)	1.51 ± 0.18	1.54 ± 0.18	0.90 (0.71–0.97)	0.06	0.15
Stride duration (sec)	1.05 ± 0.05	1.05 ± 0.05	0.90 (0.73–0.97)	0.02	0.05
Stance phase duration (sec)	0.67 ± 0.05	0.66 ± 0.04	0.83 (0.54–0.94)	0.02	0.05
Swing phase duration (sec)	0.38 ± 0.02	0.38 ± 0.02	0.89 (0.68–0.96)	0.01	0.02

n=13. SD: standard deviation; ICC: intraclass correlation coefficients; 95%CI: 95% confidence interval; SEM: standard error of measurement; MDC₉₅: minimal detectable change 95.

Table 2. Reliability of gait evaluation data during fastest walking.

	Mean ± SD		ICC (95% Cl)	SEM	MDC95
	First session	Second session
Stride length (m)	1.84 ± 0.19	1.87 ± 0.20	0.92 (0.76–0.97)	0.05	0.14
Stride speed (m/sec)	2.30 ± 0.57	2.33 ± 0.61	0.90 (0.70–0.97)	0.06	0.16
Stride duration (sec)	0.83 ± 0.09	0.82 ± 0.12	0.85 (0.59–0.95)	0.04	0.11
Stance phase duration (sec)	0.50 ± 0.06	0.48 ± 0.10	0.90 (0.71–0.97)	0.03	0.09
Swing phase duration (sec)	0.33 ± 0.03	0.34 ± 0.03	0.82 (0.52–0.94)	0.01	0.04

n=13. SD: standard deviation; ICC: intraclass correlation coefficients; 95% CI: 95% confidence interval; SEM: standard error of measurement; MDC₉₅: minimal detectable change 95.

Neither additive nor proportional errors were present in the gait evaluation data. Bland–Altman plots of stride, which are key parameter steps for comfortable and fastest walking, are also shown (Figs. 2, 3).

Fig. 2.

Bland–Altman plot of test-retest reliability for stride length during comfortable walking.

n=13. Solid line: Average difference between the first and second measurements, Dotted line: Limit of agreement.

Fig. 3.

Bland–Altman plot of test-retest reliability for stride length during fastest walking.

n=13. Solid line: Average difference between the first and second measurements, Dotted line: Limit of agreement.

DISCUSSION

In this study, we examined the intra-rater reliability and systematic errors in gait evaluation data obtained from shoes equipped with built-in motion sensors. The gait evaluation data included stride length, stride speed, stride duration, stance phase duration, and swing phase duration. In the present study, ICC (1,1) was calculated between the initial measurement and the re-measured data after one week, and intra-rater reliability was examined. Subsequently, we examined the presence of systematic and measurement errors using MDC₉₅ by analyzing Bland–Altman plots using the mean values of the initial and re-measured values and the difference between the measurements (initial measurement value, re-measured value)¹⁷⁾. This is the first report evaluating the intra-rater reliability and the presence or absence of systematic errors for the specific shoes with built-in gait sensors used in this study. We believe that the results of this study will be significant for use in clinical settings. In the general ICC criteria, 0.41–0.60, 0.61–0.80, and ≥0.81 is considered “moderate”, “substantial”, and “almost perfect”, respectively¹⁸⁾. In studies in which motion sensors were attached to the trunk, stride length demonstrated high reliability, with ICC values such as 0.81¹⁹⁾ and 0.82²⁰⁾, and ranging from 0.75 to 0.90²¹⁾. Stride length obtained from the shoes with built-in motion sensors were ICC 0.90 and ICC 0.92 for comfortable and fastest walking, respectively. Therefore, the use of shoes with built-in motion sensors may be more reliable for the evaluation than motion sensors worn on the torso. Furthermore, in other studies using shoes with built-in motion sensors, the ICCs for stride length were reported as 0.90 for comfortable walking and 0.90 for fastest walking⁹⁾. In comparison, the ICCs obtained in the present study demonstrate equivalent or slightly higher reliability, indicating that the shoe-integrated sensors used in this study perform on par with other established systems. Other ICCs for stride speed, stride duration, stance phase duration, and swing phase duration also ranged from 0.83 to 0.90, indicating that shoes with built-in motion sensors can reliably measure these parameters. Shoes with built-in motion sensors have space on the sole where the motion sensor can be installed. Hence, no difference exists in the installation method among the measurements and the motion sensor can be measured at a fixed position leads to a high reliability of the measurements. The Bland–Altman plot analysis showed that none of the gait evaluation data had additive or proportional errors, and no systematic errors existed. As systematic errors were ruled out, the only possible error that could reduce the reliability of the measurement is the measurement error. In this study, the measurement errors were examined using the MDC₉₅. Changes in measurements within MDC₉₅ are due to measurement error, and changes above MDC₉₅ are judged to be “true changes”¹⁷⁾. Therefore, the MDC₉₅ value is important for determining the effectiveness of rehabilitation interventions before and after rehabilitation interventions for gait disorders. In this study, we presented a value that can be used as an approximate guide for measurement errors, and we believe that this value can be utilized in clinical situations. For example, stride in the elderly is reportedly not only an indicator of gait function but also a predictor of physical dysfunction, falls, need for nursing care, institutionalization, and death, suggesting the importance of its evaluation²²⁾. In visual gait assessment, quantifying stride is difficult. However, they can be easily measured using shoes with built-in motion sensors. Furthermore, the use of the MDC₉₅ value can increase the reliability of the change in stride of the participant when conducting a longitudinal study on inpatient rehabilitation.

One of the limitations of the present results is that they were obtained indoors; therefore, they could not be verified on rough terrains. Additionally, the number of participants was small. In the future, conducting a large-scale clinical study in older adults and individuals with various diseases is necessary. Another limitation is that only intra-rater reliability was assessed, as all measurements were conducted by a single examiner. In clinical practice, multiple examiners may be involved in gait assessments; therefore, evaluating inter-rater reliability is essential. Future research should include assessments by multiple raters to determine measurement consistency across examiners and improve the generalizability of the results. Furthermore, shoes with built-in motion sensors may have different shapes and hardness than those worn in daily life. Therefore, utilizing the results of this study is necessary because they are gait evaluation parameters obtained from shoes with built-in motion sensors. Despite some limitations, the results of the present study indicated that the intra-rater reliability was maintained, and no systematic errors existed. Moreover, if the gait evaluation data obtained from the motion sensor shoes before and after the rehabilitation intervention showed a change of MDC₉₅ or higher, it could be judged as a change that was greater than the measurement error. These shoes are expected to be useful for the continuous evaluation of walking during inpatient rehabilitation.

Although sensor-embedded gait-assessment devices have been increasingly utilized in clinical research involving populations with stroke and orthopedic disorders, their routine use in everyday clinical practice remains limited. Several factors may contribute to this gap, including cost, device complexity, the need for technical training, a lack of validation data under real-world conditions, and the time required for preparation. By evaluating the reliability of a fully automated, wearable gait sensor under level-ground walking conditions, this study aims to help bridge the gap between research settings and clinical applicability. Further efforts to simplify device operation and integrate sensor-based data into clinical decision-making processes may facilitate wider clinical adoption.

Funding

This study was supported by a Grant-in-Aid for Research from Sendai Seiyo Gakuin College (Gakucho0610) to Yoshihiro Aramaki.

Conflicts of interests

The author declare that there is no conflict of interest regarding the publication of this article.