Classification of chronic ankle instability using machine learning technique based on ankle kinematics during heel rise in delivery workers

Abstract

Objective

Ankle injuries in delivery workers (DWs) are often caused by trips, and high recurrence rates of ankle sprains are related to chronic ankle instability (CAI). Heel rise requires joint angles and moments similar to those of the terminal stance phase of walking that the foot supinates. Thus, our study aimed to develop, determine, and compare the predictive performance of statistical machine learning models to classify DWs with and without CAI using ankle kinematics during heel rise.

Methods

In total, 203 DWs were screened for eligibility. Seven predictors were included in our study (age, work duration, body mass index, calcaneal stance position angle [CSPA] in the initial and terminal positions during heel rise, calcaneal movement during heel rise [CM_HR], and plantar flexion angle during heel rise). Six machine learning algorithms, including logistic regression, decision tree, AdaBoost, Extreme Gradient boosting machines, random forest, and support vector machine, were trained.

Results

The random forest model (area under the curve [AUC], 0.967 [excellent]; F1, 0.889; accuracy, 0.925) confirmed the best predictive performance in the test datasets among the six machine learning models. For Shapley Additive Explanations, old age, low CMHR, high CSPA in the initial position, high PFA, long work duration, low CSPA in the terminal position, and high body mass index were the most important predictors of CAI in the random forest model.

Conclusion

Ankle kinematics during heel rise can be considered in the classification of DWs with and without CAI.

Introduction

The pandemic has changed consumption to a nonface-to-face form with the development of e-commerce and online shopping. This has drastically advanced the logistics and retail industries, increasing the employment of delivery workers (DWs).^1–4 Recently, delivery systems have been combined with driving and walking. ⁵ Because DWs repeatedly get in and out of trucks, walk, and run with parcels, ⁶ ankle injuries are likely to occur in uncontrolled, unpredictable, and varied outdoor environments during delivery. ⁷ Ankle sprain is the second most common work-related musculoskeletal disorder, ⁸ and ankle injuries are most often caused by trips in the DWs. ⁹ In addition, ankle sprains have a high recurrence rate and are associated with the development of chronic ankle instability (CAI).^10,11

Ankle inversion sprains have traditionally resulted from a combination of inversion and plantar flexion. ¹² Touchdown plantarflexion increases the occurrence of an ankle inversion sprain. ¹³ It increases the moment arm along the subtalar joint axis and thus the joint torque, followed by a sudden explosive twisting motion. ¹⁴ Thus, the primary ligamentous restraint to an inversion moment in the plantarflexed position is provided by the anterior talofibular ligament. ¹⁵ During walking gait, kinematic analyses have revealed that individuals with CAI exhibit a more inverted ankle position at toe-off,^16,17 as well as an increased rear-foot inversion throughout the entire gait,^16,17 than do healthy controls. In addition, individuals with CAI also display increased ankle joint plantar flexion around toe-off and heel strike compared with that of healthy controls.^17–19

The heel-rise test is used to assess foot and ankle muscle function in individuals with a broad spectrum of conditions. ²⁰ It is generally applied to evaluate calf muscle endurance by counting the maximum number of repetitions^21,22 and is used to assess calf muscle strength in individuals with CAI. ²³ The inability to perform heel rise or abnormal kinematics during heel rise has been reported in individuals with posterior tibial tendon dysfunction, ²⁰ flatfoot deformity, ²⁴ or diabetes mellitus. ²⁵ Still, these have yet to be studied in individuals with CAI. Because heel rise requires joint angles and moments similar to those observed in the terminal stance phase of walking, the foot undergoes supination in preparation for toe-off.^26,27 There is reasonable doubt that individuals with CAI would show increased rearfoot inversion during heel rise. However, whether there is a difference in ankle kinematics between individuals with and without CAI during heel rise is unclear.

Ankle sprains generate an average of 20 lost working days, ⁸ and recurrences caused by CAI can result in more lost working days. Therefore, it is necessary to accurately classify DWs with and without CAI to prevent and manage recurrences of ankle sprains. Machine learning is being rapidly highlighted in the classification of models. The advantage of machine learning is that a suite of algorithms can model both linear and nonlinear relationships. ²⁸ Thus, machine learning can achieve superior accuracy than traditional statistical methods. Therefore, the primary purpose of the present study was to identify the difference in ankle kinematics during heel rise between DWs with and without CAI. The secondary purpose of the present study was to develop, determine, and compare the predictive performance (i.e., the area under the receiver operating characteristic curve) of statistical machine learning models to classify DWs with and without CAI using an ankle kinematics dataset.

Methods

Participants

A total of 86 DWs with CAI and 117 DWs without CAI were screened for eligibility. DWs between 20 and 60 years of age who had worked in delivery for more than six months participated in the present study. CAI classification required participants to have one or more signs associated with CAI, including (1) impaired physical activity, (2) lateral ankle pain, and (3) one or more episodes of “giving way” or feelings of ankle instability. In addition, individuals with CAI had to confirm recurrent ankle sprain instability by scoring at least four points on the Ankle Instability Instrument. ²⁹ Individuals with CAI were excluded if they (1) could not perform a heel rise, (2) had a diagnosis of ankle osteoarthritis, and (3) had a recent history (<12 months) of lower-extremity surgery involving ankle joint intraarticular fixation. To statistically analyze the contribution of CAI based on up to seven variables, a sample size of at least 70 participants was required, following the 1:10 (one variable per 10 events) rule of thumb. ²⁸ DW's data generated from musculoskeletal screening tests for preventing industrial accidents were used by visiting a musculoskeletal health care center in the delivery company from August 2021 to March 2022. Informed consent for the present study was waived by the Institutional review board before the queries and analyses (1041849-202301-BM-016-01) because it was an analysis based on data already obtained by the parcel delivery company.

Kinematic measurements using two-dimensional video analysis

Two smartphones (Galaxy S20; Samsung Inc., Seoul, Korea) with a video recording application (4 K, 3840 × 2160 pixels at 60 fps) were placed on two tripods 150 cm behind the step box and lateral side, and both were 60 cm in height. The video data were transferred to a software package (Kinovea^® version 0.8.15; Kinovea, Bordeaux, France). In Kinovea software, movement is detected and tracked through the use of automated tracking markers. The process begins when a tracking marker is positioned by the user. This action initiates the detection of the coordinate system, enabling the commencement of analytical procedures. Following the analysis, the resultant data from the motion analysis is then transferred to a spreadsheet for performing machine learning.

The participants lay in a prone position on a bed that was horizontally aligned to the floor, with their feet over the edge of the bed. The examiner drew a bisection line on the participants’ calcaneus, disregarding any fat in the area, based on two dots on the upper and lower parts of the calcaneus. ³⁰ Additionally, a round sticker was attached to the two dots when the participants stood barefoot on the ground. The participants were instructed to stand on a step box with their metatarsophalangeal joint and rearfoot over the edge of the step box. The initial position was defined as the position in which the lateral side of the participant's foot was parallel to the ground, and the participant's fibula was perpendicular to the ground. The terminal position was defined as the position of the maximum peak heel height during the heel rise. For subtalar joint kinematics, the angle between the bisection line of the calcaneus and the horizontal line to the ground was defined as the calcaneal stance position angle (CSPA) (Figure 1). ³⁰ The CSPA was analyzed at the initial and terminal positions during heel rise. In addition, calcaneal movement during heel rise (CM_HR) was defined as the difference between the CSPA in the terminal and initial positions [CSPA in the terminal position (CSPA_TP)–CSPA in the initial position (CSPA_IP)]. For calcaneal movement, calcaneal eversion and inversion were indicated by positive and negative values, respectively. For talocrural joint kinematics, the angle between the lines formed from the fibular head to the lateral malleolus and from the base of the head of the fifth metatarsal bone in the terminal position during heel rise was defined and calculated as plantar flexion angle (PFA) (Figure 1).

Figure 1.

Ankle kinematic measurements using two-dimensional video analysis during heel rise. (A) Measurement of the calcaneal stance position angle in terminal position rise and (B) Measurement of plantar flexion angle.

The DWs with CAI were assessed on their affected side. In the event of CAI on both sides, the side with the greater Ankle Instability Instrument score was used for data analysis. DWs without CAI were assessed on their dominant side, which was defined as the preferred side for kicking the ball.

Procedures

The participants were asked to complete a brief health history form and an Ankle Instability Instrument. Round stickers (15 mm in diameter) were placed on the feet (the upper and lower parts of the calcaneus, fibular head, lateral malleolus, and fifth metatarsal bone). All heel-rise trials were performed barefoot. Participants were instructed to perform unilateral heel raises by lifting the heel as high as possible at a comfortable pace over a 5- to 15-s interval. The participants stopped once they completed three repetitions to minimize discomfort and maximize peak heel height. The nontested leg was flexed to 90° at the knee, with the foot positioned behind. The participants were allowed to place two fingertips per hand at shoulder height against support for balance. They were instructed to raise their heel as high as possible and then lower it back to the initial position with each repetition.

Data source and collection

Patient characteristics included age, body mass index (BMI), and duration of delivery. All ankle kinematic data, including CSPA_IP, CSPA_TP, CM_HR, and PFA, were analyzed using an available software package (Kinovea^® version 0.8.15; Kinovea, Bordeaux, France), and the average of the values for the three trials of heel rise was calculated. If CSPA_IP and CSPA_TP were < 90° or > 90°, the calcaneus was inverted or everted, respectively. In addition, if the CM_HR had negative or positive values, the CM_HR was an inversion or eversion movement, respectively. The higher the PFA value, the higher the ankle plantar flexion.

Statistical analysis

Machine learning analysis was performed using Orange data mining software (Orange 3.3.0, Ljubljana, Slovenia) and Python (Version 3.6.15; Python Software Foundation).

Preprocessing and missing data handling

Seven numeric predictors (age, BMI, work duration, CSPA_IP, CSPA_TP, CM_HR, and PFA) were included in the present study. Exploratory data analysis was performed to detect missing data. Imputation for handling missing data was performed by eliminating instances with unknown values. The distribution of each variable was confirmed as a boxplot and scatterplot to remove outliers that contributed to the accuracy of the learning model.

Machine learning algorithm

From the complete data (n = 203), we split the data into a training set (80%, n = 163) for model development and a test set (20%, n = 40) for external validation to predict performance. Six machine learning algorithms, including logistic regression, decision tree, AdaBoost, Extreme Gradient boosting machines, random forest, and support vector machine, were trained via 10-fold cross-validation. Hyperparameter tuning was performed to optimize the model's predictive accuracy.

Model validation

The primary measure of model performance was the area under the curve (AUC) calculated for both the training and test datasets (target class: DWs with CAI). The secondary measures of model performance were classification accuracy, recall, precision, and the F1 (harmonic mean of recall and precision) score for both the training and test datasets (target class: DWs with CAI). The predictive model performance was classified as excellent (≥0.9), good (0.8–0.9), fair (0.7–0.8), or poor (<0.7) based on the AUC value. ³¹

For each predictive variable, the feature permutation importance was calculated using the training dataset to confirm the critical factors. This analysis involved computing the contribution of each feature to the model performance based on the AUC by measuring the increase in the prediction error of the model. In addition, a Shapley Additive Explanation summary plot was generated to detect the importance and direction of each predictive variable. Each predictive variable on the y-axis was sorted by relative importance, with the most important predictors on the top. For each feature (predictive variable), each point (red meaning higher values or presence of binary factors) on the x-axis represented the contribution of individual participants to the overall Shapley Additive Explanation value, with higher positive contributions represented farther to the right.

Results

DWs’ characteristics

Table 1 shows the averages and standard deviations of all variables. In total, 203 DWs’ data points were used in the machine learning analysis; 42.4% (n = 86) of the DWs had CAI, and 57.6% (n = 117) did not (Figure 2). There were no significant differences between DWs with and without CAI in terms of age, work duration, BMI, CSPA_IP, and CSPA_TP. The CM_HR in DWs with CAI was statistically lower than that in DWs without CAI. The PFA in DWs with CAI was statistically higher than that in DWs without CAI. All the DWs who participated in the present study were male. The averages and standard deviations of the Ankle Instability Instrument score were 6.1 ± 1.4 and 2.0 ± 1.3 in DWs with and without CAI, respectively. The proportions of DWs with CAI on the right, left, or both sides were 26.7% (n = 23), 44.2% (n = 38), or 29.1% (n = 25; the side with the greater Ankle Instability Instrument score: right = 8 and left = 17), respectively. The proportions of DWs without CAI having their dominant side as the right and left sides were 95.7% (n = 112) and 4.3% (n = 5), respectively.

Table 1.

Mean ± standard deviation of baseline characteristics in DWs with and without CAI.

	DW with CAIa			DW without CAI			p
Age	39.19	±	5.27	37.14	±	9.04	.061
Work duration	421.18	±	156.63	411.01	±	241.45	.733
BMIb	23.45	±	2.37	22.81	±	2.66	.074
CSPAIPc	87.47	±	3.89	87.00	±	3.64	.372
CSPATPd	83.19	±	4.79	84.35	±	3.62	.05
CMHRe	−4.25	±	2.89	−2.64	±	3.51	<.001
Plantar flexion angle	125.90	±	8.60	120.87	±	11.49	.001

DW: delivery worker.

^aCAI: chronic ankle instability.

^bBMI: body mass index.

^cCSPA_IP: calcaneal stance position angle in the initial position.

^dCSPA_TP: calcaneal stance position angle in the terminal position.

^eCM_HR: calcaneal movement during heel rise.

Figure 2.

Scatter plot and multiaxis linear projection for classification of DWs with and without CAI. (A) scatter plot and (B) multiaxis linear projection, blue dot = DW with CAI, red dot = DW without CAI, dot size is based on CM_HR (the smaller the dot size, the greater the calcaneal inversion movement).

DW: delivery worker; CAI: chronic ankle instability; CM_HR: calcaneal movement during heel rise; CSPA_IP: calcaneal stance position angle in the initial position; CSPA_TP: calcaneal stance position angle in the terminal position.

Predictive models of machine learning

The performance of the six machine learning models for predicting CAI during model training and testing is presented in Table 2.

Table 2.

Performance metrics of six machine learning algorithms in the training and test set.

Performance metrics of six machine learning algorithms in the training set
Model	AUC	Accuracy	F1	Precision	Recall
Random forest	0.894	0.785	0.762	0.767	0.757
Support vector machine	0.889	0.798	0.795	0.736	0.865
Gradient boosting	0.769	0.724	0.706	0.684	0.730
Decision tree	0.756	0.730	0.711	0.692	0.730
AdaBoost	0.708	0.706	0.692	0.659	0.730
Logistic regression	0.655	0.644	0.608	0.608	0.608

AUC: area under the curve.

The six machine learning models were ranked for their performance (based on AUC) in classifying DWs with and without CAI using the training dataset as follows: random forest (AUC, 0.894 [good]; F1, 0.762; accuracy, 0.785), support vector machine (AUC, 0.889 [good]; F1, 0.795; accuracy, 0.798), extreme gradient boosting (AUC, 0.769 [fair]; F1, 0.706; accuracy, 0.724), decision tree (AUC, 0.756 [fair]; F1, 0.711; accuracy, 0.730), AdaBoost (AUC, 0.708 [fair]; F1, 0.691; accuracy, 0.706), and logistic regression (AUC, 0.655 [poor]; F1, 0.608; accuracy, 0.644) (Table 2 and Figure 3). The six machine learning models were ranked for their performance (based on AUC) in classifying DWs with and without CAI using the test dataset as follows: random forest (AUC, 0.967 [excellent]; F1, 0.889; accuracy, 0.925) support vector machine (AUC, 0.949 [excellent]; F1, 0.774; accuracy, 0.825), extreme gradient boosting (AUC, 0.906 [excellent]; F1, 0.741; accuracy, 0.825), AdaBoost (AUC, 0.821 [good]; F1, 0.750; accuracy, 0.850), decision tree (AUC, 0.815 [good]; F1, 0.690; accuracy, 0.775), and logistic regression (AUC, 0.693 [poor]; F1, 0.471; accuracy, 0.550) (Table 2 and Figure 3).

Figure 3.

Receiver operating characteristic (ROC) curves of six machine learning algorithms. (A) In the training set and (B) in the test set.

For feature permutation importance, the most important predictors of CAI in the random forest model in the order of high impact were as follows: AUC, age, CSPA_IP, CM_HR, work duration, PFA, CSPA_TP, and BMI (Figure 4A). In addition, AUC, work duration, age, PFA, CM_HR, CSPA_IP, BMI, and CSPA_TP were the most important predictors of CAI in the support vector machine model in the order of high impact (Figure 4C). For Shapley Additive Explanation analysis, the most important predictors of CAI in the random forest model in the order of highly important predictors were as follows: old age, low CM_HR, high CSPA_IP, high PFA, long work duration, low CSPA_TP, and high BMI (Figure 4B). Moreover, the most important predictors of CAI in the support vector machine model in the order of highly important predictors were old age, low CM_HR, high PFA, long work duration, high BMI, high CSPA_IP, and low CSPA_TP (Figure 4D).

Figure 4.

(A) Feature permutation importance of random forest model in the training set; (B) Shapley Additive Explanation analyses of random forest model in the training set; (C) feature permutation importance of support vector machine model in the training set; (D): Shapley Additive Explanation analyses of support vector machine model in the training set.

CM_HR: calcaneal movement during heel rise; CSPA_IP: calcaneal stance position angle in the initial position; CSPA_TP: calcaneal stance position angle in the terminal position; BMI: body mass index; AUC: area under the curve.

Discussion

Previous studies on walking and jogging have confirmed that hindfoot abnormalities are common in individuals with CAI.^16,17,32,33 The present study confirms that similar abnormal hindfoot kinematics occur in DWs with CAI during heel rise. Calcaneous inversion determined using the CM_HR and PFA in DWs with CAI was significantly greater than that in DWs without CAI during heel rise. This study validated six machine-learning models using seven predictive variables (age, work duration, BMI, CSPA_IP, CSPA_TP, CM_HR, and PFA). The present study identified the best-performing machine learning model for classifying DWs with and without CAI in the test dataset, with an AUC = 0.967 (excellent), F1 = 0.889, and accuracy = 0.925. Age, CSPA_IP, CM_HR, work duration, and PFA were the top five predictors of CAI in the random forest model in terms of feature permutation importance. Old age, low CM_HR, high CSPA_IP, high PFA, and long work duration were the top five predictors of CAI in the random forest model for the Shapley Additive Explanation analysis. Thus, CM_HR could be considered a discriminative factor in the CAI predictive model, and holding the calcaneus in a neutral position during heel rise may be suggested as a guideline for physical therapy or therapeutic exercise to manage or prevent CAI.

Increased ankle joint inversion (+6° to +7°) ³³ and increased rearfoot inversion (+2.07°) ¹⁶ during walking were determined in the walking cycle for individuals with CAI compared to those of the control groups. Similarly, in the present study, the CM_HR in DWs with CAI (+1.60°) during heel rise was significantly greater than that in DWs without CAI. However, there was no significant difference between DWs with and without CAI in CSPA_IP and CSPA_TP during heel rise. Adequate muscle tension is necessary to support the ankle. ³⁴ Repeated ankle inversion sprains affect neuromuscular structures, making it challenging to detect forces around the ankle and avoid injury during functional ankle instability. ³⁴ It is also possible that damage to neuromuscular structures in the ankle joint due to repeated ankle inversion sprains causes force-sense deficits that affect rearfoot position during heel rise. ³⁴ In our study, calcaneus inversion movement influenced the classification of CAI more than the inverted ankle position at the initial and terminal positions did during heel rise. Thus, a high calcaneus inversion determined by CM_HR may increase the possibility of CAI in DWs. However, it is necessary to determine whether a greater degree of CAI occurs in DWs with greater calcaneus inversion determined by CM_HR in future studies because our study design was cross-sectional.

A high CSPA_IP value during heel rise was inputted in the random forest model for Shapley Additive Explanation analysis. The higher the CSPA_IP value, the greater the calcaneus inversion and foot pronation in the initial position during the heel rise. Foot pronation accompanied by navicular drop and a low medial longitudinal arch is related to impaired ankle neuromuscular control and ankle injuries.^35,36 The peroneal muscles are responsible for controlling ankle inversion during a sudden change in ankle position. Foot pronation may be due to the shortening of the peroneal muscles and can diminish the biomechanical function of the peroneal muscles.^37,38 In addition, the electromyographic activity of the peroneal muscles is lower in individuals with pronated feet than in individuals with normal feet. ³⁹ Thus, the pronated foot is associated with impaired neuromuscular control. Individuals with pronated and supinated feet are at a higher risk of lateral ankle sprains than individuals with normal feet. ³⁶ Additionally, navicular drop as an anatomical and functional risk factor of lateral ankle sprain was inputted in the logistic regression model (OR = 1.767, 95% CI: 1.408–2.217). Although it is impossible to perfectly match our research with previous studies on the relationship between pronated foot and lateral ankle sprain, greater CSPA_IP may increase the possibility of CAI in DWs.

High PFA during heel rise was inputted into the random forest model for the Shapley Additive Explanation analysis. During running, a more plantarflexed ankle ⁴⁰ (+129%) ¹⁹ (+4.8) ⁴¹ was observed in the CAI group compared to that in the control group. Similarly, in the present study, the PFA in DWs with CAI (+5.02°) during heel rise was significantly greater than that in DWs without CAI. The greater PFA in individuals with CAI could indicate that sensorimotor deficits can contribute to ankle instability. ⁴² During walking, the functional ankle instability group (complaining of giving way without mechanical laxity) exhibited a greater degree of plantarflexion at maximum than did the mechanical ankle instability group (laxity of the lateral ankle ligaments). ⁴³ However, increased ligamentous laxity may be a sensorimotor deficit contributing to increased ankle dorsiflexion to avoid potentially unstable ankle plantar flexed positions. ⁴³ Because we did not measure pathological laxity, it was difficult to classify the characteristics of CAI (functional ankle instability and mechanical ankle instability) in DWs in our study. However, pathomechanical, sensory-perceptual, and motor-behavioral impairments are interrelated, ⁴⁴ and sensorimotor deficits have been suggested in ankle joint position sense of sagittal-plane motion, with individuals with CAI demonstrating more proprioceptive errors. ⁴⁵ Thus, a greater PFA during heel rise may increase the possibility of CAI in DWs.

Old age and long work duration were inputted into the random forest model for the Shapley Additive Explanation analysis. The DWs with CAI in our study had ages ranging from 22 to 57 years. Age has been reported as a predictor of ankle sprain recurrence,^46–48 and two studies confirmed a negative relationship between age and ankle sprain status; however, age was not an independent predictor of ankle sprain occurrence.^46,47 In addition, ankle sprain injury rates in sports are higher in younger participants than in older participants. ⁴⁹ Although a reduction in lower-limb strength and power is associated with older age, ⁵⁰ the amount of delivery work does not vary according to age. Thus, older age may increase the possibility of CAI in DWs. Occupational risk factors for ankle injuries in DWs include uneven surface walking, steps and stairways, lighting conditions, and unsafe working behaviors, such as reading addresses while walking, rushing, and taking hazardous shortcuts during delivery work. ⁷ Thus, a long work duration may increase the possibility of CAI in DWs.

Our study has some limitations. First, the study population was limited to DWs, making it difficult to generalize the findings to a wider population. Further studies with larger sample sizes are necessary to generalize our results. Second, the study design was cross-sectional; hence, we could not confirm the establishment of cause-and-effect relationships. Therefore, our interpretation of cause-and-effect relationships is speculative. Third, we did not analyze pathomechanical impairments such as pathologic ligamentous laxity and sensory-perceptual impairment, which could affect CAI. If feature-classified characteristics of pathomechanical impairments and sensory-perceptual impairments for CAI are added to our model, the model performance may be increased. Fourth, there is a possibility that the best-performing model was not one of the six models we selected because of the diversity of machine learning algorithms. Fifth, although there are statistical tests to compare different machine learning algorithms based on a single test dataset, a separate validation study is required for verification.

Conclusion

The present study demonstrates the effectiveness of machine learning techniques in classifying DWs with and without CAI based on ankle kinematics during heel rise. The random forest model proved to be the most accurate, highlighting its potential in occupational health for early detection and intervention. For Shapley Additive Explanation analysis, in random forest model (AUC, 0.967 [excellent]; F1, 0.889; accuracy, 0.925), old age, low CM_HR, high CSPA_IP, high PFA, long work duration, low CSPA_TP, and high BMI were the most important predictors of CAI. Thus, ankle kinematics during heel rise can be considered for classifying DWs with and without CAI. These findings emphasize the importance of integrating biomechanical and occupational factors in CAI assessment and management. In addition, maintaining the calcaneus in a neutral position during heel rise may be considered during therapeutic exercises to manage and prevent CAI since CM_HR was found to be a predictive factor for CAI using the machine learning model.