THE USE OF MICROSOFT KINECT ™ FOR ASSESSING READINESS OF RETURN TO SPORT AND INJURY RISK EXERCISES: A VALIDATION STUDY

ABSTRACT

Introduction

Assessing readiness of return to sport after procedures such as anterior cruciate ligament (ACL) reconstruction is a complex process, complicated by the pressures that athletes face in returning to sport as quickly as possible. Advances in motion analysis have been able to demonstrate movements that are risk factors for initial ACL injury and subsequent reinjury after reconstruction. An inexpensive, objective measure is needed to determine when athletes are ready to return to sport after ACL reconstruction.

Purpose

The aim of this study was to compare the use of a single camera, markerless motion capture technology to 3D motion capture during lower extremity movements that pose as risk factors for ACL injury.

Study Design

Cross Sectional Study

Methods

This study assessed the validity of the Microsoft Kinect™ against an established 3-dimensional motion analysis system in 20 healthy subjects. Knee kinematics were assessed during impact activity in the coronal and sagittal plane specifically evaluating peak knee valgus and peak knee flexion during single leg hop and jump from box exercises. Intraclass correlation coefficients and 95% limits of agreement (LoA) were determined for each kinematic variable.

Results

For the single leg hop, the mean absolute difference in the sagittal plane was 10.4 ° (95% LoA [-11.7 °, 26.8 °]), and in the frontal plane was 5.31 ° (95% LoA [-8 °, 13.9 °]). Similarly, for the jump from box landing on one leg, there was a difference of 7.96 ° (95% LoA [-17.7 °, 21.3 °]) and 4.69 ° (95% LoA [-6.3 °, 12.6 °]) respectively. For the jump from box, two-foot land, turn and pivot, the mean absolute difference between the systems was 7.39 ° (95% LoA [-17.8 °, 19.7 °]) in the sagittal and 4.22 ° (95% LoA [-5.9 °, 11.6 °]) in the frontal plane respectively. Intraclass correlation coefficients for each activity ranged from 0.553 to 0.759.

Conclusion

The results from the Microsoft Kinect™ were found to be in poor agreement with those from a standard motion capture system. Measuring complex lower extremity movements with the Microsoft Kinect™ does not provide adequate enough information to use as an assessment tool for injury risk and return to sport timing.

Level of Evidence

Level III

INTRODUCTION

Every year, high school and collegiate athletes suffer season ending anterior cruciate ligament (ACL) injuries. The current estimated incidence of ACL injury is approximately 200,000 annually with an increased risk seen among athletes competing in high risk sports such as soccer, basketball, football, and skiing.¹ Assessing readiness of return to sport after ACL reconstruction is a complex but well-studied process that can be complicated by the pressures that competitive athletes face in returning to sport as quickly as possible. Current concepts in rehabilitation for return to sport have used objective and subjective measurements and scoring systems such as global rating scores, Knee Outcome Survey-Activities of Daily Living (KOS-ADL) scales, quadriceps index, Leg Symmetry Index, and varied types of hop testing. Each of these indices provide a measure of presence or development of neuromuscular abilities that are thought to be relevant for safe return to sport.²,³ Despite the number of assessments that have been described, there remain few objective and reliable measures that are cost effective, easy to administer, and can help healthcare providers determine when athletes might be ready to safely return to sport after ACL reconstruction.

Over the last decade, findings from motion analysis studies have been able to correlate functional movements with both initial ACL injury risk as well as reinjury after ACL reconstruction.⁴ Postoperative patients with decreased neuromuscular control are threatened by movement biomechanics that are influenced by abnormal trunk and lower extremity movements. These high-risk movements such as single limb landing activity and anterior and lateral jumping progression have been shown to accentuate post reconstruction limb deficits.⁴ Injury and reinjury risk can potentially be decreased by deep knee flexion exercises, balance and proprioception training, and improved trunk and hip control avoiding quadriceps-dominant landing techniques.⁴

These abnormal movements have been studied using multiple camera systems. Though widely used in motion analysis laboratories, these systems tend to be expensive, costing thousands of dollars depending of the system. This makes them impractical and unavailable in most clinical settings. Markerless motion capture systems such as the Microsoft Kinect™ (Microsoft Corporation, Redmond, USA), have recently become readily available, and are less expensive and easier to set up and use than traditional motion capture systems that require subjects wear retroreflective markers to track kinematic segments. The Brekel software and Kinect™ hardware can be purchased together for just under $200. These systems have been recognized as an appropriate tool for motion analysis in gait, trunk control, and standing balance.⁵,⁶

The aim of this study was to compare the use of a single camera, markerless motion capture technology to 3D motion capture during lower extremity movements that pose as risk factors for ACL injury. It was hypothesized that the markerless motion capture would provide similar accuracy to standard methods for assessing important lower extremity motions related to ACL injury and rehabilitation.

METHODS

Twenty healthy volunteers were recruited to participate in the study. There were six females and 14 males, age 30.8 years (SD 5.9), height 1.75m (SD 0.1), weight 75kg (SD 12.2) all of whom were free of any physical limitation that could prevent them from jumping and landing on one or both legs. Each subject signed an informed consent form approved by Medical Ethics Committee of the University of Washington Internal Review Board in Seattle, Washington.

Coronal and sagittal plane knee kinematics were assessed during impact landing activities. These activities were: 1) dropping from a box that was 30 cm in height and landing on a single leg, tested bilaterally; 2) dropping from the same box, landing on both feet then pivoting 90 ° using the test leg to push off to the ipsilateral side, tested on both legs; 3) and single leg hops tested on both legs. Each of these activities and measurements were chosen based on previously validated motions that would demonstrate deficiencies in ACL deficient and at-risk populations.⁷

Each motion was concurrently recorded using an 8-camera motion capture system (Vicon Motion Systems Ltd, Oxford, UK) using Nexus software (Vicon Motion Systems Ltd, Oxford, UK) and concurrently using Microsoft Kinect ™ (Microsoft Corporation, Redmond, USA) using Brekel Kinect™ software (Brekel 3D, Amsterdam, The Netherlands). Each subject wore form fitting athletic attire and was affixed systematically with reflective markers on the lower extremities and trunk. The Kinect™ sensor was placed directly in front of the subject approximately 2.5 meters away as they performed the activity. (Figure 1) Each subject performed five repetitions of the three previously described activities. Each of the activities were performed on both legs and recorded as separate data for analysis, not comparing the right leg against the left. Prior to performing each exercise, the subject was asked to perform a series of three stomps with their right leg. This allowed the two measurement datasets to be temporally aligned.

Figure 1.

Testing setup with subject wearing markers and orientation of both Kinect system and Vicon cameras.

Data processing and analysis

Marker data were processed using Visual3D (C-Motion Inc., Germantown, MD). All kinematics were evaluated using the laboratory coordinate system. Peak knee valgus and peak knee flexion during single leg hop and jump from box exercises was evaluated. Two-way, absolute agreement, single measures intra-class correlation coefficients (ICC) were calculated (Table 1) and Bland Altman plots were produced including 95% limits of agreement (LoA).

Table 1.

ICC's and 95% CI's for kinematic measures.

	ICC	95% CI lower bound	95% CI upper bound
BSD Knee Flexion Peak	0.654	0.435	0.8
BSD Knee Valgus Peak	0.589	0.231	0.785
BSDP Knee Flexion Peak	0.759	0.587	0.866
BSDP Knee Valgus Peak	0.717	0.393	0.862
OLH Knee Flexion Peak	0.594	0.166	0.801
OLH Knee Valgus Peak	0.553	0.253	0.747

BSD = Single Leg Drop from Box; BSDP = Box Drop Single Leg Pivot;

OLH = One Leg Hop; ICC = Intra-class Correlation Coefficient;

CI = Confidence Interval

RESULTS

Intra-class correlation coefficients are presented for each exercise and variable in Table 1. These ranged from 0.553 to 0.759, representing good agreement. Overall, the Kinect™ was found to systematically underestimate the peak values for knee kinematics.

The single leg hop saw the largest differences between the two systems, with the mean absolute difference in the sagittal plane being 10.4 ° (95% LoA [-11.7 °, 26.8 °]), and in the frontal plane 5.31 ° (95% LoA [-8 °, 13.9 °]). The Bland Altman plots for these results are presented in Figure 2.

Figure 2.

Bland Altman plot of knee flexion (left) and valgus (right) for one leg hop exercise. The x-axis represents the mean value of the measurement and the y-axis the difference between systems. The bold horizontal line is the mean difference between systems while the dashed horizontal lines are the 95% limits of agreement.

For the jump from box landing on one leg, the mean absolute differences between systems were 7.96 ° (95% LoA [-17.7 °, 21.3 °]) in the sagittal plane and 4.69 ° (95% LoA [-6.3 °, 12.6 °]) in the frontal plane. Bland Altman plots are presented in Figure 3.

Figure 3.

Bland Altman plot of knee flexion (left) and valgus (right) for jump from box, landing on one leg exercise. The x-axis represents the mean value of the measurement, and the y-axis the difference between systems. The bold horizontal line is the mean difference between systems while the dashed horizontal lines are the 95% limits of agreement.

The jump from box with a turn and pivot demonstrated similar differences and variability between systems, with the mean absolute difference between the systems being 7.39 ° (95% LoA [-17.8 °, 19.7 °]) in the sagittal and 4.22 ° (95% LoA [-5.9 °, 11.6 °]) in the frontal plane. Bland Altman plots are presented in Figure 4.

Figure 4.

Bland Altman plot of knee flexion (left) and valgus (right) for jump from box, landing on both legs then pivoting. The x-axis represents the mean value of the measurement, and the y-axis the difference between systems. The bold horizontal line is the mean difference between systems while the dashed horizontal lines are the 95% limits of agreement.

DISCUSSION

It is estimated that up to 350,000 ACL reconstruction surgeries are performed every year.⁸ Even with appropriate reconstructive surgery, recovery, and rehabilitation the risk of re-injury remain as high as 37%.² Of the athletes that do return to sport, as few as 50% will return to their pre-injury level of play up to three years later.⁹ Numerous studies have attempted to establish criteria for return to sports.¹⁰ Hop tests, jump tests, and lower extremity functional tests each attempt to identify residual deficits in the symmetry of the patient's quadriceps/hamstring balance and neuromuscular control.¹¹,¹² Still, there are limited objective measurements that can easily be obtained in the clinical setting to determine an athlete's readiness to return to the playing field. This pilot study, using a healthy surrogate population, has attempted to use simple motion analysis to evaluate the biomechanics about the knee during high impact activity. The goal of the study was to evaluate an inexpensive and easily transportable system that can compare to the “gold standard” Vicon motion analysis in assessing knee kinematics, with the hope that it could potentially translate to an objective strength and proprioceptive assessment when comparing to the patient's uninjured limb or even pre-injury assessment.

Previous literature has predicted a second ACL tear both in previously reconstructed knees as well as the contralateral knee based on biomechanical screening.¹³ Neuromuscular control and proprioceptive training have significantly decreased the incidence of injury and reinjury especially in female athletes.^14-16 A valgus moment imposed on an at-risk knee is commonly associated with ACL rupture. One of the goals of this study was to determine whether the Kinect™ system was able to measure knee valgus to a similar degree of accuracy as the 3-dimensional gold standard for motion analysis, the Vicon system. The Kinect™ was also used to examine sagittal plane positioning of the knee. Young athletes that demonstrate quadriceps strength asymmetry one year after returning to sport status post ACL reconstruction continue to demonstrate decreased knee-related function and lower proportions of functional recovery.¹⁷ The Kinect™ was used to assess peak knee flexion with impact activities as a correlate of quadriceps and hamstring symmetry. Previous research has determined this asymmetry to place athletes at high risk of primary rupture and graft re-rupture after primary reconstruction.^18-20

Regarding the use of Kinect™ and low impact activity, it seems that there is evidence to validate its use clinically. This inexpensive tool has been used reliably in gait and postural control analysis in children, adults, and patients with Parkinson's disease.⁵,²¹,²² The vertical drop jump has been previous studied using the Kinect™ V2 software,²³ the authors finding 95% reliability based upon their intra-class correlation coefficients of 0.84 and 0.95 with knee valgus in a validity study. Within the data presented for both sagittal and coronal planes, the Kinect™ underestimated peak values as compared to the gold standard, Vicon. In this study, the accuracy of the Microsoft Kinect™ when peak coronal and sagittal knee movement was compared to the Vicon motion analysis system proved to be poor. This data suggests that the Kinect™ cannot be validated for use in the clinical setting for high impact activities, as other authors have suggested.⁵,^21-24 Use of the Kinect™ for high impact activity and ACL testing throughout the literature shows significant variability, making its use unreliable. Specifically, for knee movement the Kinect™ has reliability coefficients ranging from 0.44 to 0.88.²⁴ Though this study was not designed to compare reproducibility or reliability of a specific motion, the presented data cannot reproduce the previously cited reliability coefficients. Intra-class correlation coefficients were calculated at peak knee flexion, but none were above ICC > 0.76. In both valgus moment and flexion measurements, the subjects showed up to 20 degrees of difference between the Vicon and Kinect™. Returning an athlete to high level sport after an ACL injury based on an assessment tool with this level of inaccuracy would warrant extreme caution.

Regarding limitations, this was a pilot study with a relatively small cohort. Other than sample size, this study's limitations include using a healthy male dominant population when ACL injuries are found to occur in females at a disproportionately higher rate than males. In order to justify the use of the Kinect™ as an assessment tool to guide return to sport, an assumption has to be made regarding the degree of difference that would be acceptable between it and the gold standard. Though this study shows statistically significant difference in the degree of measurement, it does not prove any clinical significance. People at-risk of ACL injury may or may not have a greater magnitude of angular deformity than healthy control subjects but the findings suggest that the Kinect™ is not sensitive enough to measure differences that would be acceptable for high level athletes. Lastly, as the Kinect™ program has been discontinued by Microsoft, there is growing concern for its availability for utilization as a clinical tool, though hopefully this data does inform the next generation of markerless motion capture systems.

CONCLUSION

While the Kinect™ has validity for use in motion analysis in low impact activity, it demonstrated poor validity when compared to 3-D motion analysis during lower extremity impact activity and complex movements. The use of Microsoft Kinect™ as a method to assess risk of injury and readiness of return to sport is not recommended in high level athletes. The Vicon multicamera system remains the gold standard of complex lower extremity motion analysis.