Reconstruction and EMG-Informed Control, Simulation and Analysis of Human Movement for Athletics: Performance Improvement and Injury Prevention

Abstract

In this paper we present methods to track and characterize human dynamic skills using motion capture and electromographic sensing. These methods are based on task-space control to obtain the joint kinematics and extract the key physiological parameters and on computed muscle control to solve the muscle force distribution problem. We also present a dynamic control and analysis framework that integrates these metrics for the purpose of reconstructing and analyzing sports motions in real-time.

I. Introduction

Defining, predicting, and guiding the optimal motion in athletic skills is both a promising and a challenging problem. Addressing this problem requires a full understanding of human neuromuscular biomechanics, accurate modeling and analysis of motion patterns, and precise full body sensing and reconstruction in real-time. In athletics, coaches generally assess performance through trial and error techniques. Current state of the art methods involve lengthy data acquisition processes, followed by human expert analysis, interpretation, movement recommendations and reassessment. This procedure can take days or weeks following the initial assessments. An automated coach that would provide subjects with real-time visual or haptic feedback to correct athletic motions in optimal ways would be an unprecedented tool. Some potential benefits of such a system include real-time interpretation of movements, comparison to ideal motions, and prediction of movement patterns that may result in short or long-term injury. In particular the following questions should be addressed as part of an automatic training system: 1. What characterizes and differentiates the performance of elite athletes compared to novice athletes? 2. How can this characterization process be modeled and generalized to enable automated coaching of athletes? 3. How can additional performance constraints be integrated to prevent long and short term injuries? 4. In what ways can this information be visually or haptically conveyed for intuitive, realtime guidance of movements?

II. motion reconstruction process

A. Marker Tracking

For the purpose of using motion capture systems to investigate human movements, a task/posture decomposition that uses an operational space method [1] constitutes a natural decomposition for dealing with marker data, thus avoiding the computationally intensive inverse kinematics. This decomposition allows us to represent the dynamics of a simulated human subject in a relevant task space that is complemented by a posture space (see Appendix).

For an arbitrary number of tasks, the torque decomposition (see Appendix) can be generalized to,

$$\mathrm{\Gamma }=J_{t_1}^T{F}_{t_1}+J_{t_2|t_1}^T{F}_{t_2|t_1}+\dots \dots +J_{t_n|t_{n-1}|\dots |t_1}^T{F}_{t_n|t_{n-1}|\dots |t_1},$$ (1)

where Γ is the total control torque, J_t and F_t are the Jacobian and the force associated with the task, respectively.

In our direct marker control application, we define the task space as the space of Cartesian coordinates for the motion capture markers. However, marker trajectories obtained through motion capture are not independent. To deal with this motion coupling the markers are grouped into independent subsets, m₁, …,m_n, where m_i denotes the task for a particular marker subset. At the end of the recursive process of building a marker space defined by a hierarchy of decoupled marker tasks, we obtain the overall control torque defined in marker space, specifically

$$\mathrm{\Gamma }=J_{m_1}^T{F}_{m_1}+J_{m_2|m_1}^T{F}_{m_2|m_1}+\dots \dots +J_{m_n|m_{n-1}|\dots |m_1}^T{F}_{m_n|m_{n-1}|\dots |m_1},$$ (2)

where J and F are the Jacobian and the force associated with marker space, respectively.

For the purpose of testing our algorithm, a series of slow movements performed by a tai chi master were captured using an 8-camera motion capture system. The motion was then reconstructed in the SAI software environment [2] by tracking subsets of decoupled marker trajectories in real-time. Joint angles over the entire trajectory were obtained as a natural consequence of the direct marker control approach. Additionally, the goal and tracked positions of the controlled markers were recorded during real-time simulation. The results showed that the marker tracking successfully maps the human model to the desired motion trajectories [3].

An analysis on the bounds of the joint space errors can be performed using the Jacobian associated with the marker space, J_⊗. Joint angles obtained through prioritized control in marker space deviate from the actual values but are bounded by:

$$|\mathrm{\Delta }q|\le |{\overline{J}}_{\otimes }||\mathrm{\Delta }{x}_{\otimes }|,$$ (3)

where the dynamically consistent inverse of the Jacobian, J̅_⊗, maps the joint angle error, Δq, to the marker position error, Δx.

This allows us to tune the prioritized marker controller to accommodate the desired accuracy, for given configurations. Figure 1 shows the margin of marker position errors and the margin of joint angle errors respectively. Maximum and minimum joint angle error magnitudes vary stably over the trajectory, suggesting well bounded errors on the joint angles.

Fig.1.

Margin of marker position errors and margin of joint angle errors over the trajectory. Joint angle error magnitudes show a stable variation over the trajectory, thus ensuring well bounded errors on the joint angles [3].

B. EMG Tracking

The joint angles obtained from direct marker tracking and the forces sensed by force plates will estimate the joint torques through the equations of motion. The muscle moment arms coupled with these torques will ultimately give us the forces required to generate the motion. But the muscle forces are not unique (this comes from the fact that the inverse of muscle Jacobian has multiple solutions as joints are generally over-actuated). Computed muscle control (CMC) is an effective yet computationally intensive solver to overcome this underdetermined problem. It deals with the muscle redundancy by estimating muscle activations and forces from kinematic and ground reaction force data [4]. On the other hand, data obtained from electromographic (EMG) recordings might provide a better prediction by tracking the muscle activation characteristics for superficial muscles without constraining them. The use of EMG data also may make real-time muscle force estimation more feasible.

For this purpose, we recently introduced an extended version of CMC that accounts for the EMG recordings. Conventional CMC estimates muscle forces using an optimization criteria that minimizes muscle stress. In this solution kinematics are treated as constraints. We modified the CMC minimization function P by adding a term related to the difference between the muscle excitation x and the filtered EMG data x_emg for some muscles, as follows:

$$P=\sum _{i=1}^{n_m}{\sigma }_i^2+\sum _{k=1}^{n_{\mathit{\text{emg}}}}{w}_k*f(x_k-x_{{\mathit{\text{emg}}}_k})$$ (4)

In Equation (4) n_m is the number of muscles used in the simulation, σ is the muscle stress, n_e is the number of muscles with EMG data, w is a weight and f can be any function. The first term in P is used for conventional CMC. The second term is added to track EMG data for a subset of muscles. Kinematics are still treated as constraints. When no EMG data are supplied, this reverts to conventional CMC.

In order to test the performance of the modified algorithm, the forces of the tibialis anterior and medial gastrocnemius antagonistic ankle muscle pair were estimated using conventional and EMG-informed computed muscle control for one male subject walking at 1.75m/s. The results showed that muscle forces more closely matched EMG data with the EMG-informed algorithm in all cases compared to conventional CMC [5]. Additionally, because this was an agonist/antagonist pair, when only a single muscle was tracked, predictions for both muscles improved.

III. performance metrics for athletics

A. Muscle Effort Criteria

We developed a muscular effort characterization based on a task-based muscle effort measure, specifically,

$$E=F^T\lfloor J{\left(L^T{N}_c^{2}L\right)}^{-1}{J}^T\rfloor F,$$ (5)

where N_c is the r × r muscle capacity matrix and L is the r × n muscle Jacobian matrix (moment arms) for a system of n joints and r muscles.

In Equation (5), the term in bracket connects the muscle physiology to the operational space force, F, through the Jacobian, J. We performed the characterization based on the muscular effort minimization function by analyzing an American football throwing motion. Results of this analysis showed that hand trajectory follows the direction associated with the minimum effort (see Fig 4.) [6].

Fig. 4.

Task-based analysis of muscular effort. The muscle effort variation for selected 5 configurations during a throwing motion represented by ellipsoid expansion model. The throwing hand trajectory roughly follows the direction of the ellipsoid associated with the minimum effort [6].

B. Kinematic Criteria

For this system of n equations and r muscles, Γ is the r × r vector of muscle induced joint torques and A is the n × n mass matrix. Using the operational space acceleration & muscle force relationship

$$\ddot{x}=J(q)A{(q)}^{-1}(\mathrm{\Gamma }-g(q)),$$ (6)

where g(q) is the r × 1 gravity torque vector.

The feasible range of accelerations can be determined using (6) given the bounds on the muscle induced torque capacities by,

$$0<\mathrm{\Gamma }<L^T{m}_{\text{max}},$$ (7)

where m_max designates muscle force generating capacities.

We investigated the relationship between the muscle forces and the resulting accelerations while throwing a football. We characterized hand accelerations at a series of configurations along the motion. The results showed that hand movement follows a trajectory with the direction of largest available acceleration during the performance of learned skills (see Fig 5.) [6].

Fig. 5.

The feasible set of accelerations for selected 5 configurations of a throwing motion. The throwing hand follows the direction defined by the maximum acceleration at each configuration [6].

IV. control and analysis framework

The marker tracking and EMG tracking processes described in the previous sections are integrated into our controller for real-time analysis. This platform involves motion and EMG-based tracking systems, as well as the musculoskeletal, kinematic, and dynamic models of the subjects. The overall control and analysis framework is illustrated in Figure 6. The process starts with simultaneous motion and muscle activation sensing recorded using 8-camera Vicon system (OMG plc, Oxford UK). The scaled human model [7], [8] together with the filtered EMG and marker data are used to reconstruct the motion and to estimate the corresponding muscle forces. In parallel, new control metrics are developed to accurately model and evaluate human motion performance patterns. Here, the robotics tools are used to develop new criteria that correlate to the observed motion characteristics of the athletes, which will involve physiological, kinematic, and dynamic performance parameters. Using this information, we are able to analyze in great detail the biomechanical variables of the real-time motions, and decide which ones play important roles in optimizing the performance. In the analysis, the aim is to exploit the information given by the neural-musculoskeletal models mapped into the motion of the human subject. This analysis tool provides a variety of parameters that we use to extract meaningful information that correlate with the performance.

Fig. 6.

Biocomputational platform showing the control and analysis process within SAI. Sensing data and corresponding human model are used as input to the controller. The information about the kinematics and dynamics is extracted in real-time. New control metrics are developed to model motion strategies in order to find the subject-specific “optimal motion”. These metrics are tested through motion capture experiments and key performance parameters are extracted during the analysis.

V. conclusions and future work

We have presented a task-based control and simulation architecture for reconstructing and analyzing sports motions in real-time. We also introduced human performance metrics to conduct motion analysis based on musculoskeletal parameters.

We have developed and implemented a new algorithm to reconstruct human motion sequences from motion capture data based on direct control of marker trajectories. The new reconstruction approach is able to directly project marker Cartesian trajectories into human musculoskeletal models, eliminating the need of resorting to time consuming inverse kinematic projections. We demonstrate the effectiveness of this method using a captured sequence of tai chi expert motions. To characterize and model human motion performance, we have recently developed a new method based on acceleration and effort analysis of athletic motions. We have applied these models to captured sequences of a football athlete, demonstrating that the subject minimized the muscular effort to optimize throwing performance. To extract muscle activation patterns during athletic performance, we have developed and implemented a system that integrates computed muscle control (CMC) with electromographic (EMG) sensing devices. We have validated this system on tibialis anterior and medial gastrocnemius antagonistic ankle muscles during walking. The results indicated that EMG-informed CMC can be used to improve dynamic simulations to better match experimental data.

For future work, we will investigate and model neuromuscular and biomechanical patterns involved in sport motions and use them to provide real-time visual and haptic [9] feedback to athletes for achieving optimal performance. This feedback could be visual or tactile feedback and could be used to accelerate learning a new skill or reduce the risk of injury. Achieving this goal will require the integration of muscle-level fatigue and failure criteria models in our framework and muscle capacity models for goal-based task motions.

Fig. 2.

Conventional and modified CMC simulation results along with filtered EMG data for Medial Gastrocnemius [5].

Fig. 3.

Conventional and modified CMC simulation results along with filtered EMG data for Tibialis Anterior [5].

Appendix

$$\mathrm{\Gamma }={\mathrm{\Gamma }}_{\mathit{\text{task}}}+{\mathrm{\Gamma }}_{\mathit{\text{posture}}}=J_t^T{F}_t+N_t^T{\mathrm{\Gamma }}_p$$

$$\mathrm{\Gamma }=J_t^T{F}_t+N_t^T\left(J_p^T{F}_p\right)$$

$${\mathrm{\Gamma }}_{\mathit{\text{posture}}}={\left(J_p{N}_t\right)}^T{F}_p=J_{p|t}^T{F}_{p|t}$$

$$\mathrm{\Gamma }=J_t^T{F}_t+J_{p|t}^T{F}_{p|t}$$

$$\mathrm{\Gamma }=J_{t_1}^T{F}_{t_1}+J_{t_2|t_1}^T{F}_{t_2|t_1}$$

Legend:

• Γ: total control torque.

• Γ_task: torque corresponding to the commanded task behavior.

• Γ_posture: torque associated with the posture behavior in the null space of the task.

• J_t: Jacobian associated with the task.

• F_t: force associated with the task.

• N_t: null space projection matrix.

• Γ_p: torque projected into the null space.

• J_p: Jacobian associated with the posture.

• F_p: force associated with the posture.

• J_t2|t1, F_t2|t1: Jacobian and force associated with additional tasks projected into the posture.