Probability-Based Prediction of Activity in Multiple Arm Muscles: Implications for Functional Electrical Stimulation

Abstract

Functional electrical stimulation (FES) involves artificial activation of muscles with implanted electrodes to restore motor function in paralyzed individuals. The range of motor behaviors that can be generated by FES, however, is limited to a small set of preprogrammed movements such as hand grasp and release. A broader range of movements has not been implemented because of the substantial difficulty associated with identifying the patterns of muscle stimulation needed to elicit specified movements. To overcome this limitation in controlling FES systems, we used probabilistic methods to estimate the levels of muscle activity in the human arm during a wide range of free movements based on kinematic information of the upper limb. Conditional probability distributions were generated based on hand kinematics and associated surface electromyographic (EMG) signals from 12 arm muscles recorded during a training task involving random movements of the arm in one subject. These distributions were then used to predict in four other subjects the patterns of muscle activity associated with eight different movement tasks. On average, about 40% of the variance in the actual EMG signals could be accounted for in the predicted EMG signals. These results suggest that probabilistic methods ultimately might be used to predict the patterns of muscle stimulation needed to produce a wide array of desired movements in paralyzed individuals with FES.

INTRODUCTION

Functional electrical stimulation (FES) has long been looked to as a means of restoring function to the upper limb and hand when volitional control has been lost due to paralysis. In the past three decades, a number of formidable technical obstacles to the deployment of FES in patients with spinal cord injuries have been overcome (Handa et al. 1989; Hoshimiya et al. 1989; Keith et al. 1988; Kilgore et al. 1989; Ko et al. 1977; Peckham et al. 2002; Smith et al. 1998). Consequently, worldwide, a small but growing number of patients with injuries to the cervical spinal cord are using FES systems to regain some function of the upper extremities (Peckham et al. 2001; Smith et al. 1998; Triolo et al. 1996).

FES systems, however, remain limited in their ability to produce a wide range of motor behaviors. This is largely due to the absence of a generalized control scheme that, in the best case, could specify the patterns of muscle stimulation needed to elicit any desired movement. Instead, most FES systems use a few stored programs of muscle stimulation, the playback of which is instigated and regulated by a control signal derived from a voluntary motor function retained by the patient (such as eye movements, shoulder movements, or electromyographic [EMG] activity detected from unaffected muscles). One possible approach to overcome this limitation would be to develop a generalized FES controller based on biomechanical models (e.g., Hatze 1980; Putnam 1991; Soechting and Flanders 1997; Stansfield et al. 2003). The premise for using a biomechanical model is that a set of equations could be identified that characterizes the relationship between limb kinematics and muscle activity. These equations could then be solved to yield the patterns of muscle activity associated with some desired kinematic state. However, the complex mechanics of the human arm associated with its many degrees of freedom and its multiple, nonlinear actuators, and operation according to an as yet undetermined control law, have largely thwarted attempts to find analytical solutions for controlling limb movement.

The purpose of this study was therefore to evaluate the efficacy of a probabilistic, rather than deterministic, method to predict patterns of activity in multiple muscles associated with movements of the upper limb in human subjects. This method, based on laws governing conditional probabilities, has previously been shown to accurately predict activity in a few muscles operating on a simple joint system (Seifert and Fuglevand 2002). Here we show that this method can be used to predict, with good fidelity, the complex patterns of muscular activity arising during free movements of the arm based on hand-trajectory information. These findings suggest that such an approach might provide a flexible means to control FES systems and thereby facilitate the production of a wide repertoire of motor behaviors in paralyzed individuals.

METHODS

An overview of the main features of the experimental approach is outlined schematically in Fig. 1. Approximately 20 min of surface EMG and kinematic data were recorded during continuous random movements in the sagittal plane in one subject (Fig. 1, left). These data served as inputs to an algorithm that characterized the relationships between muscle activity and kinematics using a probabilistic method. These probabilistic relations were then used to predict patterns of muscle activity associated with a new set of movements recorded in four other subjects (Fig. 1, right). The rationale for this design was that we sought to determine how well predictions based on training data from one individual could transfer to other individuals. This is of particular importance for the development of a control system to be used in paralyzed patients in whom such probabilistic relationships cannot be readily determined. Predicted patterns of muscle activity were then compared with actual EMG signals to evaluate the capability of this approach to estimate patterns of muscle activity associated with a variety of limb movements.

FIG. 1.

Overview of procedures used to predict electromyographic (EMG) signals from kinematic data. EMG signals from 12 arm muscles and kinematic data from markers placed on the hand, elbow, and shoulder were recorded during a 20-min random-movement task in one “trainer” subject (left). These data served as inputs to a probabilistic algorithm that characterized the probabilistic relationships between EMG and kinematics. Subsequently, kinematic data recorded in 4 other “test” subjects (right) during 8 specified movement tasks served as inputs to the probabilistic algorithm to predict EMG signals. The predicted patterns of EMG were compared with actual EMG signals by calculating the variance accounted for (VAF).

Experimental setup

Experiments were performed on five healthy male human volunteers (ages 25–45 yr) each of whom gave his informed consent to participate in the study, which was approved by the institutional human subjects committee. Subjects sat upright on narrow solid chairs without armrests for the duration of the experiments (Fig. 1). The back of the chair extended only up to about the midthoracic region and did not seem to interfere with movement of the scapula. EMG signals were recorded using surface electrodes from 12 muscles used in controlling arm movement (serratus anterior, anterior deltoid, posterior deltoid, pectoralis major, latissimus dorsi, teres major, biceps brachii, brachialis, brachioradialis, triceps brachii, extensor carpi radialis longus, and flexor carpi radialis). Conductivity gel was placed inside the ceramic housing of the electrodes (Ag-AgCl, 4-mm diameter) that were then fixed on the skin in bipolar configurations over the target muscles (interelectrode spacing, ∼2 cm) using adhesive disks and tape. The leads of the electrodes were secured along the length of the limb with tape and elastic bands and were connected to a small head stage positioned immediately behind the chair.

Three markers (head of a cotton swab dipped in glow-in-the-dark paint and mounted on a small plastic fixture) were taped on the skin to identify in video recording the sagittal-plane locations of the shoulder, elbow, and hand (metacarpal-phalangeal joint on the ulnar aspect of the hand) (Fig. 1). The subject's arm was not physically constrained nor supported in any way, nor was the subject asked to manipulate any objects during movement. However, the forearm and hand were maintained in a pronated orientation throughout the experiments, to ensure that the hand marker was continuously visible to the video camera.

EMG data acquisition

EMG data acquisition was similar to that used by Seifert and Fuglevand (2002). EMG signals were differentially amplified (×1,000) and band-pass filtered (−6 dB cutoff points at 100 and 1,000 Hz) by a bank of 12 analog amplifiers (model 12A5, Grass, West Warwick, RI). The low-frequency cutoff of 100 Hz was chosen to minimize what otherwise could have been substantial movement artifact. Preliminary experiments involving passive movements of the limb and movement of EMG leads indicated that most of the frequency range of the movement artifact signal fell between 0 and 100 Hz. EMG signals were sampled at 2,000 samples·s⁻¹·channel⁻¹ by a computerized data acquisition and control system (CED Power 1401, Cambridge, UK). In addition, a 0.05-s pulse (5 V) was generated every second on a digital-analog output channel of the data acquisition system. This pulse signal was sampled together with the EMG signals and was used to drive a light-emitting diode (LED) that served as visible flag in the video recording to enable synchronization of EMG and kinematic data.

Kinematic data acquisition

Experiments were conducted in low-light conditions and the position of each marker was recorded by a single Panasonic PV-DV52 digital video camera on magnetic tape at 30 frames/s. The glow-in-the-dark paint within the low-light environment provided high-contrast marks that were readily identified in each video frame using automated digitizing software. The camera was placed about 4 m from the subject with the camera axis aligned perpendicular to the sagittal plane of the subject. The flashing LED, fixed to the chair, was also recorded by the video camera and served as a datum for synchronizing the EMG signals with the kinematic data.

Experimental procedures

Subjects was asked to complete eight movement tasks, subsequently described in detail: 1) random movements, 2) figure-of-eights, 3) reverse figure-of-eights, 4) squares, 5) reverse squares, 6) reach low, 7) reach middle, and 8) reach high. Each task involved planar movements of the arm while EMG signals and kinematic data were recorded. Subjects were asked to limit their movements to the sagittal plane as much as possible but to maintain a natural and comfortable movement. For all tasks, subjects were also instructed to limit flexion of the trunk by keeping their back against the chair.

For the first task (Fig. 2A) subjects were instructed to generate a wide variety of random movements at varying speeds while attempting to move through every point in the two-dimensional workspace. Subjects were asked to focus on making natural movements without excessive muscle cocontraction. This task, referred to as the random-movement task, was performed for 15–20 min while EMG and kinematic data were recorded. Subjects were encouraged to take short rest breaks during this task during which the arm hung pendant to the side of the chair. The data from one of the subjects were used to establish the conditional probability distributions used for prediction of EMG from kinematics (see following text). No subject reported muscle fatigue during this or subsequent tasks.

FIG. 2.

Schematic diagram of movement tasks performed by “test” subjects from which recorded arm kinematics were used to predict EMG signals. All movements were carried out in the sagittal plane. Each panel indicates the path that the hand was to approximate during the associated task. A: random movements. B: forward and reverse “eights” task. C: forward and reverse “squares” task. D: high, middle, and low reaches.

Subjects were also asked to perform seven specific-movement tasks including four tasks in which the hand moved in a geometrical pattern and three reaching tasks (Fig. 2, B–D). Subjects were asked to complete 10 trials of each specific-movement task at a natural speed while keeping the velocity and position characteristics as consistent as possible across trials. Subjects were instructed to rest for 2–5 s between each trial.

For the figure-of-eights task (Fig. 2B), the subject was instructed to start from a resting position with the arm pendant, raise the arm to about shoulder level, and then trace out a sideways figure-of-eight, or infinity symbol, in the sagittal plan beginning at the center of the symbol and then moving up and away from the body. After completing the figure, the subject returned the arm to the resting position. The completion of these steps constituted a single trial. Another movement task was similar (reverse figure-of-eights) except that the subject was asked to reverse the direction of movement such that movement from the center of the symbol began down and toward the body.

For the squares task (Fig. 2C), the subject was again instructed to start from a resting position with the arm pendant, raise the arm to about chest level, then trace out a square, with the direction of movement beginning with the bottom corner of the square nearest the body and progressing outward, away from the body. After completing the motion to trace out the square, the subject returned the arm to the resting position. The reverse-squares task was similar except that the square traced out by the hand was performed in the opposite direction.

For the reaching tasks (Fig. 2D), subjects were instructed to start from a resting position with the arm pendant, and then reach to a self-specified location in space as if touching a target, although no real target existed. Subjects were asked to reach to the full extent of the arm length. For the high-reaching task, subjects reached to a location above the head and in front of the body. For the middle-reaching task, subjects reached to a location about shoulder level and in front of the body. For the low-reaching task, subjects reached to a location about knee level and in front of the body.

EMG processing

Several EMG processing stages were performed off-line using Matlab (MathWorks, Natick, MA). Visual inspection of the raw EMG signals revealed sporadic large-amplitude noise spikes that appeared to be due to movement artifact. These were large enough to distort the normalized EMG and thus degrade the probability-based predictions. In an effort to reduce the effect of these large-amplitude spikes, the following heuristic was implemented, similar to that used by Sanger (2007). For each channel, the DC bias was first removed by subtracting the mean of the recorded EMG signal. A threshold value was then found below which 99.99% of the data points fell. All data points with amplitude values above this threshold were set to the threshold value. This process was essentially equivalent to excluding EMG values that exceeded 5SDs of the mean. Visual inspection of the EMG showed that this method was effective in removing large-amplitude noise spikes.

In an effort to boost the signal-to-noise ratio beyond that achieved by analog filtering during data acquisition, an aggressive, zero-phase, digital band-pass filter was applied. The filter was a 20th-order Butterworth with a band-pass frequency range of 100 to 500 Hz applied in the forward and backward directions to minimize phase distortion. Then, according to conventional processing methods (Winter 2005), EMG signals were full-wave rectified and low-pass filtered (sixth-order, Butterworth, zero phase, −3 dB cutoff frequency of 6 Hz). The band-limited representation of muscle activation was then downsampled to 30 Hz and normalized with respect to the maximum value found for that muscle during the random-movement task.

Kinematic processing

In off-line processing, the centroid pixel position of each kinematic marker in each frame of the digital video was identified and stored using MaxTraq software (Innovision Systems, Columbiaville, MI). Next, the digital video was searched to identify each frame wherein the synchronizing LED flash was visible and its frame number was recorded. Because the LED flashed once per second and the video frame rate was about 30 frames/s, marker position data were separated into epochs each 30 frames long with the beginning of each segment corresponding in time to a specific pulse of the LED. The recorded EMG signals and the kinematic data were then aligned by matching each 30-frame segment of kinematic data with the corresponding pulse signal recorded on the synchronization channel of the EMG data acquisition system.

Position data were translated into a coordinate system where the shoulder represented the origin and the x (horizontal) and y (vertical) components for the hand and elbow markers were scaled by the maximal displacement of the hand relative to the shoulder during the random-movement task. Finally, kinematic data were also low-pass filtered by a zero-phase sixth-order Butterworth filter with a 6-Hz cutoff frequency. Then, to account for the time lag between EMG activity and muscle force (and thus kinematics), a fixed delay of 60 ms was added to the EMG signals (Manal and Rose 2007).

Probabilistic estimation of EMG

The approach that we used to predict the time course of EMG signals was based on elementary laws governing conditional probabilities. Although previously we used a Bayesian method to predict EMG signals from kinematics (Seifert and Fuglevand 2002), here we have taken a more direct approach. In general, we sought to estimate the probability that the level of activity within a muscle would attain a particular value, EMG, given that the limb was in a specified kinematic state K: P(EMG|K). In practice, we sought to know the probabilities associated with all different level of EMG, not just a single value, given a particular kinematic state. Therefore the conditional probability could be represented as P(EMG_j|K), where EMG_j indicates the set of all possible EMG values discretized into 1% increments, and P(EMG_j|K) is a probability distribution indicating the likelihood of attaining different values of EMG given that the limb is in kinematic state K.

Bayes' theorem provides a means to estimate such condition probabilities based on previously obtained probability information that usually is more readily available than the specific conditional probability of interest. For example, according to Bayes' theorem, we can represent our conditional probability as

(1)

where P(K|EMG_j) is the probability that the limb enters into the particular kinematic state K given different levels of EMG, P(EMG_j) is the overall probability that the muscle takes on different EMG values, and P(K) is the overall probability that the limb attains the kinematic state K. The information represented by the terms on the right side of Eq. 1 can be obtained from previous observations of EMG activity and kinematics in one or more subjects. In the present situation, however, we obtained the conditional probability P(EMG_j|K) directly from information gathered during experiments involving the random movement task in one subject, as subsequently described.

To illustrate how a single conditional probability distribution P(EMG_j|K) was constructed, consider an EMG signal recorded from one muscle and a single kinematic parameter, say the x (horizontal) position of the hand (Fig. 3A). If we select a horizontal hand position that is, for example, 60% of the maximum displacement (Fig. 3A, horizontal line), then for every occurrence of that hand position, we extract the associated EMG values recorded from the muscle (Fig. 3A, vertical lines). From those values, we construct the conditional probability distribution, P(EMG_j|x = 60%) (Fig. 3B), which indicates the likelihood that the muscle attained different EMG levels given that the hand was at the 60% horizontal position. This process is repeated for each increment in hand position from the minimum to maximum position (also discretized into 1% increments) to generate a stack of conditional probability distributions. Such a stack of conditional probability distributions can be represented as a colorized surface (Fig. 3C) for which the height (color) at any location on the surface indicates the probability of attaining a particular level of EMG given that the hand is at a specified horizontal location. The highlighted region on the surface shown in Fig. 3C represents the conditional probability distribution shown in Fig. 3B.

FIG. 3.

Method by which conditional probability distributions were generated for each kinematic parameter and EMG signal from training data. A: brief time segment of EMG from one muscle (serratus anterior) and one kinematic parameter (X position of the hand) recorded as part of extensive training data set. For each increment in hand position (e.g., 60% of maximum excursion, red horizontal line), the corresponding values of EMG (normalized to the maximum value recorded in the training set) were extracted (red vertical lines) and were then used to construct the conditional probability distribution shown in B. B: conditional probability distribution indicating the likelihood that the level of EMG in serratus anterior attained different values given that the X position of the hand was at 60% of maximum excursion, i.e., P(EMG_j|X = 60%). C: conditional probability surface generated from a stack of conditional probability distributions, one for each 1% increment in X position of hand. The highlighted region (thin yellow rectangle) indicates the single conditional probability, P(EMG_j|X = 60%), shown in B.

Multiple parameters are needed to fully characterize the kinematic state of the upper limb. However, as pointed out in the following text, an assumption of the probabilistic method is that the kinematic parameters be independent of one another. For this reason, we did not include kinematics of the elbow in our estimator of EMG signals because 1) hand trajectory depends on the kinematics of the elbow (particularly for planar movements), and 2) with a relatively fixed shoulder position, the elbow itself is constrained to follow an arced path, and therefore the x and y coordinates of the elbow are not independent of one another. Consequently, for the present experiments, the kinematic state of the limb was represented in terms of the positions, velocities, and accelerations of the hand relative to the shoulder. Since our movements were confined to the sagittal plane, the kinematic state of the upper limb was defined by six parameters, the horizontal and vertical components of position (x, y), velocity (ẋ, ẏ), and acceleration (ẍ, ÿ) of the hand (H). Then, under the simplifying assumption of independence among the kinematic parameters, the probability that EMG attains different values is given by the product of the individual conditional probability distributions, that is

(2)

Each element on the right side of Eq. 2 is a condition probability distribution extracted from a surface like that shown in Fig. 3C. The product of these conditional probability distributions yields the overall probability of obtaining different levels of EMG given the specific constellation of kinematic parameters representing the kinematic state of the arm at that moment. From such a resultant conditional probability distribution, an estimate is made of the most likely value of EMG associated with the kinematic state. Although there are several methods for making this estimation, for this study the expected (mean) value of the probability distribution was chosen as the estimated EMG level (Seifert and Fuglevand 2002).

Data analysis

For this study, one subject was selected to be the trainer. The trainer's preprocessed data from task 1, the random-movement task, was labeled the training set and was used to establish 72 conditional probability surfaces (12 muscles × 6 kinematic parameters). This set of conditional probability surfaces then served as the foundation for prediction of muscle activity associated with each of the movement tasks recorded in the four other test subjects. The kinematic data recorded during these tasks served as the inputs to the probabilistic algorithm (Fig. 1, right). The probabilistic algorithm essentially extracted six conditional probability distributions (one for each of the specified values of each kinematic parameter) from the conditional probability surfaces, multiplied them together (according to Eq. 2), and then found the mean of the resultant distribution to serve as the predicted level of activity in that muscle at that moment. The process was repeated for each of the 12 muscles at each time step over the entire duration of the movement task.

Linear regression between the predicted and the actual EMG was used to calculate the coefficient of determination, from which the variance accounted for (VAF) was estimated for each muscle during each movement task and for each subject (Stein et al. 2004). In addition, the root-mean-squared (RMS) error between the predicted and actual EMG signals was calculated for every muscle of every trial. A two-way repeated-measures ANOVA, with muscle and task as factors, was performed on the VAF to establish whether the match between predicted and actual muscle activity was better for some muscles and for some tasks than for others. Post hoc analysis using pairwise multiple comparisons was carried out using the Holm–Šidák method. In addition, we reperformed the prediction of EMG signals multiple times, each time successively dropping a set of kinematic parameters from the probabilistic algorithm. This was done to determine the sensitivity of the predictions to the type of kinematic information included in the algorithm. The resulting VAF values associated with inclusion of different sets of kinematic parameters were also tested for statistical significance using a one-way ANOVA. The level of significance chosen for statistical tests was P < 0.05.

RESULTS

The average speed of hand movement during the random-movement task across the five subjects was 0.83 ± 0.07 m/s and the average peak speed attained during this task was 5.3 ± 1.1 m/s. Figure 4 shows a brief time segment of hand position data (Fig. 4A) and processed EMG signals (Fig. 4B) obtained from the 12 muscles of the trainer subject during the random-movement task. From the same training-data set but depicted on a longer timescale, Fig. 5A shows the processed EMG from one muscle (serratus anterior) and one kinematic parameter (horizontal or X position of the hand). No obvious association was evident between these two signals. Indeed, a plot of EMG amplitude versus X position of the hand for each time increment (Fig. 5B) revealed no significant correlation (P > 0.2) between these two variables (VAF < 0.1%). Similar results were found for other combinations of muscles and kinematic parameters. This example illustrates, as might be expected, that no simple predictive relation existed between individual kinematic parameters and patterns of muscle activity.

FIG. 4.

Example segment of training data recorded in one subject during 20-min random-movement task. A: hand trajectory. B: rectified, filtered EMG signals from 12 arm muscles. Scale bar 0.2 mV for all EMG channels. FCR, flexor carpi radialis; ECR, extensor carpi radialis; Triceps, triceps brachii; BrachRad, brachioradialis; Brach, brachialis; Bicep, biceps brachii; Teres, teres major; Lats, latissimus dorsi; PecMaj, pectoralis major; PostDelt, posterior deltoid; AntDelt, anterior deltoid; SerAnt, serratus anterior.

FIG. 5.

Example EMG and kinematic data from the training set. A: processed EMG from one muscle (serratus anterior, top trace) and one kinematic parameter (X position of the hand, bottom trace). B: scatterplot of serratus anterior EMG amplitude vs. X position of the hand for each time increment over the period shown in A. Regression analysis revealed no significant correlation (P > 0.2) between these 2 variables (VAF < 0.1%).

From the training data, six conditional probability surfaces (one for each kinematic parameter) were generated for each muscle (using the method depicted in Fig. 3). An example illustrating how these conditional probability surfaces were subsequently used to predict EMG activity in one muscle of a test subject associated with a particular movement task is shown in Fig. 6.

FIG. 6.

Schematic diagram representing conversion of upper limb kinematics into predicted EMG signals by application of conditional probability algorithm. For clarity, only 2 of the 6 kinematic parameters used in the prediction of EMG of one muscle (serratus anterior) are shown. A: sample test kinematics (one trial of forward-eights task). At time indicated by red vertical line, X hand position was anterior to the shoulder, about 70% of maximum excursion, and the Y hand position was below (i.e., a negative value) the shoulder, about −65% of maximum excursion. These values were then used to select from the conditional probability surfaces (B), the conditional probability distributions associated with the specific values of the kinematic parameters (thin rectangles). The conditional probability distributions highlighted by the rectangles on the color plots in B are redrawn as histograms in C. C: histograms indicating the conditional probability that EMG takes on different values given that the X position of the hand is at 70% maximum excursion (top) and given that the Y position of the hand is at −65% of maximum excursion (bottom). These histograms (plus those for the other 4 kinematic parameters) were then multiplied together to yield the overall conditional probability distribution shown in D. D: probability of obtaining different levels of EMG given the constellation of 6 kinematic parameters representing the kinematic state of the limb at the moment indicated by the vertical line in A. The mean value of this distribution (red line) was used as the best estimate of the EMG given the specified values of the kinematic parameters. This process was repeated for each time increment over the entire trial duration.

Figure 6A depicts the kinematic signals representing the trajectory of the limb recorded during one trial of the forward-eights task in a test subject. For clarity, only two of the six kinematic parameters are shown. In this trial, the arm began in a pendant position with the hand well below and slightly in front of the shoulder; the arm was then raised forward such that the hand was slightly above shoulder height, the figure was traced out, and then the hand returned to the start position. The red vertical line (Fig. 6A) indicates the particular time at which a prediction of EMG is to be made. The horizontal arrows signify the specific values of kinematic parameters at that instant.

The conditional probability surfaces indicating the likelihood of attaining different levels of EMG in one muscle (serratus anterior) given x and y positions of the hand are shown in Fig. 6B. The conditional probability distributions associated with the specific values at the time instant in question (regions of colored surfaces highlighted by the thin rectangle) are redrawn as histograms in Fig. 6C. Then, in accordance with Eq. 2, these conditional probability distributions were multiplied together to yield an overall probability of obtaining different levels of EMG given the specific constellation of the six kinematic parameters (Fig. 6D). The mean value of this resultant distribution was calculated (Fig. 6D, red line), which served as the best guess of the value of EMG for that time instant (Seifert and Fuglevand 2002). This procedure was then repeated for each time increment and for all 12 muscles.

Figure 7 shows typical patterns of predicted EMG from three muscles during the random-movement task in one test subject using this probabilistic method. For each muscle shown, the predicted pattern of EMG activity (Fig. 7, black traces) matched reasonably well to the actual EMG signals recorded in the test subject (Fig. 7, gray traces). For the anterior deltoid (Fig. 7, top), the predicted EMG accounted for 60% of the variance recorded in the actual EMG over the entire duration of the random task, and was 66% for latissimus dorsi and 69% for teres major. The RMS errors between predicted and actual values for these three muscles varied from <1% for latissimus dorsi to about 8% for anterior deltoid. The RMS errors scaled in rough proportion to the intensity of muscle activity recorded during the trial (note difference in vertical scale for the three muscles). For most muscles, including those shown in Fig. 7, the predicted levels of EMG tended to undershoot the actual values, particularly during transient, high-magnitude activity. Nevertheless, it is important to recall that 1) the predicted values of EMG were based only on the kinematic state of the limb at a given instant; 2) the probabilistic algorithm itself was derived from training data in a different subject; and 3) the instantaneous value of the actual EMG can vary over a relatively wide range of values, even under identical kinetic conditions (Enoka and Fuglevand 1993; Yang and Winter 1983).

FIG. 7.

Example traces of actual (gray) and predicted (black) EMG signals for 3 muscles (anterior deltoid, top; latissimus dorsi, middle; teres major, bottom) associated with a brief segment of the random-movement task in one test subject. In general, the patterns of predicted EMG were similar to the recorded EMG signals. VAF, variance accounted for; RMSE, root-mean-squared error.

For all 384 comparisons (12 muscles × 8 tasks × 4 test subjects) between predicted and actual EMG signals, linear regression indicated that 379 (97%) were significantly correlated at a stringent level of significance (P < 0.001). Two-way ANOVA on VAF revealed significant effects of both task (F = 5.07, P = 0.002) and muscle (F = 4.48, P < 0.001), but no significant interaction between these factors (F = 1.30, P > 0.05). Figure 8A shows the mean (SD) VAF for each task, averaged across all subjects and muscles. Post hoc analysis on VAF indicated that only the random-movement task exhibited significantly different VAF values from a few other tasks (forward eights, reverse eights, reach high, and reach middle). With the exception of the random-movement task (VAF = 24 ± 22%), VAF was around 40% for most other tasks (grand mean VAF across all tasks was 38 ± 26%). The poorer prediction for the random task presumably was due to the much more extensive movement conditions associated with this task compared with those of the others. The mean RMS error was ≤10% for all tasks and the grand mean RMS error was 8.2%.

FIG. 8.

Mean (SD) VAF between predicted and recorded EMG signals across different tasks (A) and across different muscles (B). There was a significant effect of task on VAF primarily due to the relatively low VAF associated with the random-movement task (A). Likewise, there was a significant effect of muscle on VAF. This was primarily related to the high VAF for the anterior deltoid muscle.

Figure 8B shows the mean VAF for all muscles averaged across subjects and tasks. Post hoc analysis indicated that the VAF for the anterior deltoid (74 ± 8%) was significantly greater than that for six other muscles (pectoralis major, brachialis, brachioradialis, triceps, extensor carpi radialis, flexor carpi radialis). No other pairwise comparison was significant. The high degree of correspondence between predicted and actual EMG signals for the anterior deltoid compared with that for other muscles was likely related to its role as the major contributor to torque at the shoulder during sagittal-plane movements (Soechting and Flanders 1997).

The muscle for which predictions of EMG were the worst was triceps (VAF = 17 ± 19%). Also, the RMS error for the triceps (41%) was markedly higher than that for the other muscles, which tended to have RMS errors <10%. The low VAF and the high RMS error for the triceps were seen in all four of the test subjects. The poor prediction of triceps EMG appeared to be related in part to the inferior quality of the triceps EMG signal recorded during the training set. As seen in Fig. 4B, little modulation of the triceps EMG signal occurred during the random-movement task used to train the probabilistic algorithm despite the expectation that the triceps should have been strongly active during some of the arm movements.

To evaluate whether deficient training data contributed to the poor prediction of triceps EMG, we reran the analyses, although this time we selected a different subject's data to serve as the training set. As before, there were significant effects of task (F = 9.5, P < 0.001) and muscle (F = 28.1, P < 0.001) on VAF but no significant interaction between these factors. The random-movement task again was that associated with the lowest VAF (21 ± 19%) and the grand mean VAF (37 ± 25%) across all subjects, tasks, and muscles was nearly identical to that obtained originally. Once again, anterior deltoid was the muscle exhibiting the highest VAF (73 ± 10%). The prediction of triceps EMG was improved with the new training set (VAF = 25 ± 20%; mean RMS error = 3.5%), whereas a different muscle, the brachioradialis, now had the lowest VAF (14 ± 11%) and an exceptionally high value of RMS error (47%). These results highlight the importance of obtaining quality recordings from all muscles comprising the training set. More important, however, the general capability of the probabilistic method to accurately predict muscle activity patterns did not appear to be highly sensitive to the particular subject used to obtain the training data.

Another possible source of error between actual and predicted EMG relates to the amount of data included in the training set. We reperformed the prediction of EMG for all muscles in one subject during the forward-squares tasks based on conditional probability surfaces that were derived from progressively smaller segments of the original training set. Figure 9 shows the values of the mean (SD) VAF associated with different durations of training data. For the planar movements used in the present experiments, little additional improvement in EMG prediction was gained for training-data sets in excess of about 5 min.

FIG. 9.

Effect of duration of training-data set on VAF between predicted and recorded EMG signals. Kinematic data associated with forward squares task from one test subject were used as inputs to probabilistic algorithm to predict EMG signals in 12 muscles. Each data point represents the mean VAF (SD) between predicted and recorded EMG across all muscles and the 10 trials of the task for different durations of the original training data set used to form the conditional probability surfaces needed for the predictions.

It was also of interest to explore the role that the selected kinematic parameters played in the prediction of EMG. Clearly, the complete kinematic state of the upper limb was not fully represented in the subset of parameters (see Eq. 2) that we chose to characterize movements of the limb. For example, higher-order time derivatives of displacement, such as jerk, or other anatomical locations of the limb, such as the wrist and elbow, could have been included as additional parameters to represent limb kinematics.

To gain some insight into the sensitivity of EMG estimation to the selected kinematic parameters, EMG prediction was reperformed for all 12 muscles during random-movement tasks in one subject, although this time kinematic data from the elbow (positions, velocities, accelerations of its X and Y coordinates) were also included. Therefore 12 kinematic parameters (rather than 6) were used in Eq. 2 to predict EMG signals. Then, EMG predictions were repeatedly made, but each time progressively dropping one set of kinematic parameters. As shown in Fig. 10, the accuracy of EMG prediction was relatively insensitive to the particular set of kinematic parameters used to make the estimation. Indeed, there appeared to be modest improvement in VAF as fewer kinematic parameters were included in the estimator. ANOVA, however, indicated that these differences in VAF were not significantly different from one another.

FIG. 10.

Mean (SD) VAF between predicted and recorded EMG in 12 muscles for different sets of kinematic parameters included in the estimator. The labels below each bar indicate the kinematic parameters that were used in the estimator (H, hand; E, elbow; pos, position; vel, velocity; acc, acceleration). In all cases, both X and Y coordinates of each parameter were included.

The weak tendency for estimators that included more kinematic parameters to be associated with lower VAF (as shown in Fig. 10) was somewhat surprising. Such a result, however, might be partially explained by considering the following three factors. First, all kinematic parameters were given equal weight in estimating EMG. Thus the predicted value was not unlike an average of the estimates derived from the individual kinematic parameters. If, however, one or more of the kinematic parameters were poorly related to the patterns of muscle activity under investigation, then the performance of the estimator would be undermined by inclusion of such parameters. For example, had we included the degree of abduction of the little finger as a kinematic parameter in the present investigation, our predictions likely would have suffered because this parameter 1) is unrelated to the patterns of recorded muscle activity and 2) would be given as much weight in the prediction as a more salient kinematic parameter, such as the horizontal location of the hand.

The second, and perhaps more critical, factor relates to the assumption of independence among the kinematic variables used in the estimator (i.e., Eq. 2). If some kinematic variables were strongly correlated with others, then inclusion of such interrelated variables might fail to improve the prediction and may even undermine it to some degree. Indeed, inclusion of elbow kinematics tended to reduce the accuracy of the predicted EMG, presumably because of the interdependence of hand and elbow trajectories. To evaluate such assumptions of interdependence and independence more objectively, 5 min of kinematic data (i.e., 9,000 data points per parameter) were taken from the training set (including elbow data) and were used to correlate each kinematic parameter with every other kinematic parameter. The coefficient of determination (R²) was then calculated for each of these pairwise correlations. Examples of such correlations are shown in Fig. 11, A–C. The majority of correlations between kinematic variables (47/66 or 71%) had R² values <0.05, like that shown for X position of the hand versus X velocity of the hand in Fig. 11A. However, a number of comparisons across kinematic variables (15/66 or 23%), such as that between X and Y positions of the hand (Fig. 11B), exhibited moderate levels of correlation (0.5 > R² > 0.05); moreover, four comparisons (6%) had high levels of correlation (R² > 0.5), like that shown in Fig. 11C. Each of these involved comparison between a kinematic variable for the hand and the same variable for the elbow (see diagonal dashed line, Fig. 11D). Therefore as expected, hand and elbow trajectories were highly related to one another and excluding the elbow from the estimator (Fig. 10) weakly improved the overall predictions.

FIG. 11.

Evaluation of independence among kinematic parameters. Five minutes of kinematic data (i.e., 9,000 points per parameter) from the training set were used to correlate every kinematic variable with every other kinematic variable. Example scatterplots for X hand position vs. X hand velocity (A), X hand position vs. Y hand position (B), and X hand position vs. X elbow position (C). Coefficients of determination (R²) were calculated for each pairwise correlation. D: matrix showing correlations for all pairwise combinations of kinematic variables. Each combination was categorized as exhibiting low correlation (R² < 0.05, white), moderate correlation (0.5 > R² > 0.05, yellow), or high correlation (R² > 0.5, red). In addition, autocorrelations with R² = 1.0 are also shown in black. Whereas most combinations of kinematic variables had either low (71%) or moderate (23%) correlations, a few (6%) exhibited high correlation. These high R² values all involved correlations between a kinematic variable of the hand and the same variable for the elbow (dashed diagonal line). pos, position; vel, velocity; acc, acceleration; elb, elbow.

Finally, a given kinematic parameter might provide critical information for predicting EMG during certain phases of the movements but not during others. If the total duration of those phases during which the kinematic parameter contributed profitably to the prediction was briefer than that of the other phases, then inclusion of that parameter might diminish the overall accuracy of the EMG estimation. This seemed to be the case in the present study, as illustrated in Fig. 12. In this example, the black trace shows two successive bursts of activity recorded in the posterior deltoid muscle during the figure-of-eights task. The superimposed colored traces show the predicted patterns of EMG activity using different sets of kinematic parameters (blue: hand position only; green: hand velocity only; and red: both hand position and velocity). During the roughly steady state phases of EMG activity, prediction was better using hand position information only. At the outset of each burst (see inset), however, prediction was better using hand velocity information. Inclusion of both parameters (i.e., red trace), led to predictions that were intermediate between position-only–based and velocity-only–based predictions. Because the total duration of the relatively steady state phases of EMG activity in the present set of movement tasks tended to exceed the duration associated with EMG onset, the overall effect of including velocity (and acceleration) information tended to modestly diminish the overall accuracy of the EMG predictions (Fig. 10).

FIG. 12.

Example predicted EMG signals using different kinematic parameters. Black trace shows actual EMG signal recorded in posterior deltoid muscle of one test subject during the forward-eights task. Blue trace indicates predicted EMG based on hand position only, green trace indicates predicted EMG based on hand velocity only, and red trace shows predicted EMG based on both hand position and velocity. During roughly steady state phase of EMG activity, prediction was better using position data (blue trace) only. However, as shown in the inset, at the outset of each EMG burst, prediction was somewhat better using velocity information only (green trace). Inclusion of both parameters led to predictions intermediate between position-only–and velocity-only–based predictions.

DISCUSSION

The main finding of this study was that complex patterns of muscular activity associated with multijoint upper limb movements could be accurately predicted from hand kinematics using a probabilistic approach. Furthermore, probabilistic information about EMG and kinematics derived from one individual could be used to predict—with reasonable accuracy—the EMG activity in other individuals. The implications of these results for the control of functional electrical stimulation (FES) systems are outlined in the following text; but, first, some of the limitations of the present study are described.

Limitations

The goal of the present experiments was to predict patterns of muscle activity associated with free movements of the upper limb in human subject. As such, we did not attempt to characterize the complex muscular activity involved in controlling the digits. Although one of the major deficits associated with quadriplegia relates to inadequate control of the fingers, the primary aim of the present study was to test the feasibility of probabilistic methods to predict patterns of muscle activity needed to control functional electrical stimulation. For practical reasons, we limited our scope to muscles that acted on the scapula, shoulder, elbow, and wrist only. In principle, a similar approach could be implemented for the muscles controlling the digits together with muscles operating on the rest of the limb, although this would require a considerable increase in the number of sensors required to monitor the complex motions of the hand and electrodes to record the activities of the many muscles acting on the digits.

Furthermore, the movements tested here were restricted to those not involving external contact forces or loads. In theory, it should be possible to include hand or digit contact forces detected with artificial sensors as another set of inputs to the probabilistic algorithm for prediction of EMG signals. Also, the present method for predicting EMG signals assumes a fixed vertical posture of the trunk. Although such a requirement would generally be appropriate for controlling FES in quadriplegic individuals when seated in wheelchairs, it certainly would be inadequate for other situations (e.g., when lying in bed). Consequently, additional inputs to the algorithm, such as that arising from a sensor detecting trunk orientation, would be needed to handle changes in trunk posture. We felt it judicious, however, to determine first whether the probabilistic approach could yield satisfactory results under somewhat restricted conditions. Inclusion of features such as finger movements, contact forces, and trunk orientation is important for future development of this system.

A major limitation of the present study was that movements were performed in the sagittal plane only. Compared with more natural three-dimensional (3D) movements, constraints associated with planar movements may simplify the relation between EMG and kinematics (Flanders and Soechting 1990). Therefore a crucial future test of the utility of this probabilistic approach will be to evaluate its performance during 3D movements.

Another limitation of the present investigation was that the movements tested were relatively slow. Subjects were simply instructed to move at a comfortable speed, and most adopted a deliberate and moderate pace of movement. Previous studies have shown that EMG activity in the arm during slow movements primarily reflects a postural component counteracting the effect of gravity (e.g., Flanders and Herrmann 1992; Flanders and Soechting 1990; Flanders et al. 1996). Consequently, and as shown in Fig. 10, higher-order kinematic parameters (i.e., velocity and acceleration) were not critical for the prediction of EMG signals in the movements tested here. It seems likely that had we tested movements of a more rapid or ballistic nature, higher-order kinematic parameters would have played a more important role in predicting EMG. Indeed, in a similar study involving rapid finger movements, we found that such higher-order kinematics were crucial for predicting EMG signals (Seifert and Fuglevand 2002).

An additional limitation of the present approach relates to a priori assumptions of independence among kinematic parameters used to predict EMG signals. From the analyses shown in Fig. 11, such assumptions were shown not to be valid for a number of kinematic parameters. For example, as expected, elbow and hand kinematics for planar movements were highly correlated with one another. Consequently, inclusion of information about the elbow in the estimator did not improve its performance. However, this should not necessarily be considered a universal result. For example, had we assessed 3D movements, inclusion of elbow kinematics might have proved important for accurate prediction of EMG activity. Therefore a challenge for future use of this probabilistic method will be to develop an efficient means to determine which kinematic variables are best suited for prediction of muscle activity over a wide range of motor behaviors.

Previous studies using probabilistic methods

Our use of a probabilistic method to predict EMG activity from motor behavior was largely inspired from the successful application of similar approaches to a wide range of neurophysiological problems (for comprehensive reviews, see Rao et al. 2002; Rieke et al. 1997). For example, accurate prediction of the path that an animal takes while navigating through a maze can be obtained using probabilistic analyses applied to the firing behavior of populations of hippocampal neurons (Brown et al. 1998; Zhang et al. 1998). Likewise, Bayesian analysis has been applied to the recorded activities of populations of spinal interneurons to predict locomotor behavior (Tresch and Kiehn 2000) and to motor cortical neurons to predict upper limb movements (Gao et al. 2002). Furthermore, Bayesian methods have recently been used to extract accurate estimates of the torque generated at a joint from surface EMG recordings (Sanger 2007).

We have previously shown that for a simple musculoskeletal system, consisting of three muscles and three one-degree-of-freedom joints, a probabilistic method based explicitly on Bayes' theorem could predict EMG signals reasonably well (average RMS error 12%; Seifert and Fuglevand 2002). For the present experiments, we built on that approach to develop a method that provided a more direct, though identical, result to that which would be achieved using Bayes' theorem. Importantly, the present results expanded on those of Seifert and Fuglevand (2002) by showing that activity patterns can be accurately estimated using this probabilistic method for a large number of muscles operating on a complex limb during a wide range of natural movements.

Forward versus inverse approaches

A traditional method used to understand the control of limb movement involves the prediction of muscle forces or joint torques from EMG signals (e.g., Sanger 2007) that then serve as inputs to a biomechanical model of the limb to estimate resulting kinematics (Otten 1987; Soechting and Flanders 1997; Stein et al. 1988; Valero-Cuevas et al. 2003). Such a procedure is referred to as a forward approach. Similarly, nondeterministic methods using artificial neural networks have also been successfully used to generate forward predictions of limb kinematics from EMG signals (Au and Kirsch 2000; Cheron et al. 1996; Koike and Kawato 1995).

Although the forward approach has provided critical insights into the control of movement, it might not be the most appropriate choice for the particular application addressed in the present investigation. Here, our goal was to identify the patterns of activity across a population of muscles that would be needed to elicit a desired movement in paralyzed individuals using FES—thus our approach was framed as an inverse one. Moreover, such an inverse representation is not unlike that which is likely to occur in the CNS during voluntary movements: that is, the collective activity of populations of neurons in motor areas of the cerebral cortex appears to provide a moment-by-moment representation of the desired trajectory of a limb (Georgopoulus et al. 1989; Moran and Schwartz 1999a,b; Schwartz 1993; Schwartz and Moran 1999). Presumably, then, such trajectory information is conveyed to “lower” levels in the CNS and is transformed into the appropriate patterns of muscle activity needed to elicit the desired movement. Also, because a long-term goal of our work is to use brain-derived trajectory information (Chapin et al. 1999; Musallam et al. 2004; Serruya et al. 2002; Taylor et al. 2002; Wessberg et al. 2000) as input to a probabilistic controller of FES, this inverse approach (i.e., from desired movement to associated patterns of muscle activity) seems fitting.

Implications

Only a small fraction of the spectrum of possible upper limb movements can be realized with current FES systems. This is mainly due to limitations in the way that these systems are controlled. The control of most upper limb FES systems consists of selection and triggering of one of a small set of preprogrammed stimulation patterns that involve little feedback. A greater repertoire of movements has not been implemented primarily because of the enormous difficulty related to identification of the patterns of muscle activity needed to produce specified movements. Most limb movements, even those involving a single digit, require intricate coordination among multiple muscles that act across several joints (Scheiber 1995; Valero-Cuevas 2000). Such complex mechanical systems do not readily lend themselves to deterministic solutions. Although electromyographic (EMG) signals recorded from able-bodied subjects have been used to identify the patterns of muscle activity associated with a particular movement (Hoshimiya et al. 1989), this painstaking method yields control signals appropriate only for the motor task from which the EMG signals were originally obtained.

Here we have used a probabilistic method to predict the patterns of muscle activity needed to produce, in theory, an unlimited set of movements across multiple joints. We have also shown that complex patterns of muscular activity associated with arm movements could be accurately estimated based primarily on hand-trajectory information. Therefore it seems feasible that predicted patterns of muscle activity associated with a desired hand trajectory could be transformed into frequency- or amplitude-modulated pulse trains to drive a set of muscle stimulators to evoke movements in paralyzed subjects. Indeed, we have previously shown that predicted levels of EMG can be readily converted into patterns of electrical stimulation that, in turn, evoke movements that are quite close to desired trajectories (Seifert and Fuglevand 2002). Furthermore, the probabilistic relationship between EMG and kinematics derived from one individual could be used to predict patterns of activity appropriate to control muscles in other individuals. The practical implication of this finding is that a functional electrical stimulation system using this probabilistic control strategy could be trained on able-bodied subjects and then be deployed, at least as a first estimate, in paralyzed subjects.

GRANTS

This work was supported by National Institute of Neurological Disorders and Stroke Grant NS-39489 and by a grant from the BIO5 Institute of Arizona.