An Analysis of Intrinsic and Extrinsic Hand Muscle EMG for Improved Pattern Recognition Control

Abstract

Pattern recognition control combined with surface electromyography (EMG) from the extrinsic hand muscles has shown great promise for control of multiple prosthetic functions for transradial amputees. There is, however, a need to adapt this control method when implemented for partial-hand amputees, who possess both a functional wrist and information-rich residual intrinsic hand muscles. We demonstrate that combining EMG data from both intrinsic and extrinsic hand muscles to classify hand grasps and finger motions allows up to 19 classes of hand grasps and individual finger motions to be decoded, with an accuracy of 96% for non-amputees and 85% for partial-hand amputees. We evaluated real-time pattern recognition control of three hand motions in seven different wrist positions. We found that a system trained with both intrinsic and extrinsic muscle EMG data, collected while statically and dynamically varying wrist position increased completion rates from 73% to 96% for partial-hand amputees and from 88% to 100% for non-amputees when compared to a system trained with only extrinsic muscle EMG data collected in a neutral wrist position. Our study shows that incorporating intrinsic muscle EMG data and wrist motion can significantly improve the robustness of pattern recognition control for partial-hand applications.

I. INTRODUCTION

THE human hand is an amazingly precise and agile apparatus, used to perform actions that range from delicate and intricate to forceful and strenuous. Full or partial loss of the hand thus profoundly affects many activities of daily living. Of the 18,500 individuals in the United States who undergo upper-limb amputations each year, 91% have amputations that are distal to the wrist [1, 2]. These partial-hand amputations include transmetacarpal amputations (1.5%), thumb amputations (18%) or amputations of one or more digits [1]. Because partial-hand amputation can involve various levels of longitudinal and transverse loss, it is difficult to treat successfully with a prosthesis [3], consequently partial-hand amputees have relatively fewer prosthetic options than individuals with higher-level amputations [4].

Passive prostheses can provide good cosmesis and act as an oppositional post for a remaining finger but do not offer active functionality [3]. Body-powered devices provide more function and grasps, but are uncomfortable to wear, have cumbersome control mechanisms, and often limit the range of motion of the wrist [5, 6]. Partial-hand amputees perceive themselves to be at a higher disability level than those with unilateral transradial or transhumeral upper-limb amputations [7, 8] and are more likely to reject their prostheses, citing limited function as a key reason [9]. Moreover, more than half of partial-hand amputees are unable to return to previous employment and of those that do, most do not find their prosthesis useful [10]. Partial-hand amputees are therefore in need of better, more functional prostheses.

Though externally-powered prostheses have been available to upper-limb amputees for decades [11], they have only recently become accessible to partial-hand amputees. The commercially available i-limb digits (Touch Bionics Inc.) and Vincentpartial (Vincent Systems GmbH) systems are externally-powered prostheses with independently functioning digits built into a custom socket that accommodates the patient’s remaining hand [6]. These devices offer a wide range of functional hand grasps not previously available to partial-hand amputees.

Though force sensitive resistors can be used, these devices are typically controlled using conventional myoelectric strategies, where an estimate of the surface electromyogram (EMG) amplitude controls the speed of an actuated joint [6, 12, 13]. The controlling muscles can be located in the hand itself (intrinsic hand muscles) or in the forearm (extrinsic hand muscles). Controlling the prosthesis using intrinsic hand muscles has the advantage of providing finger control independent of wrist motion [6] but it is challenging to obtain separate signals from these small, closely spaced muscles. Thus, the number of available independent control sites is limited.

Alternatively, extrinsic hand muscles, which remain mostly intact in partial-hand amputees, may be used for conventional myoelectric prosthesis control; however, doing so compromises normal wrist movement and thus limits hand function. Though widely clinically accepted, conventional myoelectric control, using either the intrinsic or extrinsic hand muscles, is limited to the control of one or two degrees of freedom [12, 14] and mode switching through co-contraction is required to control additional degrees of freedom.

Pattern recognition–based myoelectric control of upper-limb prostheses offers a promising alternative for control of powered partial-hand prostheses. It has the potential to restore control of more degrees of freedom than conventional myoelectric control because it can combine and utilize information across multiple EMG signal sources [15–17]. Previous research has shown that pattern recognition techniques, using EMG from the extrinsic hand muscles of transradial amputees, can control multiple hand grasps in real time with high accuracy [18, 19].

Unlike individuals with higher-level amputations, partial-hand amputees may possess residual intrinsic hand muscles that may provide additional information-rich EMG data for improved prosthetic control. Li et al. used EMG from intact and residual arms of unilateral transradial amputees and found that the intact arms were more successful at performing grasps [18]. Although this was presumably due to EMG data from intrinsic hand muscles, the contribution of intrinsic muscle EMG to the improvement of offline accuracies and online control was not explicitly evaluated. Since intrinsic and extrinsic muscles play different roles in control of the intact hand [32] incorporating EMG information from intrinsic muscles may improve prosthetic hand control.

Most partial-hand amputees have a functional and useful wrist. Preservation of wrist range of motion allows positioning of the hand in space, greatly adding to overall hand function [3, 5, 20]. Thus, a clinically successful pattern recognition control system for a partial-hand prosthesis must provide high performance while enabling use of the wrist. Studies have shown that variations in arm position substantially impact the ability to classify hand grasps in higher level amputees [21, 22] and have suggested training with multiple limb positions or multi-stage classification to improve performance. Recent studies with non-amputees show that static and dynamic wrist motion adversely affect pattern recognition performance in offline studies [23, 24], and training with multiple wrist positions improves performance. As the impact of wrist posture in amputee subjects cannot be inferred from non-amputee studies [25], it is not clear if and to what extent wrist position will affect performance in partial-hand amputees.

In this paper we quantify the contribution of EMG data from extrinsic and intrinsic hand muscles to pattern recognition–based control of hand grasps and finger movements in amputees and non-amputees. We build upon previous results [23] and quantify the effect of wrist motion on pattern recognition control in partial-hand amputees and the effect of training with multiple wrist positions in online studies.

II. Methods

Nine non-amputees (5 males, 4 females, mean age = 23 ± 2 years) with no known neurological or physical deficits performed the experiments described in the first part of the offline studies. Seven non-amputees (4 males, 3 females, mean age = 23 ± 2 years) performed the experiments described in the second part of the offline studies. Four individuals with partial-hand amputations participated in both parts of the offline study [Table I]. None of the amputees routinely used a prosthesis, and all had good residual wrist mobility. All subjects gave written consent, and experiments were performed at the Rehabilitation Institution of Chicago under an approved Northwestern University Institutional Review Board (IRB) protocol. All subjects were naïve to pattern recognition control with the exception of two non-amputee subjects.

TABLE I.

DEMOGRAPHIC INFORMATION OF THE FOUR PARTIAL-HAND SUBJECTS

Subject	Gender, Age	Partially-Amputated Hand	Time since Amputation
PH1	M, 56	Both	3 years
PH2	F, 38	Left	1 year
PH3	M, 47	Right	10 years
PH4	F, 26	Left	7 months

For all subjects, 9 self-adhesive bipolar surface Ag/AgCl EMG electrodes (Bio-Medical Instruments) were evenly spaced around the dominant forearm with an inter-electrode distance of 2.5cm: 5 electrodes on the proximal forearm, 2–3cm distal to the elbow and 4 electrodes on the distal forearm, 2–3 cm proximal to the wrist (Fig. 1. (a)–(d)). For non-amputees, 12 bipolar electrodes were placed on the dominant hand (7 electrodes on the palmar side and 5 electrodes on the dorsal side) to provide a dense spatial sampling of intrinsic muscle activity (Fig. 1(e)–(f)). For amputees, due to limited surface area on the residual hand, only 4 electrodes were placed on the hand: 2 electrodes on the palmar side and 2 electrodes on the dorsal side (Fig. 1(a)–(d)). For one amputee subject (PH4) only one electrode was placed on the dorsal side. For all subjects, ground electrodes were placed on the olecranon of the elbow. Subjects were seated facing a computer screen, with their elbow at a 90 degree angle.

Fig. 1.

(a) and (b) Anterior and posterior view, respectively, of the right forearm and hand of a partial-hand amputee (PH1). (c) and (d) Anterior and posterior view, respectively, of the left forearm and hand of PH1. (e) and (f) Anterior and posterior view, respectively, of the hand of a non-amputee subject.

A. Familiarization

All pattern recognition–naïve subjects received 2 hours of practice with pattern recognition control in a virtual environment on a separate day prior to performing the experiments. Custom computer software called Control Algorithms for Prosthetics Systems (CAPS) [17] prompted users to perform functional hand grasps, an open hand posture, and a rest posture. Data were collected and used to train a linear discriminant analysis classifier for online classification. Subjects were then allowed to practice performing each grasp in a virtual environment, where a virtual hand provided continuous visual feedback of the classifier’s output.

B. Procedure

I. Classification of Hand Grasps and Finger Motions

CAPS software was used to prompt subjects to perform 5 functional hand grasps (key grip, chuck grip, power grip, fine pinch grip, tool grip), an open hand posture, and a rest posture. Subjects were also asked to perform 12 individual finger motions: thumb adduction, abduction, flexion, and extension, index finger flexion and extension, middle finger flexion and extension, ring finger flexion and extension, pinky finger flexion and extension. When flexing or extending each digit, subjects were asked to follow a visual prompt where the metacarpophalangeal (MCP) and proximal interphalangeal (PIP) joints were both flexed or both extended. Each grasp and finger motion was held for 3 seconds (s) and repeated 10 times while subjects held their wrist in a neutral position, for a total of 30s of data collection per hand or finger motion. Subjects rested for 3s between each repetition. To avoid fatigue, subjects were allowed 2–5 minutes of rest between trials, where a trial consisted of 15 hand or finger motions. All data were aligned with the stimulus image.

II. Classification of Hand Grasps with Wrist Motion

Subjects were prompted to perform 2 functional hand-grasps (chuck and key grip), an open hand posture, and a rest posture. Each hand posture was held for 3s and repeated 10 times while subjects held their wrist in a neutral position. Subjects rested for 3s between each repetition. This protocol was repeated while the subjects held their wrist comfortably in the following 6 static positions: flexed, extended, pronated, supinated, abducted, and adducted. A total of 210 s of data were collected for each hand posture over all 7 static wrist positions.

To ensure that non-amputee subjects maintained the same pinch force throughout each grasp, subjects received visual feedback of pinch forces produced during chuck and key grip using an electronic pinch gauge (12-0023, Fabrication Enterprises). Subjects were required to maintain a grasp force that was 15–20% of their maximal voluntary grasp force made in a neutral wrist position; this force level was comfortable for all grasps in all wrist positions. To avoid fatigue, subjects were allowed 2–5 minute rests between trials, where a trial consisted of 5 repetitions of the four hand postures in one wrist position.

Subjects were also asked to make combined movements of the wrist and hand. They performed chuck grip and the open hand posture while simultaneously moving their wrist from a neutral position to one of the following positions: fully flexed, extended, pronated, supinated, abducted, and adducted. Subjects were also explicitly asked to move their wrist from neutral to each of these 6 positions with relaxed hand and fingers. Subjects were asked to move their wrist at a medium, comfortable, and constant velocity. Each combined movement was performed within 3s and repeated 10 times. A total of 180s of data were collected over all 6 dynamic wrist motions. Constraints were placed on pinch forces using feedback as previously described. Amputee subjects were not provided with any visual feedback on pinch forces.

C. EMG Signal Processing

EMG signals were acquired using a custom built EMG amplifier with a gain of 2000x for each channel. Data were digitally sampled at 1000 Hz using a custom-built A/D converter based on a TI AD1298 bioamplifier chip and band pass-filtered (30–350Hz) with a Type 1, 8^th order Chebyshev filter.

D. Data Analysis

Offline analyses were performed using MATLAB 2012a software (The Mathworks, Natick, MA, USA). Data were segmented into 200ms windows with a 20ms frame increment [26]. A combination of four EMG time-domain features (mean absolute value, number of zero-crossings, waveform length, and number of slope sign changes) and six coefficients of a 6^th order autoregressive model were extracted from each EMG channel. Preliminary studies showed that the linear discriminant analysis (LDA) classifier performed comparably to a support vector machine and Gaussian mixture model classifier. Thus the LDA classifier was used for all analyses presented here. The combination of LDA classifier and time domain and autoregressive features has been used extensively and shown to result in high classification performance [15, 27]. The LDA classifier was trained using (1) only extrinsic muscle EMG data, (2) only intrinsic muscle EMG data, or (3) a combination of all extrinsic and intrinsic muscle EMG data. Data were divided evenly into training data sets and testing data sets and each classifier was evaluated using two-fold cross-validation with these sets.

I. Classification of Hand Grasps and Finger Motions

A classifier was trained and tested using data from (1) all hand grasps, the open hand posture, and the rest posture (2) all finger motions and the rest posture, or (3) all hand grasps, all finger motions, the open hand posture, and the rest posture. Classification error, defined as the number of incorrectly classified samples divided by the total number of test samples and averaged across hand grasps, finger motions, or a combination of hand and finger motions was used to assess classifier performance. To determine the effect of each EMG channel on classifier performance, we used an electrode selection algorithm based on the sequential forward searching (SFS) method described by [28].

II. Classification of Hand Grasps with Wrist Motion

A classifier was trained using data from the neutral wrist position and tested on data from (1) a neutral wrist position, (2) all static wrist positions, or (3) a neutral wrist position and all dynamic wrist motions. A classifier was also (4) trained and tested on data from all static wrist positions or (5) trained and tested on data from the neutral wrist position and all dynamic wrist motions.

E. Statistical Analysis

To analyze the effect of muscle set on the classification of hand grasps and finger motions, three, one–way repeated measures analysis of variance (ANOVA) tests were performed with subject as a random effect and muscle set as a fixed effect. Post-hoc comparisons were made using a Bonferonni correction factor to determine significance. To determine the effect of static wrist positions or dynamic wrist motions on classifier performance, a two–way repeated measures ANOVA was performed with subject as a random effect, and muscle set and the training/testing paradigm as fixed effects. Post-hoc comparisons were also made using a Bonferonni correction factor to determine significance. Analyses were performed separately for amputees and non-amputees using Minitab 16.2.4 (Minitab Inc. PA, USA), with a significance level set at 0.05.

F. Online Study

We completed an online study to quantify the benefits of combining extrinsic and intrinsic muscle EMG and training with multiple wrist positions on real-time pattern recognition control of hand grasps. Six non-amputees (4 males, 2 females, mean age = 25 ± 2 years, who did not take part in the offline studies) and three partial-hand amputees (PH2, PH3, PH4) participated in this study. Electrode placements for amputees, EMG signal processing and familiarization for naïve subjects were as described for offline studies. For non-amputees, only 4 channels of EMG were recorded from intrinsic muscles at the locations depicted in Fig. 1(a)–(b).

Subjects performed 2 hand grasps (chuck grip, key grip), an open hand posture and a rest posture, all in a neutral wrist position. Each posture was held for 4s and repeated four times for a total of 16s of data for each hand posture. Because training with multiple static wrist positions and dynamic wrist motions is time-consuming, we used a hybrid static/dynamic training protocol. Subjects were asked to perform each hand posture while holding the wrist in a comfortable flexed position for 2s after which they were instructed to move the wrist to a comfortable extended position over a period of 2s while maintaining the grasp. They then held the posture in that extended position for 2s then moved the wrist back to a comfortable flexed position over a period of 2s. This hybrid protocol lasted a total of 8s. Subjects repeated this sequence of movements moving the wrist from (1) flexed to extended to flexed (2) extended to flexed to extended (3) supinated to pronated to supinated (4) pronated to supinated to pronated (5) abducted to adducted to abducted and (6) adducted to abducted to adducted. In total, 48s of the hybrid wrist training protocol data were collected for each grasp. No visual pinch force feedback was provided to subjects during data collection.

Four separate classifiers were trained with data collected from either (1) extrinsic EMG signals in a neutral wrist position, (2) extrinsic EMG signals with the hybrid wrist motion training protocol, (3) extrinsic and intrinsic EMG signals in a neutral wrist position, or (4) extrinsic and intrinsic EMG signals with the hybrid wrist motion training protocol. A 5-vote majority vote was implemented. The offline performance of these classifiers was also evaluated using two-fold cross validation and random sub-selection of 16s of training data.

Motion tests as described by Kuiken et al. using a virtual prosthesis were performed immediately after classifier training [17]. Subjects were instructed to follow visual prompts for each hand grasp. The virtual hand allowed subjects to observe the real-time outputs of the classifier. A hand grasp could only be selected when the hand was fully open. Thus, if the initial hand grasp selected was incorrect, the subject would have to fully open the virtual hand and try again.

One of the four classifiers was randomly assigned to a trial and used for online control. Within each trial, each of the three hand motions was randomly presented twice. This was repeated while the subject held their wrist in one of 7 randomly assigned static wrist positions. These four trials were repeated twice (i.e., 8 trials) for a total of 84 hand motions per classifier. Subjects were blinded to the control method. The subset of channels used during each condition was the same as that used to train the classifier.

Four metrics were used to quantify real-time prosthesis control performance. The motion selection time was the time taken to correctly select a target movement. The motion completion time was the time taken to successfully complete a movement through the full range of motion, measured as the time from the onset of movement to the completion of the intended movement. Fifty accumulated correct classifications were required for a motion completion. The minimum possible time to complete any motion was thus normalized to 1s, corresponding to fifty consecutive correct classifications with a new classification occurring every 20ms. The motion completion rate was defined as the percentage of successfully completed motions within a time limit of 10s. Dynamic efficiency was defined as the percentage of correct classifications for each trial. Thus misclassifications would increase motion selection and completion times and decrease motion completion rates and dynamic efficiency. To determine the effect of classifier training paradigm on online performance, a two–way repeated measures ANOVA was performed with subject as a random effect, and muscle set and wrist position training paradigm as fixed effects.

III. Results

A. Classification of Hand Grasps and Finger Motions

For all subjects, classifiers trained with a combination of extrinsic and intrinsic muscle EMG data performed better than classifiers trained with extrinsic muscle EMG data alone (Fig. 2). These results were significant for non-amputees for hand grasps (p<0.001), finger motions (p<0.05), and all hand grasps and finger motions (p<0.001). For amputees, these same trends were observed but were not statistically significant for hand grasps (p=0.60), finger motions (p=0.7), or all hand grasps and finger motions (p=0.15). A classifier trained with intrinsic muscle EMG alone was worse at distinguishing between individual finger motions than one trained with extrinsic muscle EMG for both non-amputees (p<0.05) and amputees (p<0.01).

Fig. 2.

Average classification errors across all non-amputees and amputees. HG: all hand grasps (7 classes), FM: all finger motions (13 classes), HGFM: all hand grasps and finger motions (19 classes). (*) represents p<0.05; (**) represents p<0.01; (***) represents p<0.001. Error bars represent standard error.

Fig. 3 shows confusion matrices for a representative subject (PH1, right partial-hand amputation). A classifier trained with intrinsic muscle EMG alone was worse at distinguishing between flexion and extension of digits two through five than a classifier trained with extrinsic muscle EMG alone; however, it was able to decode individual thumb movements with high accuracy. This result was observed across all subjects.

Fig. 3.

Confusion matrices showing the error distribution for a partial-hand amputee (PH1, right hand). All classifiers were trained to distinguish between all 19 motion classes of hand and finger motions. (a) Classifier trained with only extrinsic muscle EMG. (b) Classifier trained with only intrinsic muscle EMG. (c) Classifier trained with both intrinsic and extrinsic muscle EMG. The symbols represent the following hand grasps–rest posture (nm), hand open (ho), chuck grip (cg), key grip (kg), power grip (pg), fine pinch grip (fp), tool grip (tg); and finger motions– thumb abduction (ab), adduction (ad), flexion (f1), and extension (e1); index finger flexion (f2) and extension (e2); middle finger flexion (f3) and extension (e3); ring finger flexion (f4) and extension (e4); pinky finger flexion (f5) and extension (e5).

Fig. 4 shows the relationship between the number of electrodes and classification accuracy. The first 9 channels are all extrinsic muscle channels. Accuracy increases considerably at the beginning of the curve and eventually plateaus; however, the additional of intrinsic EMG data (to the right of the vertical line) allows higher accuracies to be achieved.

Fig. 4.

Classification accuracy for 19 motion classes (all finger motions and hand grasps) as a function of number of EMG channels. For both subject groups all 9 extrinsic channels were analyzed (left of the dashed line). Each available intrinsic channel was then added in order of most to least important (right of the dashed line). Error bars represent standard error.

Fig. 5 illustrates the relative importance of each intrinsic EMG channel location in non-amputees. Each circle represents the approximate location of one bipolar electrode; the intensity of the circle color is proportional to the importance of that particular intrinsic EMG channel. This was determined by how often that channel was one of the six channels that contributed the most to classification accuracy as determined by applying the SFS algorithm to the full set of all 9 extrinsic and 12 intrinsic muscle EMG channels. For hand grasps, finger motions, or both hand grasps and finger motions, electrodes placed over the thenar eminence and the palmar and dorsal region between the first and second metacarpals (indicated by the arrows in Fig. 5) were consistently the most important intrinsic EMG channels.

Fig. 5.

Relative importance of intrinsic muscle electrode locations. The classifier was trained using a combination of all extrinsic and intrinsic EMG channels. The figure depicts how often an intrinsic EMG channel was one of the six channels that contributed the most to classification accuracy for all non-amputees. Each circle represents the approximate location of one bipolar electrode. White circles correspond to no occurrence and the deepest red circle corresponds to 7 occurrences. Arrows indicate channels that were consistently the most important intrinsic EMG channels.

B. Classification of Hand Grasps with Wrist Motion

I. Effect of Static Wrist Positions

Training and testing the classifier using data collected with a neutral wrist position resulted in low average error rates of less than 2% and 7% for non-amputees and amputees, respectively. Error was significantly higher for a classifier trained with data collected in a neutral wrist position and tested with data from all static wrist positions for both non-amputees (p<0.001) and amputees (p<0.001). This classifier performed worse when trained with extrinsic EMG data alone than when trained with intrinsic or combined intrinsic and extrinsic EMG data for both subject groups as implied by the significant interaction term between muscle set and training/testing paradigm for non-amputees (p<0.001) and amputees (p<0.01).

For non-amputees, there was no significant difference between the performance of a classifier trained and tested with data from all static wrist positions and a classifier trained and tested with data collected in a neutral wrist position (p=0.46, Fig. 6). However, for amputees, a classifier trained and tested using data from all wrist positions did not perform as well as one trained and tested with data acquired for a neutral wrist position (p<0.05) though it performed better than a classifier trained in a neutral wrist position and tested with data from all wrist positions (p<0.001).

Fig. 6.

Effect of varying static wrist positions on classification error. The three classifier training/testing paradigms were (A) train and test in a neutral wrist position (B) train in a neutral wrist position and test in all wrist positions (C) train and test in all wrist positions. (*) represents p<0.05; (***) represents p<0.001. Error bars represent standard error.

II. Effect of Dynamic Wrist Motions

Error was significantly greater when the classifier was trained with EMG data collected in a neutral wrist position and tested with data from all dynamic wrist motions than when the classifier was trained and tested with data from a neutral wrist position for both subject groups (p<0.001, Fig.7). This classifier performed worse when trained with extrinsic EMG data alone than when trained with intrinsic or combined intrinsic and extrinsic EMG data for both subject groups as implied by the significant interaction term between muscle set and training/testing paradigm for non-amputees (p<0.001) and amputees (p<0.05).

Fig. 7.

Effect of dynamic wrist motions on classification error. The three classifier training/testing paradigms were (A) train and test in a neutral wrist position (B) train in a neutral wrist position and test with all dynamic wrist motions (C) train and test with all dynamic wrist motions. (***) represents p<0.001. Error bars represent standard error.

For non-amputees, there was no significant difference in performance between a classifier trained and tested with data from all dynamic wrist motions and a classifier trained and tested with data collected in a neutral wrist position (p=0.36, Fig. 6). However, for amputees, a classifier trained with data from all dynamic wrist motions did not perform as well as one trained and tested in a neutral wrist position (p<0.05) though it performed better than a classifier trained in a neutral wrist position and tested in all wrist positions (p<0.001).

C. Online Studies

For the online studies we used a modified training protocol that was less time-consuming, and more clinically feasible. Fig. 8 demonstrates the benefits of using additional intrinsic muscle channels and including wrist motion with this modified protocol. For non-amputees and amputees, the hybrid training paradigm resulted in lower offline classification errors than the neutral wrist training paradigm (p<0.001 and p<0.05, respectively). The combination of extrinsic and intrinsic muscles resulted in lower classification errors than extrinsic muscles alone for both non-amputees and amputees (p<0.001 and p<0.05, respectively). There was no significant interaction term for non-amputees and amputees (p=0.2 and p=0.09, respectively).

Fig. 8.

Effect of classifier training paradigm on offline classification error. All classifiers were tested with data collected from different static wrist positions and wrist motions. Each classifier was trained with (1) data from the extrinsic muscles alone or with data from the extrinsic and intrinsic muscles and (2) data collected from a neutral wrist position or with data collected from hybrid static and dynamic wrist motions as indicated by the legend. (*) represents p<0.05; (***) represents p<0.001. Error bars represent standard error.

I. Motion Completion Rate

For non-amputees and amputees, the combination of extrinsic and intrinsic muscles resulted in higher completion rates than extrinsic muscles alone (p<0.001, p<0.05, respectively). For non-amputees, the hybrid training paradigm resulted in higher completion rates than the neutral wrist training paradigm (p<0.001, Fig. 9(a)). The same trends were observed for amputees and approached significance (p=0.06). There was no significant interaction term (p=0.74).

Fig. 9.

Effect of classifier training paradigm on online performance. (a) completion rate (b) dynamic efficiency across successfully completed trials (c) dynamic efficiency across all attempted trials (d) completion time across all successfully completed trials, (e) selection time across all successfully completed trials (f) selection time across all attempted trials.

II. Dynamic Efficiency

For successful trials completed by non-amputees, the hybrid training paradigm resulted in higher dynamic efficiencies than the neutral wrist training paradigm (p<0.01). The combination of extrinsic and intrinsic muscles resulted in higher dynamic efficiencies than extrinsic muscles alone (p<0.001). There was no significant interaction term (p=0.31, Fig. 9(b)). This was also true for the dynamic efficiency across all trials attempted by non-amputee subjects. For successful trials completed by amputees, there was no significant main effect of wrist position training paradigm (p=0.98) or muscle set (p=0.80) on dynamic efficiency. However, across all trials attempted by amputees, the hybrid training paradigm performed better than the neutral wrist training paradigm, (p<0.01). The combination of extrinsic and intrinsic muscles resulted in higher completion rates than extrinsic muscles alone (p<0.05). There was no significant interaction term (p=0.78, Fig. 9(c)).

III. Motion Completion Time

The motion completion time for a given movement was counted if the movement was successfully completed within 10s. For non-amputees, the hybrid training paradigm resulted in lower completion times than the neutral wrist training paradigm (p<0.05). The combination of extrinsic and intrinsic muscles resulted in lower completion times than extrinsic muscles alone (p<0.01). There was no significant interaction term (p=0.30) between the two factors. For amputees, there was no significant main effect of wrist position training paradigm (p=0.74) or muscle set (p=0.85) on completion time (Fig. 9(d)).

IV. Motion Selection Time

For successful trials completed by non-amputees, there was no significant main effect of wrist position training paradigm on selection time (p=0.61). The combination of extrinsic and intrinsic muscles resulted in lower selection times than extrinsic muscles alone (p<0.05). There was no significant interaction term (p=0.78, Fig. 9(e)). However, across all trials attempted by non-amputees, the hybrid training paradigm performed better than the neutral wrist training paradigm (p<0.001) and the combination of extrinsic and intrinsic muscles resulted in lower selection times than extrinsic muscles alone (p<0.001, Fig. 9(f)). For successful trials completed by amputees, there was no significant main effect of wrist position training paradigm on selection time (p=0.74). The combination of extrinsic and intrinsic muscles resulted in higher selection times than extrinsic muscles alone (p<0.05). There was no significant interaction term (p=0.89). However, across all trials attempted by amputees, the hybrid training paradigm performed better than the neutral wrist training paradigm approaching significance (p=0.06). The combination of extrinsic and intrinsic muscles resulted in lower selection times than extrinsic muscles alone though this was not statistically significant (p=0.2, Fig. 9(f)).

IV. Discussion

Pattern recognition is a promising control method that circumvents many of the disadvantages of conventional myoelectric control. In this study we demonstrated that combining extrinsic and intrinsic hand muscle EMG data and incorporating multiple wrist positions during training improves pattern recognition control in offline and online studies. These results are promising for the clinical application of pattern recognition to partial-hand prosthesis control.

We found that even after classifier performance seemed to have plateaued when trained with EMG from 6–8 extrinsic muscle channels, the addition of intrinsic muscle EMG further increased classifier performance. Though both extrinsic and intrinsic hand muscles in the intact hand interact and contribute to modulate fingertip forces, they can be functionally differentiated by their roles in grasping and manipulation. It is thought that the intrinsic muscles, which are generally smaller in size than the extrinsic muscles, play a key role in hand grasps (particularly chuck grip, fine pinch grip and key grip) [29, 30], have a greater impact on fingertip direction than force magnitude, and are important for subtle manipulation of finger movements, whereas the extrinsic muscles provide stability of the joints [31, 32]. Our results indicate that intrinsic muscle EMG provides information that, though not sufficient for control, complements extrinsic muscle EMG data to improve classification performance, perhaps because intrinsic muscles provide more precise control of finger position and direction than extrinsic muscles.

A closer examination of the confusion matrices shows that the higher classification errors of a classifier trained with only intrinsic muscle EMG is mostly due to confusion between flexion and extension of the second through fifth digits. These observations are expected given that finger flexion and extension forces are particularly dependent on the extrinsic muscles [33]. The intrinsic thumb muscles are an exception and provide enough information to discriminate between individual thumb motions. Though there is no specific extensor of the thumb in the thenar compartment, it is possible that EMG from the both the abductor pollicis brevis and adductor pollicis contributed to the control of thumb extension due to their attachment to the thumb extensor mechanism.

Given that the passive biomechanics and interconnections of soft tissues within the hand significantly couple finger movements [34], and that common neural input across the finger muscles limits the extent to which fingers can move independently [35–37], a classifier trained with only extrinsic EMG is still able to equally discriminate between individual finger motions with accuracies of 91% for non-amputees. Other studies have also shown that extrinsic muscle EMG data is sufficient to discriminate between individual finger motions [27, 38, 39].

Though it may seem that the extrinsic muscles can provide relatively good control, their performance significantly deteriorates with changes in wrist position, with offline error rates up to 28% and 35% in non-amputees and amputees, respectively, for control of only 4 hand motion classes. Not surprisingly, extrinsic muscle EMG was more sensitive to static wrist position and dynamic wrist motion than intrinsic muscle EMG. Here, we focused on the effect of wrist motion on two grasps (chuck and key grip) because these grasps are the most common grasps used in activities of daily living [40]. Moreover, controlling 2–3 grasp patterns and preserving residual wrist motion without the cumbersome switching methods required for conventional myoelectric control would be a significant improvement over current partial-hand prosthesis control methods.

We tested two methods to mitigate the wrist position effect: (1) using both extrinsic and intrinsic muscle EMG data and (2) training with a hybrid static/dynamic wrist motion protocol. In online studies, using a classifier trained with extrinsic muscle EMG in a neutral wrist position resulted in completion rates of 73% for amputees and 87% for non-amputees. Adding information from the intrinsic muscles increase the completion rates to 84% and 98%, respectively. Thus an improvement in control was achieved without the need for the extensive multi-wrist position training. It is worthwhile to note that the intrinsic muscles may be damaged or absent in some partial-hand amputees thus rendering them unsuitable as an EMG data source. For such cases, we show that classifier training with extrinsic muscle EMG and a hybrid wrist motion training protocol instead of a neutral wrist position increased completion rates from 73% to 82% for amputees and from 87% to 98% for non-amputees. Combining both intrinsic muscle EMG data and a hybrid wrist motion training protocol ultimately achieved completion rates of up to 96% in amputees. For amputees, we found no significant effect of wrist position training paradigm or muscle set on dynamic efficiency or completion time, in fact combining extrinsic and intrinsic muscle EMG increased selection times for successful trials. These results are most likely confounded by the completion rates for each training condition since we do see the expected effects of wrist position training paradigm and muscle set when we consider all attempted trials instead of only successful trials.

A virtual arm was used in these experiments since a physical partial-hand prosthesis was not available. There are several important differences between virtual and physical prostheses including differences in visual feedback and the effect of prosthesis weight, particularly on the intrinsic muscles. It is possible that loading from the prosthesis may make intrinsic muscle EMG too unstable to allow for good control, in which case a control strategy that uses extrinsic muscle EMG and a hybrid wrist motion training paradigm may outperform all other control methods. Further studies to evaluate control of powered, multifunctional partial-hand prostheses are warranted.

V. Conclusion

This research study examined the (i) contribution of intrinsic hand EMG data to pattern recognition–based classification of hand grasps and finger motions and (ii) the effect of static wrist position and dynamic wrist motion on the classification accuracy of a pattern recognition control system. We found that intrinsic muscles provide valuable EMG data that enables a pattern recognition system to better discriminate between up to 19 different hand grasp patterns and individual finger motions. Real-time studies demonstrated that wrist motion significantly decreased control system robustness and that this decrease was mitigated by training the pattern recognition system with multiple static and dynamic wrist motions and with both extrinsic and intrinsic muscle EMG. Based on the results of this study, the clinical implementation of a pattern recognition system capable of controlling multiple hand grasps while allowing use of the wrist appears promising for partial-hand amputees.

Biographies

Adenike A. Adewuyi (S’13) received her B.S. degree in Biomedical Sciences and Engineering from Harvard University, Cambridge, MA in 2009. She is currently with the Medical Scientist Training Program at Northwestern University where she is pursuing both her M.D. degree at the Feinberg School of Medicine, Chicago, IL and her Ph.D. degree at the Center for Bionic Medicine, Rehabilitation Institute of Chicago.

Her research interests include neural control of movement, biological signal processing, neurophysiology and control of powered prostheses.

Levi J. Hargrove (S’05-M’08) received his B.Sc., M.Sc. and PhD degrees in electrical engineering from the University of New Brunswick (UNB), Fredericton, NB, Canada, in 2003, 2005, and 2007 respectively.

He joined the Center for Bionic Medicine at the Rehabilitation Institute of Chicago in 2008. He is a Research Assistant Professor in the Department of Physical Medicine and Rehabilitation (PM&R) and Biomedical Engineering, Northwestern University, Evanston, IL, USA. His research interests include pattern recognition, biological signal processing, and myoelectric control of powered prostheses.

Dr. Hargrove is a member of the Association of Professional Engineers and Geoscientists of New Brunswick.

Todd A. Kuiken (M’99–SM’ 07) received his B.S. degree in biomedical engineering from Duke University, Durham, NC, in 1983; his Ph.D. degree in biomedical engineering from Northwestern University, Evanston, IL, in 1989; and his M.D. degree from the Northwestern University Feinberg School of Medicine, Chicago, IL, in 1990. He completed a residency in physical medicine and rehabilitation at the Rehabilitation Institute of Chicago (RIC) and Northwestern University Medical School, Chicago, IL, in 1995.

Dr. Kuiken is currently Director of the Center for Bionic Medicine and of Amputee Services at the RIC, Chicago, IL, and Professor in the Departments of Physical Medicine and Rehabilitation, Biomedical Engineering, and Surgery, Northwestern University, Evanston, IL