Extrinsic Finger and Thumb Muscles Command a Virtual Hand to Allow Individual Finger and Grasp Control

Abstract

Fine-wire intramuscular electrodes were used to obtain EMG signals from six extrinsic hand muscles associated with the thumb, index, and middle fingers. Subjects’ EMG activity was used to control a virtual three-DOF hand as they conformed the hand to a sequence of hand postures testing two controllers: direct EMG control and pattern recognition control. Subjects tested two conditions using each controller: starting the hand from a pre-defined neutral posture before each new posture and starting the hand from the previous posture in the sequence. Subjects demonstrated their ability to simultaneously, yet individually, move all three DOFs during the direct EMG control trials, however results showed subjects did not often utilize this feature. Performance metrics such as failure rate and completion time showed no significant difference between the two controllers.

I. INTRODUCTION

Traditional myoelectric prosthetic hands have a single-degree of freedom (DOF), which transradial amputee users control with electromyogram (EMG) signals from their residual forearm muscles [1, 2]. These hands are often comprised of rigid curved digits connected to a motor that moves the fingers and opposed thumb either towards or away from one another. Direct myoelectric control generally uses the difference in the magnitude of the two EMG signals to proportionally control the velocity of a device’s DOF. Despite having only velocity control, these hands have been used for decades and provide functionality that is otherwise lost due to amputation.

Several research groups have recently developed robotic hands that more closely mimic the form and function of human hands [3–6]. Commercial hands are now available that use multiple motors to individually move fingers, allowing for more complex motions and hand postures that could provide more functionality. However, novel control methods need to be developed in order to fully utilize these hands’ capabilities. Initial work is only just beginning that explores intuitive methods for controlling multi-DOF hands using physiologically realistic neural control signals [7].

Direct myoelectric control uses two individually controllable EMG sites to command each DOF, but obtaining more than two sites is very difficult because the extrinsic hand muscles in the forearm are small, close together, and vary in depth from the skin’s surface. This leads to difficulty in distinguishing individual muscle activation using surface EMG sensors. Many groups have devised pattern recognition controllers [8–14] and principal component analysis [15, 16] to map between user’s intent and prosthesis motion. While pattern recognition control has shown promising results, it is currently limited to controlling a single classified pattern (e.g. one hand posture) at a time.

Implantable wireless EMG sensors have been developed that can provide measurements of muscle activity from individual extrinsic hand muscles within the forearm [17–19]. This allows additional information to be acquired from the user and could potentially provide simultaneous control of multiple DOFs [20].

Additionally, Smith and Hargrove have demonstrated that intramuscular EMG measurements decreased classifier error, when compared to surface EMG signals, for parallel pattern recognition classifiers, but not for a single classifier conditions [21]. Kamavuako et al. have also shown that the inclusion of intramuscular and surface EMG measurements can improve performance metrics when controlling a four-function, two-DOF hand, compared to using surface EMG measurements alone [22]. They have also shown that intramuscular EMG signals could be mapped to grasping force, which can further aid in prosthesis control [23].

Single-digit motions in healthy individual has been demonstrated [24, 25], which is due in part to extrinsic finger muscles. Extensor digitorum communis (EDC), flexor digitorum profundus (FDP), and flexor digitorum superficialis (FDS) are extrinsic muscles that have individual finger and tendon compartments and have been shown to be physically and functionally distinct [26–31]. Regardless, these individual digit motions are the coordinated effort of multiple muscles and usually result in both force being exerted by adjacent digits (force enslavement) [32, 33] and activity emitted from adjacent muscle compartments (spillover recruitment) [30, 34–36]. In controlling an artificial hand, the spillover recruitment, or co-activity, may cause an issue, however intramuscular electrodes used in these finger muscle compartments would provide more localized activity measurements [19, 37] and have been demonstrated to measure from these muscles free of EMG crosstalk [38].

In addition, abductor pollicis longus (APB), extensor pollicis brevis (EPB), extensor pollicis longus (EPL), and flexor pollicis longus (FPL) are the four extrinsic thumb muscles and their activity could provide additional information to command a robotic thumb. The three multi-tendon extrinsic finger muscles and the four extrinsic thumb muscles have previously been studied for their ability to activate individually from one another [39, 40].

This work tests the feasibility of using the extrinsic thumb muscles and finger muscle compartments to command individual digits on a virtual hand to mimic grasping hand postures. Furthermore, subjects tested two controllers, direct myoelectric control and a pattern recognition classifier, and the results from each were compared.

II. Methods

Seven able-bodied subjects gave informed written consent before participating in this study, which was approved in advance by the Northwestern University Institutional Review Board. These subjects had no history of neurological or physical ailments and were right-hand dominant.

Intramuscular nickel-alloy bi-polar fine-wire electrodes (ϕ0.002” Stablohm 800A with HPN insulation, California Fine Wire Co., Grover Beach, CA) were inserted using 27 gage hypodermic needles and standard intramuscular electrode insertion techniques [37]. Electrodes were placed into the flexor pollicis longus (FPL) and extensor pollicis longus (EPL) muscles of the thumb, and the first two compartments of the extensor digitorum communis (EDC1, EDC2) and flexor digitorum profundus (FDP1, FDP2) muscles, which are associated with the index and middle fingers, respectively. Each muscle and muscle compartment was initially located by palpation and ultrasound. During electrode insertion, EMG activity was observed on a monitor and played through a loudspeaker to minimize signal detection from adjacent muscles and compartments. Once the desired muscle or compartment was located with the electrode, the needle was withdrawn and the wires shortened to the minimum length required for connection to the data acquisition system.

After insertion, each electrode’s location was verified by electrical stimulation using a constant-current stimulator (Digitimer Ltd. Model DS7A, Hertfordshire, England) and observing the resultant motion during the muscle’s stimulation. EMG signals were measured using a Delsys Bagnoli-16 system (Delsys Inc., Boston, MA) connected to a PC running custom software (described below) and sampling at 1,000 Hz. The Delsys system has hardware band-pass filters set at 20–450 Hz. Intramuscular electrodes were used to access EMG activity from individual muscle and muscle compartments, and not to measure higher frequency EMG content, which would not have improved the velocity controllers used in this study. Subjects placed their hand and wrist in a standard ball splint, after electrodes were placed, to allow for isometric muscle contractions and to standardize hand position across subjects.

These six muscles provided EMG signals to command the three DOFs in a virtual hand. Our previous work demonstrated that intramuscular electrodes residing in these muscles and compartments were capable of measuring and quantifying individual EMG activities [38, 40]. Those findings provided the foundation for this study.

A. CAPS and the TAC Test

CAPS is custom software developed by the Center for Bionic Medicine to acquire EMG signals, test prosthesis control systems, and send output commands to either a virtual or physical prosthesis [10]. CAPS was used in this experiment to measure EMG from extrinsic finger and thumb muscles, display a virtual hand, and move the hand according to one of two controllers: direct proportional EMG control or a pattern recognition classifier.

These experiments required subjects to conform a virtual hand into one of six randomly selected hand postures: cylindrical grasp, fine pinch, hook grasp, palmar grasp, thumb enclosed (in a fist), and lateral grasp (Fig. 1). These postures were chosen for three reasons: 1) They represent functional grasping postures used in activities of daily living, except the thumb enclosed posture, 2) they span the range of motion (ROM) of the virtual hand’s three controllable DOFs from the fully open position, and 3) they require different combinations of the DOFs to be coordinated in order to conform to the posture.

Fig. 1.

Six hand grasp postures represented in the virtual environment with CAPS: from top left (clockwise) are cylindrical, tip/fine pinch, hook, palmar, thumb enclosed, and lateral grasps.

A Target Achievement Control (TAC) test tasked subjects to conform the virtual hand into displayed target postures [9, 10]. An example is shown in Fig. 2A with the controllable avatar, which appears tan to the subjects, and the target pose in semi-transparent dark gray. Subjects had control over a single DOF thumb, index finger, and middle finger using the direct EMG controller. The ring and little fingers were slaved to follow the position of the middle finger. Using the pattern recognition classifier, the subjects had control over the hand’s end posture and the classifier coordinated the fingers’ movements to move the hand towards that posture. Subjects could not command individual fingers while using the pattern recognition classifier.

Fig. 2.

TAC test visual display example. A) TAC test avatar (tan colored hand) with semi-transparent target posture overlaid (dark gray). B) When the subject conformed the virtual hand to the target posture, the hand turned green.

The avatar turned green while the hand was aligned within the target (Fig. 2B). If the hand remained in the target zone for two seconds the trial ended in success and the hand would turn from green to dark green to signal successful completion of that trial. Subjects had 15 seconds to get the hand within the target and hold it there for the required two seconds before the end of the 17-second trial., Otherwise, the hand would turn yellow, the trial would end, and was counted as a failed attempt. This trial length was chosen based on previous studies [10], discussions with prosthetists on how long prosthesis users typically require to complete a grasping task with their device, and a consideration for how long subjects would be willing to sit for an experiment. There was a 10-degree target zone of acceptable error centered about the target posture for each degree of freedom. This target window has been shown to be effective in previous studies [10]. Each TAC test comprised 12 consecutive randomized trials, which tested the six hand postures twice, and was constrained so that no two consecutive trials tested the same posture.

B. Signal Processing

Two control algorithms were configured for these experiments: a direct proportional EMG control setup and a pattern recognition classifier. For both controllers a 250 ms window incremented every 25 ms was used to process the data and output a decision. The direct EMG controller used the mean absolute value (MAV) of the windowed EMG signals to determine motion of the virtual hand’s DOFs. Agonist/antagonist muscle pairs were used to proportionally control each DOF. The pattern recognition controller used a well-studied configuration of time-domain and auto-regressive signal features and a linear discriminate analysis algorithm to determine the output class [9, 10, 13, 24–27]. EMG activity from the six control sites were fed into a pattern recognition classifier, which was trained to choose one of the six hand postures, a hand open command, or a relaxed “no motion” state. This controller used a pre-determined linear path between the start and end postures when moving the virtual hand and subjects controlled the speed the hand moved towards that posture.

This pattern recognition system included proportional speed control of the motions and smoothed the output decisions with a velocity ramp [41]. Computation times for both algorithms were completed in less than 5 ms and the remainder of each frame was used to update and render the virtual environment. All output motions were restricted to have a maximum angular velocity of 100°/sec.

C. Controlling the Virtual Hand

The same six control sites were used as the inputs to each controller. The sites measured EMG activity from two muscles each for the thumb (FPL, EPL), index (FDP1, EDC1), and middle finger (FDP2, EDC2).

1) Direct EMG Control Setup

The two muscles associated with each digit acted as an agonist-antagonist pair under the direct proportional EMG control paradigm. The difference between their two activity levels determined the direction and magnitude of the output motion of the associated DOF in the artificial hand. This setup is most-commonly implemented for modern prosthesis users to control their devices.

The virtual fingers’ MCP and IP joints extended or flexed together with no abduction/adduction motion. The resultant motion for the fingers ranged from a straight position coplanar with the palm (all joints fully extended) to being curled with fingertips touching the palm (all joints flexed). The thumb’s motion combined flexion-extension and abduction-adduction of its joints. The range of motion started from “fully open” with the thumb abducted with slight flexion of the MCP joint and the IP joint fully extended. The “fully open” position approximated the position of your thumb just before shaking someone’s hand and is seen in Figure 2A (tan hand). The thumb was “fully closed” when it was adducted with the CMC, MCP, and IP joints flexed in a position as if to hold a coin against the palm of the hand using only the thumb tip.

2) Pattern Recognition Control Setup

Each subject trained the pattern recognition classifier to recognize eight patterns of his/her activity. Subjects were sequentially shown a picture of a hand posture (six grasps, hand open, or no motion), and they contracted their hand muscles as if to move their hand to that posture. The subjects performed isometric contractions because their hand was in a splint. Data were collected for three seconds for each posture so the classifier could learn to associate that pattern of activity with the desired posture. Subjects repeated each posture twice during a training phase. Three-second training trials have been demonstrated to be effective in prior TAC test studies [10].

The classifier output a single motion class for each decision cycle after training was completed. Classifying any of the six hand posture classes when the hand was at its fully open position (0% range of motion – ROM) would result in the hand moving toward that posture (i.e. “key grip”). Once the hand reached 10% ROM, only additional EMG activity pattern of the motion class selected at 10% ROM would continue to close the hand towards that particular posture. Functionally, this meant that if the subject wanted to change the hand to a different posture, they would have to fully open the hand and then close the hand in a different posture. This constraint was imposed to mimic implementation in a physical prosthesis that would limit object release due to inadvertent classifier selection once a grasp was already in use holding an object.

The range of motion of each hand posture class spanned from 0–115%. The hand in the fully open position was 0%, the target posture was 100%, and the full range of motion allowed for 15% overshoot. Once the hand was in the target posture, the subject would have to relax, classifying as “no motion”, in order to hold the hand in the target region.

D. Experimental Conditions

There were two conditions for which both the direct EMG control and the pattern recognition classifier TAC tests were conducted. The first condition tested each of the six grasps, starting the hand from the fully open position (Neutral Condition). The hand was reset to the fully open position between each trial. The second condition did not reset the hand to the open position between trials, but instead started each trial from the previous trial’s target posture (Posture-to-Posture Condition). The posture-to-posture condition was used to test the subjects’ ability to move the hand from varying postures to other postures. This demonstrated the subjects’ adaptability and versatility in controlling the hand throughout the hand’s full ROM and in several different starting configurations.

Each subject completed four TAC tests for each of the four conditions. Each TAC test consisted of 12 randomized trials (postures) giving a total of 48 trials for each condition. Each subject was given as much time to practice as desired before data collection for each new controller and condition. In general, subjects took 10–15 minutes to become familiar with the controller interface and to learn how to move the hand. Unbeknownst to the subjects, the first TAC test of each new set was treated as a practice and was not used in the data analysis. Consequently, 36 trials for each condition-controller combination were analyzed.

The order for which controller was tested first, direct proportional EMG control or pattern recognition control, was randomized The neutral condition was tested first. The more visually complex posture-to-posture condition was performed second.

E. Statistical Analysis

An analysis of variance (ANOVA) was performed on the data to determine if significant differences (P-value less than 0.05) existed between controllers in the same testing condition. ‘Failure Rate’ and ‘Completion Time’ were tested and compared between ‘Direct Control’ and ‘Pattern Recognition.’

III. RESULTS

A. Direct Control of Simultaneous Degrees of Freedom

The percentage of trial time spent moving one, two, or three DOFs was measured for successfully completed trials. Median values are shown in Fig. 3 for single-, two-, and three-DOF movements. Error bars depict the inter-quartile range.

Fig. 3.

Median amount of normalized trial time spent simultanesouly moving DOFs. Data is shown for one, two, and three DOFs together for the Neutral condition tests (top) and Posture-to-Posture condition tests (bottom). Error bars depict inter-quartile range. Fig. 4 Simultaneous DOF percentages during increments of 10% of normalized trial time. Median values are shown for the Neutral condition (top) and Posture-to-Posture condition (bottom) tests. Traces represent No Movement (white), single DOF (green), two DOF (purple), and three DOF (red) movements.

During neutral condition tests (Fig. 3A), subjects moved a single DOF and two DOFs of the hand 33% and 42% of trial-time, respectively. Subjects coordinated the movement of all three DOFs simultaneously only 2.7% of trial-time. However, during posture-to-posture tests (Fig. 3B), subjects spent more time moving single DOFs (40% of the time) than they did moving two of the DOFs together (30% of the time).

B. Did the Number of Simultaneously Moving DOFs Change During the Trial? (Neutral Condition)

Fig. 4 shows how subjects simultaneously moved the hand’s DOFs during the trials. All trials were normalized by their trial time and then divided into 10 divisions. The amount of time spent moving no DOFs, a single DOF, two DOFs, or three DOFs together was tabulated for each segment. Median data is shown across all subjects.

Fig. 4.

Simultaneous DOF percentages during increments of 10% of normalized trial time. Median values are shown for the Neutral condition (top) and Posture-to-Posture condition (bottom) tests. Traces represent No Movement (white), single DOF (green), two DOF (purple), and three DOF (red) movements.

During the Neutral condition tests (Fig. 4A), subjects initially moved a single DOF at the start of a trial, but shortly afterwards initiated a second DOF to move simultaneously. Trials can be distinguished into two phases: the first phase focused on two simultaneously moving DOFs and a second phase where the amount of trial-time spent moving a single DOF increased. This trend was representative of each subject. However, the point within the trial when the change between phases occurred varied between subjects.

Fig. 4B shows how subjects moved the DOFs simultaneously during the posture-to-posture condition tests. Similar to the Neutral condition breakdown, there was an increase in two degree of freedom movement during the middle half of the trials and an increase in single degree movement during the final quarter. However, unlike the Neutral condition, single degree of freedom movement was greater than two degree of freedom movement throughout the trial.

C. Which Degrees-of-Freedom Moved Together?

Under Direct Control, Fig. 5 shows the median amount of time spent moving each combination of the hand’s three DOFs during successful trials and errors bars depict the interquartile range. Calculations were made for each successful trial based on the output from the controller, and a tally was made for each DoF at each time step to determine whether or not it was moving. Each successful trial was then normalized by its completion time. Examining the breakdown for single DOF movement (T, I, or M) demonstrates that subjects were capable of independently moving each degree of freedom. During the neutral condition, there was more isolated index finger movement (11.6% of normalized trial time) than thumb or middle finger movement (6.6% and 2% of normalized trial time). From the neutral position fine pinch and hook grasps only required the movement of two DOFs to reach the target posture. All other grasps required all DOFs to be moved.

Fig. 5.

Simultaneous DOF combinations. Breakdown showing how subjects moved combinations of degrees of freedom across all target postures during Neutral condition (top) and Posture-to-Posture condition tests (bottom). Median values with error bars depicting the interquartile range across all successful trials and all subjects. Combinations are shown for Thumb (T), Index Finger (I), and Middle Finger (M).

The strategies for controlling the hand’s DOFs were different between the Neutral condition and the Posture-to-Posture condition; namely, there was more time spent in the latter moving single DOFs (Fig. 5, bottom). However, the thumb performed most of the single degree of freedom movements during this condition. The index finger was the primary individual mover during the Neutral condition tests. These results are interesting as Fraser and Wing have observed that during grasping between the thumb and index finger, that one side (usually the thumb) tends to remain more stationary while the other moves to accommodate the object [42].

The movements that involved two DOFs also differ from the patterns seen in the Neutral condition tests. Subjects spent more time coordinating movements of the thumb and index finger and less time moving the fingers together. There is still very little time spent moving the thumb along with the middle finger and moving all three DOFs simultaneously, similar to the results from the Neutral condition.

D. Pattern Recognition Classifier Accuracy

Average classifier accuracy across subjects was 85% with a standard deviation of 8%, when tested off-line on training data. This was acceptable as a previous study has reported a classifier accuracy of 69% for hand grasp postures using a pattern recognition classifier discriminating surface EMG signals [9].

E. Failure Rate

Performance metrics were calculated for each controller and condition in order to compare the direct myoelectric controller to the pattern recognition classifier. These included the failure rate, completion time, and percent of completed trials.

The failure rate measures the proportion of trials that were unsuccessfully completed during the TAC tests and were averaged across subjects. Fig. 6 shows the TAC test failure rates for each of the four control conditions. Failure rates were below 20%, which shows that subjects were generally able to complete the task, but it was not an easy task. This also coincides with previously reported failure rates for wrist and hand motions [10, 41]. There was no significant difference (P>0.05, ANOVA) between the failure rates of the two controllers in the neutral condition tests or the posture-to-posture condition tests. Posture-to-posture conditions resulted in lower failure rates when compared to the Neutral condition tests.

Fig. 6.

TAC test failure rates. Mean failure rates of TAC test trials shown for Direct EMG control and Pattern Recognition Control for both test conditions: Neutral and Posture-to-Posture (P2P). Error bars depict the standard error.

F. Completion Time

The completion time is the time it took the subjects to conform the virtual hand to the target. The completion time does not include the two seconds of dwell time that were required in order to successfully complete each trial. Only successful trials were used to compute the average completion times shown in Fig. 7. Averages were computed across subjects for each controller and condition and error bars represent the standard error.

Fig. 7.

TAC test completion times. Average completion time of TAC trials for Direct EMG control and Pattern Recognition Control and both the Neutral and Posture-to-Posture conditions. Error bars represents standard error. Shaded region represents average time spent initially performing “Hand Open” in the Posture-to-Posture condition using Pattern Recognition control.

In the neutral condition, pattern recognition control resulted in a significantly lower completion time (P<0.05, ANOVA) than the direct controller. The pattern recognition controller was faster with the same completion success rate as direct control.

The completion times were not significantly different (P>0.05, ANOVA) between the direct EMG control and the pattern-recognition control in the Posture-to-Posture condition. The software limitation imposed during pattern-recognition control required subjects to return the hand to the fully open position before switching to a different posture. Therefore, the posture-to-posture test using pattern recognition control was essentially a neutral condition test with an additional step of first opening the hand. Average pattern recognition completion time of the posture-to-posture condition was 6.8 sec, and the hand-opening phase averaged 1.61 seconds (gray shaded region).

G. Completion Time vs. Percentage of Completed Trials

Fig. 8 shows the cumulative percentage of completed trials, out of 36 total trials for each controller and condition pair, as a function of the trial time. All four TAC tests followed a similar and expected pattern.

Fig. 8.

TAC test completion time versus percentage of completed trials. More trials were completed as time increased with a majority of the trials being completed after 5–7 seconds.

In the neutral condition tests, pattern recognition control (triangles) resulted in faster completions when compared to the direct control (plus signs) results. The Posture-to-Posture condition TAC tests (circles and squares) resulted in higher completed trials than the Neutral condition tests (plusses and triangles). These results show that it took subjects approximately 10 seconds to achieve 80% completed trails across the six grasping postures under the Posture-to-Posture condition.

IV. DISCUSSION

A. Direct Control Extrapolations: Moving Degrees-of-Freedom Simultaneously and the Bias of Target Postures

Providing the ability to simultaneously control several prosthetic fingers is a laudable goal and one would expect greater dexterity with such capabilities. However, providing this ability does not ensure that the subjects will actually use that ability. The hand postures in this experiment required the index and middle fingers to be stopped next to one another when commanded using Direct Control – except for the ‘fine pinch’ posture. The fingers also started in the same configuration during the Neutral condition tests and therefore could follow the same trajectory from start to target configurations. This may have contributed to the large portion of the time spent moving the index and middle fingers together. The similar motions of these digits seemed to be easier for subjects to accomplish than dis-similar motions such as moving digits different directions, distances, or to/from different start or target configurations.

From Fig. 4 and Fig. 5 it is surmised that subjects started moving two DOFs simultaneously early on and through the middle portion of the trials, and those two DOFs were most often the index and middle fingers. Subjects would intuitively move the fingers together since five of the six postures required the fingers to have the same position in the target posture.

It was surprising how little time was spent simultaneously moving all the digits. From the neutral position, the cylindrical, chuck, lateral, and thumb-enclosed postures each required movement of all the hand’s digits. Fine pinch and the hook grasp only required movement from two DOFs to reach the target from the neutral position. It was hypothesized that subjects would initially move all the digits close to the target simultaneously and then move each digit into the target one at a time. This strategy seems to have been implemented with only two DOFs instead of all three.

The index finger spent more time moving than any other digit; moving by itself and in conjunction with the middle finger degree of freedom. The index finger was the only digit that was required to move from the neutral pose to conform the hand for the six target postures. All target postures except fine pinch showed that the index finger DOF was moving a majority of the time, either alone or with the middle finger DOF.

Fig. 3 shows that, using the Direct Control setup, very little time was spent simultaneously moving all DOFs. Analysis of the movements during each target posture showed that simultaneous three-DOF movements occurred most frequently during the “thumb enclosed” target posture (12.8% of the time). The chuck, key, and cylindrical postures also included simultaneous three-DOF motions (9.5%, 5.4%, and 2.7% of time, respectively). These amounts are not enough to affirm our initial assumptions that subjects would prefer to move all DOFs simultaneously towards the target, and then sequentially once the hand was relatively close to the final target posture. Instead, subjects chose to move simultaneous DOFs given the desired target posture, and it is assumed that different simultaneous DOF movement patterns would be observed with different target postures.

B. Limitations in Simultaneously Controlling Multiple Degrees of Freedom

Physiologically realistic mappings between the muscles being used as control inputs and the resultant hand movement sought to minimize the cognitive load on the subjects by reducing how much they had to “think” about what they were doing. However, it was found that subjects did not coordinate movements of all of the digits of the hand simultaneously, as is done with a healthy and intact hand. There are many differences to consider between normal hand movement and that done by EMG in these experiments. Perhaps the two keys issues are the number of control inputs used and the lack of proprioceptive feedback. In these experiments, the subjects were moving digits with the input of just six muscles. In the normal hand, many more extrinsic and intrinsic muscles would be used. Although in the direct control experiments the EMG signals used were functionally related to the DOFs they were controlling, they were far from a complete or normal set of muscle inputs. Thus, there was still a much larger cognitive burden than in a normal hand. Perhaps this is why the subjects generally used few simultaneous DOFs.

Additionally, subjects were asked to control their EMG activity, which is a very unusual task. In an intact hand, single-digit movements generally require the activation of multiple muscles and the person is typically focused on the position, velocity, and/or force of the digit. In this study, subjects were wearing a splint which constrained movement but allowed large force generation, and all the while they were asked to focus on their muscles’ activity.

The lack of proprioception is also an important difference. The subjects had to rely on visual feedback to control all three DOF’s. This mimics how a prosthesis user is limited to primarily visual feedback from the device. Simultaneously controlling up to three DOFs, based on visual feedback, may be the limitation of human attention. Williams and Kirsch have found that subjects controlling a two-DOF cursor using four EMG sites tended to move sequential DOFs instead of simultaneously [20, 43]. However, that may have been due to the non-intuitive nature of using face and neck muscles as control sites for the computer cursor. Masliah and Milgram measured subjects’ coordination of simultaneous controlling 6-DOFs, and found that they tended to focus on subsets of the DOFS, limiting themselves to at most three at a time [44]. Our lab has documented amputee subjects using a prosthetic arm and hand to simultaneously move four DOFs to perform gross positioning motions of the limb, however, subject’s focused on one or two simultaneous DOFs once the hand was close to the intended target [45].

Future work could increase the number of control inputs to an artificial hand by using activity from additional extrinsic finger muscles and compartments. Researchers are currently working on bringing commercially available intramuscular electrodes to market [18, 46–49]. This could allow for an increase in controllable digits and DOFs in artificial hands, however, simultaneous control of all of the DOFs may be limited. Furthermore, additional training may improve performance of simultaneous DoF control. Subjects in this study were not limited in the amount of training they wished to take, but none took more than 30 minutes. Typical prosthesis users, by contrast, train for weeks with their device.

C. Direct EMG control versus Pattern Recognition Control for the Hand

The results from these experiments strongly suggest that the control of a multiple DOF hand is feasible using intramuscular EMG with either direct or pattern recognition control. This is required for the direct EMG control setup because surface electrodes are not capable of distinguishing the muscle activity of individual finger muscle compartments. Our hypothesis was validated that extrinsic finger and thumb muscles can serve as control inputs and allow for individual finger control of a multifunctional artificial hand. Pattern recognition control has also been effective in controlling multiple DOFs with extrinsic muscles. In these experiments, intramuscular direct EMG and pattern recognition control of a three-DOF hand performed equally well.

Another consideration is how the user expects to interact their prosthetic hand with the environment. Pattern recognition control is limited to a few, pre-defined grasping patterns and does not allow for the user to change or modify the grasping motion once it has been initiated. Using the direct EMG control could enable users to move individual digits during and after the initiation of a grasping motion. This potentially allows the user more adaptability in grasping different sized and shaped objects; as well as gesturing and doing many other complex hand functions needed every day.

D. Summary of Important Results

• Subjects could individually control multiple muscles for each digit including agonist/antagonist muscle pairs. EMG activity from EDC1, FDP1, EDC2, FDP2 of the fingers as well as EPL and FPL of the thumb were used as EMG control sites to command individual digits on a 3-DOF artificial hand.

• Subjects were able to use direct proportional EMG control and pattern recognition control to conform the virtual hand prosthesis into six functional grasps from a neutral position. Furthermore, subjects using direct EMG control also demonstrated their ability to command the hand to change from one posture directly into another.

• Functional performance metrics suggested that direct EMG control performed similar to pattern recognition control, when moving a hand from a neutral, fully open start posture.

• Subjects demonstrated simultaneous control of two of the three DOFs using direct EMG control; however, they rarely controlled all three of the DOFs simultaneously. Simultaneous control of two DOFs was frequently observed, but the determination of which two DOFs was biased by the target posture.

• Evidence suggested that subjects used simultaneous control during gross motions to approximate the hand to the target posture before commanding single DOFs for fine-position control into the desired posture.

Biographies

J. Alex Birdwell (S’11-M’12) earned his B.S. degree in mechanical engineering from the Georgia Institute of Technology, Atlanta, GA, USA, in 2004, the M.S. and Ph.D. degrees in mechanical engineering from Northwestern University, Evanston, IL USA in 2006 and 2012, respectively.

He is currently a lecturer of mechanical engineering at Northwestern University, Evanston, IL USA. His research interests lie in the intersection of robotics and biomechanics and mainly focus on prosthetics, human-machine interactions, and rehabilitation engineering. He was previously a graduate research assistant in the Center for Bionic Medicine at the Rehabilitation Institute of Chicago, Chicago, IL USA. He currently enjoys teaching an array of manufacturing, design, and experimental methods courses.

Levi J Hargrove (S’05-M’08) received his B.Sc. and M.Sc. and PhD degrees in electrical engineering from the University of New Brunswick (UNB), Fredericton, NB, Canada, in 2003 2005, and 2007 respectively. He joined the Center for Bionic Medicine at the Rehabilitation Institute of Chicago in 2008. His research interests include pattern recognition, biological signal processing, and myoelectric control of powered prostheses. Dr. Hargrove is a member of the Association of Professional Engineers and Geoscientists of New Brunswick. He is also a Research Assistant Professor in the Department of Physical Medicine and Rehabilitation (PM&R) and Biomedical Engineering, Northwestern University. Evanston, IL, USA.

Richard F. ff. Weir received a BA in mathematics and a BAI in microelectronics and electrical engineering from Trinity College Dublin in 1983. After working as a control engineer in England, he moved to the USA and obtained his MS and PhD degrees in biomedical engineering from Northwestern University in Evanston, IL.

He is Director of the Biomechatronics Development Laboratory at the University of Colorado Denver | Anschutz Medical Campus. He is also a Research Healthcare Scientist for the VA Eastern Colorado Health Care System – Denver VA Medical Center, and holds Research Associate Professor appointments in the Departments of Bioengineering and Physical Medicine & Rehabilitation at the University of Colorado Denver | Anschutz Medical Campus.

Dr. Weir specializes in the design of advanced artificial hand/arm replacements. Dr. Weir's research covers all aspects of the problem ranging from development of neural control interfaces and clinical deployment of these systems, to mechatronic design and development of novel prosthetic components. Current projects involve the development of prosthetic hand/arm controller systems based on implantable myoelectric sensors (IMES) to create a neural interface for the user. These sensors when placed in the muscles of an amputee’s residual limb, sense the muscle’s inherent electrical activity and send signals out of the body to an external controller, which will use these signals to control a multifunction prosthesis. We also are conducting research into novel ways to interface with peripheral nerves using opto-genetic approaches as well as a number of straight forward mechanical hand and arm component design projects.

Todd A. Kuiken received his MD and Ph.D. in biomedical engineering from Northwestern (1990) and completed his residency in PM&R at the Rehabilitation Institute of Chicago (1995). Dr. Kuiken currently is the Director of the Center for Bionic Medicine at the Rehabilitation Institute of Chicago, Chicago, IL, USA. He is a Professor in the Departments. of Physical Medicine and Rehabilitation, Biomedical Engineering, and Surgery, Northwestern University, Evanston, IL, USA.