Abstract
Context
Improving the function of prosthetic arms remains a challenge, as access to the neural control information for the arm is lost during amputation. We have developed a surgical technique called targeted muscle reinnervation (TMR) which transfers residual arm nerves to alternative muscle sites. After reinnervation, these target muscles produce an electromyogram (EMG) on the surface of the skin that can be measured and used to control prosthetic arms.
Objective
Assess the performance of TMR upper-limb amputee patients using a pattern-recognition algorithm to decode EMG signals and control prosthetic arm motions.
Design
Surface EMG signals were recorded on participants and decoded using a pattern-recognition algorithm. The decoding program controlled the movement of a virtual prosthetic arm. Participants were instructed to perform various arm movements, and their abilities to control the virtual prosthetic arm were measured. In addition, TMR patients used the same control system to operate advanced arm prosthesis prototypes.
Setting
This study was conducted between January 2007 and January 2008 at the Rehabilitation Institute of Chicago.
Participants
This study included five patients with shoulder disarticulation or transhumeral amputations who received TMR surgery between February 2002 and October 2006. It also included five non-amputee (control) participants.
Main Outcome Measure
Performance metrics measured during virtual arm movements included motion-selection time, motion-completion time, and motion-completion (or `success') rate. Three of the TMR patients were also able to test advanced arm prostheses.
Results
TMR patients were able to repeatedly perform 10 different elbow, wrist and hand motions with the virtual prosthetic arm. For TMR patients, the average (standard deviation (SD)) motion-selection and motion-completion times for elbow and wrist movements were 0.22 s (0.06) and 1.29 s (0.15), respectively. These times were 0.06 s and 0.21 s longer than the average times of control participants. For TMR patients, the average (SD) motion-selection and motion-completion times for hand-grasp patterns were 0.38 s (0.12) and 1.54 s (0.27), respectively. TMR patients successfully completed an average (SD) of 96.3% (3.8) of elbow and wrist movements and 86.9% (13.9) of hand movements within 5 s, compared to 100% (0) and 96.7% (4.7) completed by controls. Three of the patients were able to demonstrate the use of this control system in advanced prostheses including motorized shoulders, elbows, wrists and hands.
Conclusion
These results suggest that reinnervated muscles can produce sufficient EMG information to control advanced artificial arms.
Introduction
The loss of one or both arms is a major disability that profoundly limits the everyday capabilities and interactions of upper-limb amputees. Currently available prostheses do not adequately restore the function of an individual's arm and hand. The most commonly used prostheses are body-powered. These devices capture remaining shoulder motion with a harness, and transfer this movement through a cable to operate the hand, wrist or elbow. With this control method, only one joint can be operated at a time. Myoelectric prostheses use the electromyogram (EMG) signals (the electrical signals generated during muscle contraction) from residual limb muscles to control motorized arm joints. Current control strategies use the amplitudes of the EMG signals from one or two remaining muscles to sequentially operate each function in the prosthesis.1 For example, the biceps and triceps muscles are used by a transhumeral amputee to control the elbow, wrist and hand. The user must trigger a “mode switch” to sequentially select which of these devices is to be actuated. This type of operation is not intuitive, as the residual muscles control physiologically unrelated movements. The use of currently available arm prostheses is cumbersome and slow for people with transhumeral or shoulder disarticulation amputations—the people whose disability is the most severe.2–8
We have developed a new surgical technique, called targeted muscle reinnervation (TMR), to improve myoelectric prosthesis control.9–14 With TMR, remaining arm nerves are transferred to residual chest or upper-arm muscles that are no longer biomechanically functional due to loss of the limb. Once reinnervated, these muscles serve as biological amplifiers of motor commands from the transferred arm nerves and provide physiologically appropriate EMG signals for hand, wrist and elbow control. TMR has been successfully performed in people with transhumeral and shoulder disarticulation amputations and has markedly improved their functional use of prostheses.10, 12, 13 Using a simple control paradigm based only on the amplitude of EMG signals from reinnervated muscles, TMR amputees can intuitively and simultaneously control hand open/close and elbow extension/flexion.
Further investigation has shown that TMR provides a rich source of motor control information. Electrode arrays were used to record EMG signals as TMR patients attempted 16 different motions involving the elbow, wrist, thumb and fingers. The patterns produced by the combined EMG signals during the performance of different movements were used by a computer to create a “classifier.” The classifier was then used to predict motions being performed based on the current pattern of EMG signals. This strategy is called “pattern recognition.” Analysis of the data revealed that the intended motions could be classified with an average classification accuracy of 95 %.15, 16 However, it is unknown whether reinnervated muscles can stably and accurately provide myoelectric signals for real-time control of multifunction prostheses. Real-time performance metrics are required to examine the clinical robustness and accuracy of myoelectric prosthetic control with TMR.
This study assessed the real-time control of multifunction prostheses based on TMR combined with a pattern-recognition algorithm. Performance metrics (motion selection time, motion completion time and completion rate) were quantified by training and testing with a virtual multifunction prosthesis. It also was demonstrated that participants were able to successfully operate advanced experimental upper limb prostheses. A qualitative assessment of the control of these advanced arm systems is presented including videos of the patients using the devices.
METHODS
This study was conducted with five patients who had undergone TMR surgery 11–70 months prior to testing. For comparison, five non-amputee control participants were included in the study. This study was approved by the Northwestern University Institutional Review Board and conducted between January 2007 and January 2008 at the Rehabilitation Institute of Chicago. All participants gave informed consent and provided permissions for publication of videos and photographs for scientific and educational purposes.
Five of the six shoulder-disarticulation or transhumeral amputees who had undergone TMR surgery in collaboration with the Rehabilitation Institute of Chicago agreed to participate in this study. Three of these participants had shoulder-disarticulation amputations. Patient S1 was a 54-year-old man who underwent bilateral shoulder-disarticulation amputations in May 2001 following high-voltage electrical injuries to both arms. During TR surgery, which took place in February 2002, his residual musculocutaneous, median, radial, and ulnar nerves were transferred to the pectoralis major and pectoralis minor muscles (Figure 1a).11, 17, 18 Patient S2 was a 24-year-old woman with a left shoulder-disarticulation (very short residual humerus) amputation resulting from a motor-vehicle accident in May 2005. During TMR surgery, the musculocutaneous, median, radial and ulnar nerves were transferred to portions of the pectoralis major and serratus anterior muscles (Figure 1b).13 Patient S2 began fitting with her TMR prosthesis in February 2007. Patient S3 was a 37-year-old man who underwent right shoulder-disarticulation amputation in February 2005 following severe electrical burns. During TMR surgery, which took place in July 2006, the musculocutaneous, median, radial and ulnar nerves were transferred to the pectoralis major, pectoralis minor, and latissimus muscles (Figure 1c). Two patients with transhumeral amputations also participated in the study. A 50-year-old man with a right transhumeral amputation resulting from a motor-vehicle accident in April 2004 (T4) had the median nerve transferred to the medial biceps and the distal radial nerve transferred to the brachialis muscle during TMR surgery in January 2005.14 A 38-year-old woman with a left transhumeral amputation due to a motor-vehicle accident in April 2006 (T5) had the median nerve transferred to the medial biceps, and the distal radial nerve transferred to the lateral triceps during TMR surgery in October 2006. For comparison, five healthy non-amputees (three males and two females, aged 20 to 45 years) participated in the study. The control participants were chosen to have representation of both genders and an age range similar to the TMR patients.
Figure 1.
Schematic of TMR surgery in a participants S1 (a), S2 (b) and S3 (c).
EMG Data Collection
For each TMR patient, 12 self-adhesive bipolar EMG electrodes were placed on the skin over the reinnervated muscles. Four electrodes were placed at sites chosen previously through clinical evaluation to control the amputees' prostheses.10–12 The eight additional sites were determined by an electrode-placement optimization algorithm16 which sought to maximize the classification accuracy for different movements. For control participants, 12 electrodes were used to record EMG signals from physiologically appropriate muscles in the arm and hand. One electrode was placed over the biceps muscle, and a second over the triceps muscle; six electrodes were placed around the proximal forearm; one electrode was placed on the dorsal side of the wrist; and three electrodes were placed on the hand (medial and lateral thenar eminence and hypothenar eminence). The EMG signals were amplified and band-pass filtered from 5–400 Hz. Data were sampled at 1 kHz by an analog-to-digital converter (Measurement Computing, USB-116FS) and processed in real time on a desktop computer using the software platform Matlab (The Mathworks, Natick, MA).
Classifier Training and Testing
The 11 motion classes were elbow flexion, elbow extension, wrist flexion, wrist extension, wrist pronation, wrist supination, hand opening, three types of hand grasps and a no-movement class. TMR patients were allowed to try five different hand-grasp patterns: three-jaw chuck, fine pinch, key grip, power grip and tool grip (Figure 3a). Each patient chose three of these grips based on relative ease and reliability of control. For control participants, the three grasps were three-jaw chuck, fine pinch and tool grip; the three most commonly used grasps chosen by the patients. The participants were instructed to follow a demonstration of each movement displayed in random order on the computer screen (Figure 3b) and to perform the movement with a comfortable and consistent level of effort. The prompt was displayed with a countdown during the rest time between trials to give patients time to prepare. EMG data were collected in eight consecutive trials. In each trial, each motion was repeated twice and held for 4 s, producing 8 s of EMG recordings per motion. There was a 3 s time interval between motions in the four even-numbered trials. A variable rest time of 0–3 s was used in the four odd-numbered trials in an attempt to keep the participants engaged and enhance the classifier's robustness. EMG data from the eight trials were split into two groups: the four odd-numbered trials were combined and used to train the classifier; the four even-numbered trials were combined and used to test the classifier. The pattern-recognition algorithm used in this study was implemented as follows: EMG recordings were segmented into a series of 150 ms analysis windows with 50 ms of overlap, resulting in a new classification every 100 ms. Four time-domain features3, 15 were extracted from EMG signals in each analysis window. The combined features from the even-numbered trials were used to train a linear discriminant analysis (LDA) classifier.3, 15, 16, 19 This LDA classifier was then used to classify the combined features from the testing set. The classification accuracy for each movement was the percentage of total analysis windows for that class which were correctly classified. The overall classification accuracy was the average of these values for all (11) movements. The LDA classifier was then used in real-time to classify features extracted from real-time EMG signals, produce a new prediction of the motion class every 100 ms, and control a virtual reality arm or a physical prosthesis, as described below. Computational time for each analysis window was less than 3 ms.
Figure 3.
(a) Five hand-grasp patterns used in this study. (b) Screen shot of the prompted movement and responding virtual arm.
Virtual Prosthesis Control
Experiments with a virtual prosthesis were performed immediately after classifier training. Participants were instructed to follow visual prompts for each movement, and a virtual arm that responded to the classifier output was displayed on the screen (Figure 3b). Once the participants correctly selected the desired movement, they were asked to maintain it until the virtual arm completed the movement. The time of movement onset was identified as the time of the last `no movement' classification (Figure 4). Each of the 10 motions was randomly presented three times in a trial and the trials were repeated six times for a total of 180 movements: 72 hand-grasp motions and 108 elbow and wrist motions. These data were used to evaluate the speed and consistency of control using real-time pattern recognition.
Figure 4.
Two performance metrics: motion-selection time (MSt) and motion-completion time (MCt). The target motion classes are shown by green dots and the decisions of the classifier are depicted as blue circles. Each target movement started from a state of rest. The classifier made a motion prediction every 100 ms. The mean absolute values of the recorded EMG signals are shown in the bottom panel.
The performance metrics used to assess virtual prosthesis control were motion-selection time, motion-completion time, and motion-completion (or `success') rate. The motion-selection time was the time taken to correctly select a target motion and was defined as the time from movement onset to the first correct classification (Figure 4). This quantity measures how quickly motor commands can be translated into correct motion predictions. The motion-completion time was defined as the time from movement onset to the tenth correct classification (which represented the full range of motion for any movement) (Figure 4). The fastest possible speed to complete any motion was 1 s, corresponding to 10 consecutive correct classifications with new classifications occurring every 100 ms. If the correct class was not selected within a 5 s time limit, the movement was considered a failure. The motion-completion rate was the percentage of successfully completed motions out of the total attempted motions (72 attempted motions for the hand, and 108 attempted motions for the elbow and wrist) within the time limit. Because the motion-selection and motion-completion data for each participant was highly skewed, the median value for all six arm movements (elbow and wrist) and all four hand movements (hand open and three hand-grasps) were calculated for each participant, and these values were averaged across the five TMR patients and five control participants.
Preliminary research demonstrated that hand-grasp patterns were more difficult to perform than elbow and wrist movements. Therefore, the control scheme for hand grasps was modified. A hand grasp could only be selected when the hand was fully open. Once a grasp was selected, any hand-grasp pattern would close the hand in the initially selected grasp. However, if the initial hand-grasp pattern selected was incorrect, the patient would have to fully open the hand and try again.
Physical Prosthesis Control
Three of the TMR patients were able to test advanced upper arm prosthesis prototypes developed under the Defense Advanced Research Project Agency's Revolutionizing Prosthetics program. Video of this initial testing is presented as supplemental information.
Johns Hopkins University Applied Physics Lab (JHUAPL) and their collaborators developed a seven-degree-of-freedom prosthetic arm that was tested with patient S1 in January 2007. S1 controlled flexion and extension of the motorized shoulder by using residual shoulder motion to operate a mechanical rocker switch. A motorized humeral rotator was controlled with EMG signals from the residual deltoid and latissimus dorsi muscles. Powered elbow flexion/extension, wrist pronation/supination, wrist flexion/extension and a hand that allowed three-jaw chuck and lateral pinch grip were controlled with EMG signals from TMR muscles and the pattern recognition algorithm.
DEKA Integrated Solutions Corporation and collaborators developed a 10 degree-of-freedom prosthetic arm system that was tested with patients S1, S2 and T5 in May, June and July 2007, respectively. A shoulder controller operated with residual shoulder movement allowed shoulder-disarticulation patients to simultaneously operate shoulder flexion/extension and abduction/adduction. Humeral rotation was controlled with EMG signals from the latissimus dorsi and deltoid muscles. The powered elbow, wrist and hand were controlled with pattern recognition of EMG signals recorded over TMR muscles. For patient T5, the humeral rotator was controlled with a switch, while the elbow, wrist, and hand were controlled with pattern recognition of EMG signals recorded over TMR muscles. The DEKA hand had multiple motors and was able to form a variety of hand-grasp patterns including those shown in Figure 3a.
Surface electrodes were either self-adhesive or built into the patients' prosthetic sockets. The arm systems were trained at the beginning of each session with a short pattern-recognition protocol similar to the one described above. Training and testing with the prostheses occurred over a two-week period for each patient. Sessions generally lasted two to three hours with one session in the morning and one in the afternoon.
RESULTS
Virtual Prosthesis Testing
The mean (SD) classification accuracy was 97% (2) for control subjects and 88% (7) for TMR patients.
The majority of movements were selected quickly, with motion-selection times less than 0.3 s (Figure 5a). The average (SD) motion-selection times for elbow and wrist movements (elbow flexion/extension, wrist rotation and wrist flexion/extension) were 0.16 s (0.03) for control subjects and 0.22 s (0.06) for TMR patients (Table 1). The motion-completion rate (SD) for the 180 movements was high; 96.3% (3.8) for TMR patients and 100% (0) for control participants. For TMR patients, the selection of the appropriate hand-grasp patterns took longer and had, on average, a 9.4 % lower success rate than wrist and elbow movements (Table 1). For control participants, the median motion-selection time for hand grasps was similar to that of elbow and wrist movements (Table 1). The motion-completion rate for hand grasps was slightly (3.3%) lower (Table 1).
Figure 5.
(a) Motion-selection time histogram for both TMR patients and control participants. (b) Motion-completion time histogram for both TMR patients and control participants. Both times were calculated from all the completed movements with a time limit of 5 s. The time bin is 0.1 s for motion-selection times and 0.5 s for motion-completion times.
Table 1.
Performance metrics for virtual arm protocol for a 5.0 second time limit.
| Performance metrics | TMR Patients (n=5) | Control Participants (n=5) |
|---|---|---|
| Motion-Selection Time (s) | Average (SD) | Average (SD) |
| Elbow & wrist classes* | 0.22 (0.06) | 0.16 (0.03) |
| Hand grasp classes** | 0.38 (0.12) | 0.17 (0.09) |
| Motion-Completion Time (s) | Average (SD) | Average (SD) |
| Elbow & wrist classes* | 1.29 (0.15) | 1.08 (0.05) |
| Hand grasp classes** | 1.54 (0.27) | 1.26 (0.18) |
| Motion-Completion Rate (%) | Average (SD) | Average (SD) |
| Elbow & wrist classes* | 96.3 (3.8) | 100 (0) |
| Hand grasp classes** | 86.9 (13.9) | 96.7 (4.7) |
for 108 attempted elbow and wrist movements
for 72 attempted hand-grasps
The movements performed by both TMR patients and control participants were also completed quickly, consistent with the high classification rates (Figure 5b). The fastest possible motion-completion time was 1 s, representing perfect classification of the intended movement. The average (SD) motion-completion times for elbow and wrist movements were 1.29 s (0.15) for TMR patients and 1.08 s (0.05) for control participants. For both groups, hand grasps took longer to complete than arm movements: the average (SD) motion-completion times for hand grasps were 1.54 s (0.27) for TMR patients and 1.26 s (0.17) for control participants.
The average (SD) motion-completion rates within a 3 s time limit were 94.1% (out of 108 elbow and wrist movements) and 80.3% (out of 72 hand grasps) for TMR patients, and 99.6% (out of 108 elbow and wrist movements) and 93.6% (out of 72 hand grasps) for control participants (Figure 6c). The motion-completion rate for hand grasps was lower for two of the five TMR patients. The motion-completion rate increased dramatically as the time limit was increased up to approximately 2 s and then began to plateau; the maximum motion-completion rate was generally reached by 6 s.
Figure 6.
Completion rate vs. time. (a) Elbow and wrist completion rates for the five TMR patients. (b) Hand-grasp completion rates for the five TMR patients. (c) Average completion rates for TMR patients (black lines) and control participants (red lines), respectively.
Real-Time Control of Advanced Prosthetic Arm Systems
Building upon the virtual arm training experience, three TMR patients were able to demonstrate control of physical arm systems as shown in the supplemental videos (Videos 1–4) (http://www.ric.org/research/centers/necal.…). All three patients were able to perform basic operations using pattern-recognition control on the first day of testing. Over a two-week trial period, their proficiency improved with practice, systems debugging and minor systems improvements.
The patients were able to operate all functions of the prosthetic arm prototypes. Control of arm and hand movements using pattern-recognition of EMG signals from TMR muscles provided the ability for intuitive, sequential control of the elbow, wrist and hand. The shoulder disarticulation patients were able to simultaneously operate the shoulder and arm. Patients generally performed one motion at a time and would occasionally operate two joints simultaneously for reaching tasks.
The joints on these prostheses were capable of relatively high speeds. The speed range was customized to each patient as the patients preferred to operate the arms at slower speeds to allow more accurate control. This control is demonstrated by patient S1 catching checkers rolling across a table (Video 1), patient S2 stirring a spoon in a cup (Video 3) and patient T5 moving small blocks (Video 4).
The powered shoulder systems markedly increased the workspace of the prostheses. Motorized shoulder flexion allowed the shoulder disarticulation patients to reach above their heads. Motorized shoulder flexion also allowed these patients to have a deeper workspace as demonstrated by S2 performing reaching motions over a table (Figure 7a). Humeral rotation and shoulder abduction widened the workspace for all patients and facilitated reaching to the midline for bimanual tasks. The powered wrist served mostly to preposition a functional wrist angle and facilitate better hand operation. The increased number of powered joints also allowed more precise orientation of the hand in space. For example, a ring could be moved across a geometric wire, as shown in Figure 7b.
Figure 7.
Patients using experimental arm prostheses. (a) Patient S2 was shown reaching to catch a tissue box using the DEKA arm. (b) Patient S1 was shown moving a ring across a geometric wire using the JHUAPL arm. (c) Patient T5 was shown grabbing a pen using the DEKA arm.
Many different hand grasps were possible with the DEKA arm system; patients were able to try three-jaw chuck, lateral pinch, fine pinch, power grasp and tool grip. Patients varied in their abilities to control different grasps: one patient could reliably select four hand-grasp patterns, another could control up to three, and the third could reliably operate two. Training with a smaller number of hand grasps improved performance. Choosing different hand-grasp patterns was also more difficult than operating the wrist or elbow motions, consistent with the virtual data presented above. The different grasps facilitated different functional activities. For example, the power grip allowed a firm grasp of a hammer and the fine pinch enabled the picking up of small objects (Figure 7c).
COMMENT
This study presents experiments on real-time control of highly articulated artificial arms in patients with targeted muscle reinnervation—a novel neural-machine interface. In this study, we have demonstrated that a pattern-recognition algorithm can be used to decode surface EMG data from reinnervated muscles and provide intuitive control of powered elbows, wrists and hands.
The quantification of outcomes is always a challenge with respect to arm function. EMG-pattern-recognition studies generally report classification accuracies found with able-bodied participants. The accuracies are generally in the range of 90–100% regardless of the classification algorithm chosen, demonstrating a ceiling effect. Classification accuracy, however, is a limited metric of control function: a study by Lock et al. failed to demonstrate a high correlation between pattern classification accuracy and simple performance testing.20 In this study, classification accuracy for TMR patients was lower than that for control participants. The values measured here were similar to previously reported data from our lab and other investigations of using pattern recognition of EMG signals to classify intended movements.16, 21, 22 It is important to note that this is only the averaged accuracy of being able to hold a motion for 4 s—it is not a dynamic estimation of performance.
In this study, we have developed a protocol to assess the control of a virtual arm; a protocol we believe is more challenging and allows measurement of more insightful performance parameters. The data from control participants performing the same tests are presented as a reference, and represent the performance possible with a more complete EMG data set. Our protocol allows the quantification of several key performance metrics. The motion-selection time is the speed at which the user can access a function in the prosthesis. Faster is clearly better, but what is good enough? Farrell et al. found that participants did not appreciate a time delay of less than 100 ms23 and others have advocated that a delay of up to 300–400 ms is acceptable.21, 22, 24 Thus the motion-selection times of TMR patients for arm function are quite good (220 ms or less) and the motion-selection time for hand grasps is perhaps marginal at 380 ms. It should be noted that this delay is not due to computational processing, which only takes a few milliseconds. The motion-selection time delay is intrinsic to EMG control, as it represents the need for sufficient data to accumulate before an accurate decision can be made.
The motion-completion time is a measure of speed of use. We set a normalized gain so that a task could be completed in 1 s, which correlates to a reasonable speed for prosthesis function of 90–120 degrees/s. Here we see that the TMR patients did quite well compared to the control subjects. Perhaps the most important metric was the motion-completion rate: a measure of robustness. A very high success rate is needed to allow adequate function and prevent user frustration. One patient had excellent completion rates of 99% for the elbow and wrist and 96% for the hand grasps (Figures 7a and 6b); this was comparable to the control participants (Figure 6c). Two others clearly struggled to perform multiple hand-grasping functions. In general the success rates for elbow and wrist functions were high but the success rates for controlling multiple hand grasps were much lower, demonstrated that this is more challenging to some patients.
Quantifying operation of a virtual arm allowed measurement of some useful metrics in the laboratory. However, the ultimate goal is for amputees to operate more dexterous prosthetic arms. Controlling a real prosthesis introduces many practical challenges such as EMG signal recording stability, interference from muscles controlling remaining joints, and the effects of tissue loading and arm dynamics.25 As part of this Defense Advanced Research Agency program, we had the opportunity to do initial testing of advanced robotic arms with three of our patients. The arms were operated using practical control schemes including: pattern recognition control using EMG signals from TMR muscles; conventional myoelectric technology; and custom powered shoulder controllers. The supplemental videos provide compelling qualitative results. Each subject was able to gain some mastery of the systems in the first day of testing. Within two weeks they were all able to demonstrate encouraging control of these complex devices. Our training and testing periods were brief. Improvement in control and function would be expected with more practice. Simultaneous operation of all shoulder motions and one arm movement was demonstrated. However, patients would only use simultaneous operation of two joints for reaching tasks and usually operated only one motion at a time. This is likely due to the cognitive burden of operating such a complex device.
The lack of sensory feedback is an obstacle for controlling such complex devices. Patients are currently forced to rely on visual feedback. Improved sensory feedback, especially proprioception, will be critical to the long-term goal of neural integration and more natural control of complex robotic arms.
These early trials demonstrate the feasibility of complex multifunction prosthesis control with TMR. Additional research and development need to be done before field trials can be performed. Improving EMG signal recording repeatability and stability are required to minimize or eliminate daily classifier training. Work is ongoing to develop more robust surface EMG recording systems and prosthetic interfaces. Implantable EMG systems eliminate some of the surface EMG recording problems and may provide more stable systems.26 Adaptive pattern-recognition algorithms also may improve the stability of control with extended use. There are various hierarchical control schemes that may be more robust for some patients; customization of control hierarchy is an accepted practice in modern prosthetics. The JHUAPL and DEKA robotic arms performed very well as early prototypes. Further improvements are needed, and planned, including reducing the size and weight and increasing the robustness of these advanced prostheses.
Supplementary Material
The video shows that Patient S1 operated the JHUAPL arm for grasping and moving small objects and catching checkers rolling across a table.
The video shows that Patient S1 operated the DEKA arm for performing four different hand-grasp patterns and arm movements with TMR pattern-recognition control, drinking with a cup, and catching and moving different small objects.
The video shows that patient S2 operated the DEKA arm for stirring a spoon in a cup, catching a tissue box from table, and grabbing small things such as key and cookie.
The video shows that Patient T5 operated the DEKA arm for performing hand-grasp patterns and moving a ball and small blocks.
Figure 2.
Timeline of relevant history of TMR patients including original amputation, TMR surgery, practice with the virtual arm, and testing for this study.
Acknowledgments
Funding/Support: This work was supported by the Defense Advanced Research Projects Agency, Army Research Office, DEKA Integrated Solution Corp. Prime Contract W911NF-06-C-001, the National Institute of Child Health and Human Development Grants R01 HD-043137, and the JHU-APL-DARPA Contract No. 908090 under Prime Contract No. N66001-06-C-8006R01 HD0-044798 and the National Institute of Child Health and Human Development Grants R01 HD-043137.
Additional Contributions: We thank the engineering teams at Johns Hopkins University and DEKA Integrated Solutions Corp. for the development of the advanced prosthesis and the collaborative work to implement these systems with targeted reinnervation patients. We would also like to thank Aimee Schultz for assistance with the preparation of this manuscript.
Footnotes
Author Contributions: Drs Kuiken and Li had full access to all the data in the study and take equal responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Kuiken, Li, Lock, Lipschutz, Miller, Stubblefield, Englehart
Acquisition of data: Li, Lock, Lipschutz, Miller, Stubblefield
Analysis and interpretation of data: Li, Kuiken, Lock
Drafting of the manuscript: Kuiken, Li
Critical revision of the manuscript for important intellectual content: Kuiken, Li, Lock, Lipschutz, Miller, Stubblefield, Englehart
Statistic analysis: Li
Obtained funding: Kuiken
Administrative, technical, or material support: Kuiken, Li, Lock
Study supervision: Kuiken
Financial Disclosers: None reported.
Conflicts of Interest: None reported
Role of the Sponsors: The study sponsors (Defense Advanced Research Projects Agency) had no role in the design or conduct of the study; the collection, management, analysis, or manuscript preparation. The sponsor did require approval of all manuscripts associated with the Revolutionizing Upper Limb Prostheses programs.
References
- 1.Sears HH. Trends in upper-extremity prosthetics development. In: Bowker HK, Michael JW, editors. Altas of limb prosthetics: Surgical, prosthetic, and rehabilitation principles. 2 ed. American Academy of Orthopedic Surgeons; Rosemont, IL: 1992. [Google Scholar]
- 2.Parker PA, Scott RN. Myoelectric control of prostheses. Crit Rev Biomed Eng. 1986;13(4):283–310. [PubMed] [Google Scholar]
- 3.Hudgins B, Parker P, Scott RN. A new strategy for multifunction myoelectric control. IEEE Trans Biomed Eng. 1993;40(1):82–94. doi: 10.1109/10.204774. [DOI] [PubMed] [Google Scholar]
- 4.Huang YH, Englehart KB, Hudgins B, Chan ADC. A gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses. IEEE Trans Biomed Eng. 2005;52(11):1801–1811. doi: 10.1109/TBME.2005.856295. [DOI] [PubMed] [Google Scholar]
- 5.Ajiboye AB, Weir RF. A heuristic fuzzy logic approach to emg pattern recognition for multifunctional prosthesis control. IEEE Trans Neural Syst Rehabil Eng. 2005;13(3):280–291. doi: 10.1109/TNSRE.2005.847357. [DOI] [PubMed] [Google Scholar]
- 6.Edell DJ. A peripheral-nerve information transducer for amputees - long-term multichannel recordings from rabbit peripheral-nerves. IEEE Trans Biomed Eng. 1986;33(2):203–214. doi: 10.1109/TBME.1986.325892. [DOI] [PubMed] [Google Scholar]
- 7.Strange KD, Hoffer JA. Restoration of use of paralyzed limb muscles using sensory nerve signals for state control of fes-assisted walking. IEEE Trans Rehabil Eng. 1999;7(3):289–300. doi: 10.1109/86.788466. [DOI] [PubMed] [Google Scholar]
- 8.Branner A, Stein RB, Fernandez E, Aoyagi Y, Normann RA. Long-term stimulation and recording with a penetrating microelectrode array in cat sciatic nerve. IEEE Trans Biomed Eng. 2004;51(1):146–157. doi: 10.1109/TBME.2003.820321. [DOI] [PubMed] [Google Scholar]
- 9.Kuiken TA. Consideration of nerve-muscle grafts to improve the control of artificial arms. Technology and Disability. 2003;15:105–111. [Google Scholar]
- 10.Kuiken TA, Dumanian GA, Lipschutz RD, Miller LA, Stubblefield KA. The use of targeted muscle reinnervation for improved myoelectric prosthesis control in a bilateral shoulder disarticulation amputee. Prosthet Orthot Int. 2004;28(3):245–253. doi: 10.3109/03093640409167756. [DOI] [PubMed] [Google Scholar]
- 11.Hijjawi JB, Kuiken TA, Lipschutz RD, Miller LA, Stubblefield KA, Dumanian GA. Improved myoelectric prosthesis control accomplished using multiple nerve transfers. Plast Reconstr Surg. 2006;118(7):1573–1578. doi: 10.1097/01.prs.0000242487.62487.fb. [DOI] [PubMed] [Google Scholar]
- 12.Kuiken TA. Targeted reinnervation for improved prosthetic function. Phys Med Rehabil Clin N Am. 2006;17(1):1–13. doi: 10.1016/j.pmr.2005.10.001. [DOI] [PubMed] [Google Scholar]
- 13.Kuiken TA, Miller LA, Lipschutz RD, et al. Targeted reinnervation for enhanced prosthetic arm function in a woman with a proximal amputation: A case study. Lancet. 2007;369(9559):371–380. doi: 10.1016/S0140-6736(07)60193-7. [DOI] [PubMed] [Google Scholar]
- 14.O'Shaughnessy KD, Dumanian GA, Lipschutz RD, Miller LA, Stubblefield KA, Kuiken TA. Targeted reinnervation to improve prosthesis control in transhumeral amputees. A report of three cases. J Bone Joint Surg Am. 2008;90(2):393–400. doi: 10.2106/JBJS.G.00268. [DOI] [PubMed] [Google Scholar]
- 15.Zhou P, Lowery M, Englehart K, Hargrove L, Dewald JPA, Kuiken TA. Decoding a new neural-machine interface for superior artificial limb control. J Neurophysiol. 2007;98:2974–2982. doi: 10.1152/jn.00178.2007. [DOI] [PubMed] [Google Scholar]
- 16.Huang H, Zhou P, Li G, Kuiken TA. An analysis of emg electrode configuration for targeted muscle reinnervation based neural machine interface. IEEE Trans Neural Syst Rehabil Eng. 2008;16(1):37–45. doi: 10.1109/TNSRE.2007.910282. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Kuiken TA, Dumanian GA, Lipschutz RD, Miller LA, Stubblefield KA. The use of targeted muscle reinnervation for improved myoelectric prosthesis control in a bilateral shoulder disarticulation amputee. Prosthetics and Orthotics International. 2004;28(3):245–253. doi: 10.3109/03093640409167756. [DOI] [PubMed] [Google Scholar]
- 18.Kuiken TA, Marasco PD, Lock BA, RN H, Dewald JPA. Redirection of cutaneous sensation from the hand to the chest skin of human amputees with targeted reinnervation. Proc Natl Acad Sci U S A. 2007;104(50):20061–20066. doi: 10.1073/pnas.0706525104. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Duda RO, Hart PE, Stork D. Pattern classification. 2 ed Wiley-Interscience; 2000. [Google Scholar]
- 20.Lock BA, Englehart K, Hudgins B. Real-time myoelectric control in a virtual environment to relate usability vs. Accuracy. MyoElectric Controls Symposium; August, 2005; New Brunswick, Fredericton. [Google Scholar]
- 21.Englehart K, Hudgins B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans Biomed Eng. 2003;50(7):848–854. doi: 10.1109/TBME.2003.813539. [DOI] [PubMed] [Google Scholar]
- 22.Englehart K, Hudgins B, Chan AD. Continuous multifunction myoelectric control using pattern recognition. Technology and Disability. 2003;15(2):95–103. [Google Scholar]
- 23.Farrel TR. PhD Dissertation. Northwestern University; 2007. Multifunctional prosthesis control: The effect of targeting surface vs. Intramuscular electrodes on classification accuracy, and effect of controller delay on prothesis performance. [Google Scholar]
- 24.Hefftner G, Zucchini W, Jaros GG. The electromyogram (emg) as a control signal for functional neuromuscular stimulation--part i: Autoregressive modeling as a means of emg signature discrimination. IEEE Trans Biomed Eng. 1988;35(4):230–237. doi: 10.1109/10.1370. [DOI] [PubMed] [Google Scholar]
- 25.Farina D. Interpretation of the surface electromyogram in dynamic contractions. Exerc Sport Sci Rev. 2006;34(3):121–127. doi: 10.1249/00003677-200607000-00006. [DOI] [PubMed] [Google Scholar]
- 26.Weir REF, Troyk PR, DeMichele G, Kuiken TA. Implantable myoelectric sensors (imes) for upper-extremity prosthesis control. Preliminary work. 25th Silver Anniversary International Conference of the IEEE Engineering in Medicine and Biology Society (EMBS); Cancun, Mexico. September 17–21, 2003. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
The video shows that Patient S1 operated the JHUAPL arm for grasping and moving small objects and catching checkers rolling across a table.
The video shows that Patient S1 operated the DEKA arm for performing four different hand-grasp patterns and arm movements with TMR pattern-recognition control, drinking with a cup, and catching and moving different small objects.
The video shows that patient S2 operated the DEKA arm for stirring a spoon in a cup, catching a tissue box from table, and grabbing small things such as key and cookie.
The video shows that Patient T5 operated the DEKA arm for performing hand-grasp patterns and moving a ball and small blocks.