Surgically Implanted Electrodes Enable Real-Time Finger and Grasp Pattern Recognition for Prosthetic Hands

Abstract

Currently available prosthetic hands are capable of actuating anywhere from five to 30 degrees of freedom (DOF). However, grasp control of these devices remains unintuitive and cumbersome. To address this issue, we propose directly extracting finger commands from the neuromuscular system. Two persons with transradial amputations had bipolar electrodes implanted into regenerative peripheral nerve interfaces (RPNIs) and residual innervated muscles. The implanted electrodes recorded local electromyography with large signal amplitudes. In a series of single-day experiments, participants used a high speed movement classifier to control a virtual prosthetic hand in real-time. Both participants transitioned between 10 pseudo-randomly cued individual finger and wrist postures with an average success rate of 94.7% and trial latency of 255 ms. When the set was reduced to five grasp postures, metrics improved to 100% success and 135 ms trial latency. Performance remained stable across untrained static arm positions while supporting the weight of the prosthesis. Participants also used the high speed classifier to switch between robotic prosthetic grips and complete a functional performance assessment. These results demonstrate that pattern recognition systems can use intramuscular electrodes and RPNIs for fast and accurate prosthetic grasp control.

I. INTRODUCTION

Hands are incredibly important to people because they provide the opportunity to handle tools, operate machines, and are essential components of social interaction and communication. The loss of an upper extremity can severely impact a person’s ability to interact with the world around them. Advanced robotic prosthetic devices have been developed which can provide 5 to 30 degrees of freedom (DOF) of the human hand and provide adequate gripping force for functional tasks [1]–[3]. However, current control schemes are either cumbersome, unintuitive, or unreliable, leading many users to abandon their devices [4]. In spite of these issues, myoelectric hands are still an attractive approach for prosthetic rehabilitation following limb loss due to their high grip strength and potential for intuitive control [5], [6].

Commercial control systems typically use a mode selection scheme, where the user toggles between different movements and then secondarily activates them. A more fluid control strategy is pattern recognition in which the intended movement is classified from the users’ recorded electromyography (EMG) signals and can be proportionally activated based on signal amplitude [7]. However, grasp distinction is not common, even amongst persons with transradial amputations where extrinsic flexors and extensors are still present in the residual limb [8]. Classifiers require a rich set of distinct inputs to accurately interpret movement intentions. Existing systems, that use surface EMG, record a spatiotemporal summation of motor unit action potentials from a distant location which is complicated by cross-talk between channels [9] and reduced signal strength [10]. Furthermore, signals from deep finger flexors (i.e. flexor digitorum profundus to the index finger), are obscured by more superficial muscle activity in the forearm (i.e. flexor digitorum superficialis to the middle finger) or lost entirely in more proximal amputations. In addition to these factors, the reliability of commercial surface electrodes is limited by impedance and position alterations due to sweating or contact shifting [11]. Hence, distinguishing grasps is challenging for pattern recognition systems that use surface EMG because finger-specific signals are hard to capture and sensitive to environmental changes.

II. RELATED WORK

A. Software and External Sensing

Over the past decade, researchers have investigated software and sensing techniques to improve the reliability of pattern recognition systems and expand their capabilities to include grasp distinction. Participants with transradial amputations have used surface EMG and pattern recognition to select between 6 and 7 randomly cued hand postures with average movement completion rates of 53.9% [8] and 77% [12], respectively. In simulations, other studies used deep networks or additional input features to distinguish up to 10–12 hand and wrist postures with greater than 90% offline accuracy [13], [14]. However, pattern recognition algorithms can struggle to generalize to new contexts. For example, classifiers often issue incorrect predictions while supporting the prosthesis weight in untrained arm positions [15]. One study found that more robust classifiers can mitigate this issue [16]. However, this study did not include multiple grasp distinctions. Another study modelled transitions to show that three grasps could be distinguished in a variety of static arm positions and used in a functional task [17]. Alternatively, another group used sensor fusion of EMG and inertial measurements to control four grasps and hand open. They found that it could improve performance across the workspace [18], but required context-specific training. Previous studies that quantified multiple grasp control have used combinations of software innovations, adhesive electrodes to improve signal consistency, or additional sensing modalities.

B. Surgical Interventions

Surgical interventions can directly access the peripheral nervous system and improve the quality and fidelity of motor control signals. For users with more proximal amputations (i.e. above elbow amputation), EMG-based systems are limited due to the fact that the muscles used for finger movements are missing. Therefore, a pattern recognition system would rely on subtle co-activations of remaining musculature, which is inherently difficult. Targeted muscle reinnervation (TMR) is a surgical procedure in which transected peripheral nerves are used to reinnervate surgically denervated areas of muscle. The target muscles are typically superficial and after reinnervation produce functional control signals that can be recorded via surface EMG [19]. TMR patients equipped with adhesive electrodes selected four hand postures in a virtual reality environment with an average completion rate of 86.9%. Some patients have been able to control multiple grasps of robotic prostheses in laboratory environments [20], [21] and home trials [7]. In combination with TMR, decomposition algorithms can estimate independent nerve signals from global recordings [9]. This approach requires high-channel electrode grids and real-time control studies have focused on separation of wrist and gross hand movements to date [22]. Implantable electrodes have also been proposed to record efferent motor commands directly from individual nerves. However, these electrodes are limited by low amplitudes or lack of chronic stability [23], [24]. Conversely, electrodes implanted into muscle tissue can record strong and stable EMG signals. In one study, indwelling electrodes eliminated the need to re-calibrate a regression controller in a virtual reality environment for months [25]. Intramuscular electrodes have also been shown to improve signal strength and control precision independent of changes to the mechanical interface or control strategy [10]. In another study, electrodes with multiple contacts were implanted into individual muscle bellies, enabling 6 degree of freedom (DOF) dexterous control including individual fingers of a robotic hand [26]. However, the ability for intramuscular electrodes to capture finger movements is dependent on the availability of residual innervated musculature or other surgical interventions.

C. Regenerative Peripheral Nerve Interfaces

At the University of Michigan, we have developed the Regenerative Peripheral Nerve Interface (RPNI) which consists of a free muscle graft sutured to the end of a divided peripheral nerve. The muscle graft regenerates and becomes reinnervated by the regenerating peripheral nerve axons. RPNIs have been shown to effectively prevent and treat neuroma pain and phantom pain [27], [28]. Most importantly, RPNIs serve as a stable biological amplifier for efferent motor signals, retaining anatomical consistency between the host nerve and RPNI functions [29]–[31]. In a clinical trial, high resolution control signals recorded from electrodes implanted into RPNIs remained stable for up to 300 days (the duration of observation)[32]. Vu et al. also used a pattern recognition algorithm in a virtual environment to validate RPNI control capabilities [32]. Pattern recognition is the state of the art for intuitive device control and a natural match for most multi-articulating hands that are designed to switch between grip patterns. Therefore, it is both an easily comparable and immediately applicable control paradigm for implantable technologies.

III. STUDY APPROACH

In this study, we continue experiments with two persons with transradial amputations and electrodes surgically implanted into RPNIs and residual innervated muscles. We investigate the signal quality and control capabilities of the full set of RPNIs and residual innervated muscles. We demonstrate that indwelling electrodes captured stable EMG control signals from both RPNIs and residual muscles with a highly favorable Signal-to-Noise ratio (SNR) in both participants. In addition to large amplitude, the EMG was also highly specific for motor decoding. A variety of decoding algorithms could be implemented to translate these signals into prosthetic movement commands. Real-time pattern recognition often presents a trade-off between responsiveness and stability when determining the length of the EMG processing window for classification [33]. To mitigate this issue, we utilize a Hidden Markov Model (HMM-NB) which learns transitions between latent states to rapidly decode grasps without sacrificing stability [34]. We use a virtual posture switching task to evaluate real-time finger and grasp control. To our knowledge, this is the fastest and most accurate signal acquisition and pattern recognition system that can switch directly between individual finger movements. We quantified performance for two posture sets: a 10 class set including extrinsic and intrinsic individual finger and hand movements and wrist flexion, and a five class set of functional grasps. To demonstrate robustness across physical contexts, we quantified stability of a grasp classifier in one participant across novel static arm postures with a prosthesis donned. Finally, both participants completed a functional assessment where they used robotic prostheses to move differently shaped objects that required the use of multiple hand grasps.

IV. MATERIALS AND METHODS

A. Participant Anatomy

Two persons with transradial amputations, underwent surgery to have eight pairs of bipolar electrodes (Synapse Biomedical, Oberlin OH) chronically implanted into RPNIs and residual innervated muscles. The intramuscular recording technique was approved by the Food and Drug Administration under Investigational Device Exemption G160229/S002. All experiments were conducted in accordance with the University of Michigan’s Institutional Review Board, study ID HUM00124839. For both participants, informed consent was obtained after the nature and possible consequences of the study was explained. Both participants also signed consent forms for media deidentification.

P1 is a 30 year old male who sustained a right wrist disarticulation as the result of a traumatic hand injury. In 2015 he underwent surgery to resect neuromas on the median, ulnar, and radial nerves. A single RPNI was created on each nerve using free skeletal muscle grafts from his ipsilateral vastus lateralis. P1 is not a body powered user due to a shoulder injury. In March of 2018, P1 underwent an additional surgery to have eight pairs of bipolar intramuscular electrodes chronically implanted into the median and ulnar RPNIs. P1 had the following residual muscles targeted for implantation: Flexor Pollicis Longus (FPL), Flexor Digitorum Profundus - Index Section (FDPI), Flexor Digitorum Profundus - Small Section (FDPS), Extensor Pollicis Longus (EPL), Extensor Digitorum Communis (EDC), and Flexor Carpi Radialis (FCR). P1 completed experiment sessions for this study between November 2018 and February 2019. The electrodes remained implanted for approximately one year and were partially explanted in March of 2019 by removing any exposed and subcutaneous wire lengths not embedded in muscle. In February 2020, P1 had his electrodes fully explanted from RPNIs and residual muscles.

P2 is a 53 year old female who, during treatment for septic shock and acute renal failure, suffered an intravenous extravasation of calcium into her right hand and forearm, which led to tissue necrosis and required a partial hand amputation. Her hand became contracted with extremely limited function and she underwent a voluntary transradial amputation in October of 2017. Single RPNIs were created on each of the median and radial nerves while an intraneural dissection was performed on the ulnar nerve to create two RPNIs. All RPNIs were creating using free muscle grafts from the ipsilateral vastus lateralis. P2 currently wears a body powered prostheses outside of the study, although she reports to seldom use the open-close function. In October of 2018, P2 underwent chronic implantation of eight pairs of bipolar electrodes into the median and both ulnar RPNIs and five residual muscles. The same residual muscles were targeted for both patients except for FDPS, which was only targeted for P1. P2 completed experiments for this study between and February 2019 and February 2020. At the time of writing, P2 remains implanted.

B. Signal Processing and Experiment Set-Up

EMG from the implanted electrode contacts was sampled at 30kSps, and referenced with a Cerebus Neural Signal Processor (Blackrock Microsystems, Salt Lake City, UT). The Cerebus simultaneously recorded and sent referenced EMG to a Matlab xPC (Mathworks, Natick, MA) for real-time processing. Fig. 1d shows the signal processing chain on the xPC for online decoding. The xPC applied the 100–500Hz band-pass filter and down-sampled EMG to 1 kSps, from which mean absolute value (MAV) was the only feature extracted for online decoding. The xPC sent UDP packets to manipulate two virtual hands [35] which were presented to the subject on an external laptop. A lead hand to cue postures was positioned in the foreground, while a secondary hand which the subjects could control was positioned in the background. The xPC’s real-time guarantee ensured that EMG was decoded and time-synced with behavioral data within one millisecond.

Fig. 1.

Posture switching task and performance metrics. (a) The HMM-NB model contained three interconnected states for rest and holds with one transition state to and from rest. Point shown for example. Posture probabilities were determined by summing the probabilities of rest and hold states. (b) To optimize model parameters, an iterative learner was carefully initialized by sequentially placing rest, hold, and transition states. (c) P2’s online activity while switching to point. (d) Participants controlled a secondary hand (background) to match posture cues (foreground). (e) The optimized model captured EMG dynamics in (c). One state (red) reflects a sharp increase in EDC activity while two states (green and purple) capture continued hold with the Ulnar RPNIs. (f) Participants were required to hold the cued posture for one uninterrupted second. Trial latency was measured as the time difference between new EMG activity and a successful hold. Online accuracy was evaluated across individual timesteps for the first second of transition to the cued posture. 1 of 10 posture set with errors shown.

C. Hidden Markov Model for Real-Time Control

A variety of classification algorithms could be implemented to decode EMG into movement commands. Since the implanted electrodes record high resolution motor activity from specific muscles and RPNIs, it is likely that many algorithms would achieve good results. Here, we implemented a Hidden Markov Model (HMM-NB) that models transitions between latent states and closely resembled previous work [34]. The probability of a latent state occurring is determined both by the observed EMG inputs as well as the occurrence of previous states. A linear transition matrix determined the likelihood of moving from one latent state to another. As shown in Fig. 1a, a Naive Bayes model represented three interconnected “hold” states per posture and “transition” states going to and from rest. The HMM-NB was explored as a principled solution to reduce the length of the EMG processing window and necessary output filtering. The latter methods improve performance during transitions, but automatically introduce a compromise between classifier stability and responsiveness [33].

The HMM-NB was trained by instructing participants to mimic 5–7 repetitions of each posture with their phantom limb. P1 completed calibration runs with pseudo-randomly ordered cues with alternating rest and hold periods of 2.5 seconds each. P2 preferred a slower pace and performed the calibration run with rest and hold periods of 3 seconds each and consecutively ordered cues. An unsupervised expectation-maximization algorithm optimized state and transition parameters. Latent states were carefully initialized by splitting the active and rest periods of the training data into thirds as in Fig. 1b. Active periods were automatically marked in training data per trial by retroactively finding active channels and selecting the starting point of when their MAV exceeded 4 times the standard deviation of that trial’s rest data for at least 200ms. The transition matrix for “hold” and “rest” states was initialized with an equal probability of remaining in or moving to another connected state. “Transition” states were initialized with a 0.9 probability of remaining in the state. We limited the learner to five iterations and selected parameters with the highest likelihood on held out test data [36]. During online decoding, a posture was output if the sum of the probabilities of the three “hold” or “rest” state probabilities exceeded 0.8. The “transition” states were not selected for output because they represented the beginning of activations from rest, and participants could directly switch between grasps during real-time control. Fig. 1c shows an example of P2 initiating a point during online control, and Fig. 1e shows the solution found by the HMM-NB captured dynamic EMG patterns. These features allow the HMM-NB to operate with less time history and more accurately represent movement phases than single-state classifiers. A performance comparison to a single-state NB classifier is included in the Appendix.

D. Virtual Posture Switching Task

Table I shows the posture sets explored in this study. MAV from all eight channels and calculated with 50 ms of time history, was used to control the virtual hand for both the 1 of 10 and grasps posture sets. P1 had significant time constraints during the study. He performed one session of each virtual posture switching task, while P2 completed three. The calibration routine was relatively quick (approximately five minutes or less) and usually completed the day of online sessions. However, older calibrations – 18 days for P1’s 1of 10 posture switching task and 16 days for P2’s arm position test – were used twice to save time and effort. For the grasps posture set, P1’s visual prompt showed finger extension instead of finger abduction. In preliminary calibration sessions, we found these two cues produced similar EMG responses. P1’s prompt for point also resembled small finger flexion. This cue was consistent with other physical prostheses experiments P1 performed with the LUKE arm, which couples the middle and ring fingers to the small finger.

TABLE I.

Posture Sets Used in the Study

Posture Set	No. of Classes	Classes
1 of 10	10	Thumb, Index, Middle, Ring, and Small Finger Flexion (T,I,M,R,S), Wrist Flexion (WF), Finger Abduction (Ab), Finger Adduction (Ad), Thumb Opposition (TO), Rest (Re)
Grasps	5	Fist (F), Pinch (Pi), Point (Po), Finger Abduction (Ab), Rest (Re)
Functional	4	Fist (F), Pinch (Pi), Point (Po), Rest (Re)

P2 performed three experiment sessions of the virtual task with the 1 of 10 and grasps posture sets, while P1 performed one session with each set. The functional posture set was used by both participants to control robotic prostheses and for the static arm position test performed by P2.

During online control sessions, participants controlled the secondary hand and attempted to match the cue hand. The decoder was a free running classifier, meaning the output was never automatically reset to rest or a particular posture. The secondary hand turned green when the decoder output matched the cued posture as an additional success indicator. Participants were required to hold the cued posture for one second without interruption within a 10 second timeout period. A five second timeout was used for P2’s arm position test. After a successful hold or failure, a new posture was immediately cued in a pseudo-random order.

This task encourages participants to directly switch between postures at a faster pace than the training procedure. The requirement of a continuous hold ensures that successful controllers must be both stable and responsive. Fig. 1f highlights two of P2’s real-time control trials with errors to demonstrate the analysis metrics for the posture switching task. Online classification accuracy was evaluated across individual timesteps for the first second of transitions to the cued posture. Perfect accuracy required the classifier to switch and immediately maintain a successful hold. The one second hold period was chosen to be similar to the selection period in previous work [8], [20]. Sometimes, participants would pause before attempting the cued posture, possibly due to physical or mental fatigue. Rest outputs under those circumstances were ignored, however moving from the cue to rest was penalized. Trial latency measured decoder responsiveness and controllability. It was calculated as the time difference between the onset of new muscle activity and the beginning of a successful hold. EMG onset was determined visually by viewing the filtered and rectified EMG for each trial and marking the beginning of a new EMG pattern. EMG onset approximates reaction time and averaged 604.9±268.8 ms for P1 and 517.8±281.6 ms for P2 (mean±s.t.d.). Latency was not calculated for trials without a distinct EMG change or when the pseudo-random order produced a duplicate. These were rare occurrences, accounting for 3.5% of all trials. Failed trials were marked with a 10 second latency to reflect task timeout. Median and i.q.r. were used to characterize latency since the distributions did not have a normal shape.

E. Functional Testing

To be of practical use for patients with upper limb amputations, decoders must be reliable in different physical contexts. P2 donned a myoelectric prostheses and performed the virtual posture switching task to quantify decoder performance across seven novel arm orientations. P2 had limited elbow range of motion and was asked to match the arm positions shown in Fig. 4 to the best of her ability. P2’s right passive elbow range of motion was recorded in clinic November 2017 to be 20 – 120° of flexion and in October 2020 measured by experimenters to be 20 – 125° of flexion. Clinicians also noted limited ability to supinate her forearm with maximal supination in neutral at 0°. In August 2020, experimenters measured her passive right shoulder range of motion to be 160° shoulder flexion and 90° external rotation.

Fig. 4.

Robustness across static arm positions. P2 calibrated the HMM-NB with her arm resting on a table and completed the online posture switching task in seven novel arm positions. (a) P2 donned her prostheses but used the virtual reality environment to switch between fist (F), pinch (Pi), point (Po) and rest (Re). (b) Latency histograms binned in 50 ms increments for most positions aggregated (blue, n = 116 trials), as well as the two positions with more frequent and longer transition errors (red, n = 37 trials). (c) In addition to online accuracy, the number of transition errors was also reported for each of the eight positions.

Both participants donned myoelectric prostheses to perform a Southampton Hand Assessment Procedure (SHAP)-inspired task. Participants relied solely on their robotic hand for visual feedback of the pattern recognition system. The functional posture set directly activated closed grips (fist, pinch, point) of the robotic prostheses and opened the hand during rest. Future studies may consider proportional control of motor speed by including finger abduction to activate an open signal. The robotic hand interface implemented here did not support this capability and was sensitive to instabilities. We also observed some instances of unintentional extensor activity during arm movements, which could erroneously trigger open predictions. P2’s functional decoder used MAV from her RPNIs, FPL, FDPI, and EDC. EPL was observed to activate unintentionally and held out. FCR was not needed to distinguish grasps and removed as a precaution. P1 used a decoder that received MAV from his RPNIs, FPL, FDPI, and FDPS. Alternate channel combinations were not explored due to time constraints.

To control robotic hands, the xPC sent commands via serial connection to a custom circuit board which issued the appropriate CAN messages for each hand. The software interface controlled prostheses by directly translating predicted postures to closed grasps while rest commands opened the hand. P1 used a LUKE arm (Mobius Bionics, Manchester NH) which operated in position control mode. Finger positions corresponding to the endpoints of the current predicted posture were sent to the LUKE arm, which processed updates every 10ms. P2 used an i-Limb Quantum™ XS (Ossur, Reykjavik, Iceland). A state machine was coded which required a rest command to open the i-Limb before transitioning to a new closed grasp. The i-Limb interface introduced a hardware delay due to the time to process a grip change. However, the compact form factor was greatly preferred for P2, who could not lift the LUKE arm.

Durplex sockets for the prostheses were fabricated for each participant’s residual limb at the University of Michigan Orthotics and Prosthetics center by a certified prosthetist. A custom adapter was made to connect the LUKE arm to P1’s socket, which was secured to his forearm with an Otto Bock silicone liner (Ottobock, Duderstadt, Germany). His socket also featured a window to allow access to the percutaneous electrode connectors on his medial forearm. For P2, the i-Limb was connected to her socket with a quick wrist disconnect (QWD) that allowed manual wrist rotation. The QWD was embedded in a PVC adapter that connected to her socket which was secured to her forearm with an Iceross Upper-X liner (Ossur, Reykjavik, Iceland). Her percutaneous leads exited on her lateral bicep and did not interfere with her socket.

V. RESULTS

A. Speed and Accuracy of Finger and Grasp Classifications

In offline simulations, the HMM-NB distinguished the same 1 of 10 posture set in P1 and P2 with 95.4% and 94.1% accuracy (Fig. 2a,d). Both participants also controlled the virtual hand to complete the fast-paced posture switching task (Movie 1). In one experiment session, P1 successfully maintained a one second hold within the timeout period on 100% of his trials. Online classifier accuracy captured errors during the initial attempt to move to a cued posture. P1 achieved an online accuracy of 93.0% in his experiment session (Fig. 2b). As shown in Fig. 2c, P1 used the HMM-NB to rapidly switch between postures with a trial latency of 159±237 ms (median±i.q.r.). P2 performed the same calibration and decoding routine for three single day experiment sessions with the 1 of 10 posture set. Across all sessions, she achieved an online accuracy of 80.0% with a latency of 344±924 ms (Fig. 2d,f). P2 was able to make corrections and maintain successful holds on 89.3% of her trials. She had the most difficulty performing finger adduction, thumb opposition, and relaxing to rest with the virtual hand.

Fig. 2.

Decoding individual finger and wrist postures. Participants controlled the virtual hand in real-time with the HMM-NB to match 10 postures: flexion of all five fingers (T,I,M,R,S), wrist flexion (WF), finger abduction (Ab), finger adduction (Ad), thumb opposition (TO), and rest (Re). (a) Simulated offline performance during rest and hold periods of P1’s training data (5-fold cross validation, 5–6 repetitions per movement). (b) Online accuracy shows transition errors to cued postures while P1 controlled the virtual hand. (c) Trial latency histograms binned in 50 ms increments and overlaid with the median (dashed line) and middle 50% (shading) of trials (n = 30). Trials with latency greater than a second (>1) are aggregated in the orange rectangle. (d) Offline simulation for P2 using training data from one experiment session. (e-f) Same as above for P2 across three single day sessions (n = 181 trials). Unsuccessful trials (F) are aggregated in the red rectangle.

Unsurprisingly, online performance improved with a reduced number of postures (Fig. 3). Both P1 and P2 completed the same set of experiments using the HMM-NB to rapidly switch between five postures: three functional grasps, finger abduction, and rest (Movie 2). Offline accuracy was relatively low for P2, notably the HMM-NB issued incorrect predictions of rest during simulated holds. However, when controlling the virtual hand in real-time, P1 and P2 achieved online accuracies of 99.5% and 96.3% across one and three experiment sessions respectively. Both participants were able to recover from errors and maintain successful holds on 100% of their trials. Trial latencies were lower as well: 96±30 ms and 173±151 ms (median±i.q.r.) for P1 and P2 respectively. To compare to previous studies that classified finger and hand postures, movement completion metrics were calculated ignoring wrist flexion and rest trials (Table II).

Fig. 3.

Decoding hand grasps. A fewer number of grasp postures could be predicted in real-time with higher online accuracy and lower latency. (a) Simulated offline performance of the HMM-NB distinguishing fist (F), pinch (Pi), point (Po), finger abduction (Ab), and rest (Re) for P1 (5-fold cross validation, seven repetitions per movement). (b-c) Online accuracy and decoding latency for P1 (n = 26 trials). (d) Offline simulation for P2 using training data from one experiment session (five trials per movement). (e-f) Same as above for P2 across three single day experiment sessions (n = 79 trials). Trials with latency greater than a second (>1) are aggregated in the orange rectangle.

TABLE II.

Comparison to Previous Work

Study	Participants	EMG Chans.	Hand Classes	Completion Delay (s)	Completion Rate (%)
Kuiken et al. 2009	3 SD, 2 TH	12 surface gelled	4	0.54±0.27	86.9±13.9
Li et al. 2010	5 TR	12 surface gelled	6	0.45±0.35	53.9 ±14.2
Cipriani et al. 2011	5 TR	8 surface gelled	7	0.86*	77*
Vaskov et al. 2022	2 TR	8 intramuscular	8	0.26±0.09	96.3±5.3
			4	0.14±0.06	100±0.0

Three earlier studies quantified real-time classification including multiple hand (finger or grasp) movements for persons with shoulder disarticulations (SD), transhumeral (TH), or transradial (TR) amputations using similar randomized control tasks. Movement completion metrics for hand postures were calculated consistent with previous work. The task in those studies differed from the posture switching task in two ways: a rest period was presented in between cues and the completion requirement was a cumulative selection instead of a continuous hold. Completion rate and time were defined as the percentage of trials and median time in which one second of the correct posture was cumulatively matched. Completion delay is presented as the difference between the reported completion time and the selection length, which varied between earlier studies. By nature, completion rate is greater than or equal to success rate and completion delay is less than or equal to trial latency. Metrics were averaged across subjects (mean±s.t.d.)

^*variance across subjects not reported.

B. Robustness Across Arm Positions

P2 performed 20–22 trials of the virtual posture switching task in eight static arm positions (Fig. 4a, Movie 3). The HMM-NB used for this exercise was trained with her arm and prosthesis resting on the table. Classifier performance remained robust across the majority of the arm positions where transition errors were infrequent and quickly corrected (Fig. 4b,c). In six of eight arm positions, P2 was able to control the virtual hand with online accuracy never dropping below 96% and a trial latency of 173±113 ms (median±i.q.r). In two arm positions, arm raised and behind the back, there was a noticeable increase in transition errors to the cued posture, which occurred 7 and 6 times respectively. These positions at the superior and posterior extremes of her workspace had lower online accuracy and a higher trial latency of 301±299 ms. Even though there were more frequent errors in these two positions, the classifier did not completely fail and P2 was able to recover and achieve a successful hold on 100% of trials. During preliminary testing and demonstrations (Movie 4), P1 was able to activate the same set of postures using the LUKE arm in three arm positions. P2 also used a six posture HMM-NB to activate individual flexion of fingers on the i-Limb and return to rest in three arm positions. She correctly executed 14 out of 15 flexion attempts.

C. Functional Prostheses Use

Both participants used the functional posture set to complete five trials of the SHAP-inspired task (Fig. 5, Table III, Movie 5). Each trial consisted of five object interactions and required the activation of three different grasps. Participants were ructed to use specific grasps for the timer and each SHAP object. Performance was evaluated by trial time and two qualitative metrics. A sub-task was counted as complete if it was performed in a single attempt. However, it was possible to complete tasks without using the instructed grasp. The number of sub-tasks where the correct grasp was achieved without multiple activations was also counted. P1 used the LUKE arm to move SHAP heavy objects and completed the task with an average time of 18.75±3.42s (mean±s.t.d.). P2 used the i-Limb Quantum^™ to move SHAP light objects and completed the task in 36.60±7.66s on average. P2 completed the objects in the reverse order. This made it easier for experimenters to assist with manual wrist adjustment for the power object. Both participants were largely successful, completing 23 out of 25 sub-tasks without dropping an object or failing to press the button. Additionally, P1 and P2 performed the instructed grasp during their first EMG activation attempt on 20 and 19 out of the 25 sub-tasks. The main issue P1 encountered was an inability to use point for timer stop. Interestingly, this error never occurred at trial start indicating the point could be activated under the right conditions. Both participants had instances where they led with a pinch to pick up the sphere. P1 seamlessly corrected and engaged the instructed fist grasp in all but 1 trial. P2 needed to perform additional EMG activations for correction due to the state machine coded for the i-Limb. This and the need for manual wrist adjustment contributed to the variation in completion times between participants.

Fig. 5.

SHAP inspired functional assessment. Both participants used their prostheses with the functional posture set to perform a task in which they were instructed to use specific grasps to interact with three objects and a timer. (a) P1 used the LUKE arm and was instructed to use point to start the timer, fist for sphere, fist for power, and pinch for extension objects, before using point to stop the timer. (b) P2 used an i-Limb Quantum XS and completed the object trio in the reverse order. Completion time (mean±s.t.d, n = 5 trials) included manual wrist adjustments for P2.

TABLE III.

Functional Assessment Results

P1	Button Start	Sphere	Power	Extension	Button Stop	Total
Sub-task Complete	5/5	4/5	4/5	5/5	5/5	23/25
Correct Grasp	5/5	4/5	5/5	5/5	1/5	20/25
P2	Button Start	Extension	Power	Sphere	Button Stop	Total
Sub-task Complete	5/5	5/5	5/5	3/5	5/5	23/25
Correct Grasp	5/5	3/5	4/5	3/5	4/5	19/25

Sub-tasks were counted as complete if the participant succeeded on their first movement attempt, and correct grasps were counted when the instructed grasp was achieved on the first EMG activation attempt.

D. Signal Strength and Specificity

The bipolar intramuscular electrodes recorded specific and spatially segregated EMG activity particular to individual finger movements. To visualize posture segregation, we used linear discriminant analysis (LDA) to define the top two discriminating dimensions for the 1 of 10 posture set (Fig 6). For this analysis, MAV from the eight bipolar electrode pairs were processed with time history parameters used for real-time control in the Appendix. Rest data was excluded to focus on movement distinctions. Overall, posture holds were well separated in the low dimensional space on real-time and trial-averaged timescales. For P1, thumb movements were the most distinguished and separated along the first discriminant axis. For P2, wrist flexion was the best separated posture, along with flexion of the thumb or index finger. These movements were amongst those explicitly targeted by electrode implantation and were well represented by individual channels (Fig. 11 and Fig. 12). Additional discriminating dimensions would further separate movements, with 4-D embedding capturing approximately 90% of the variance in training data.

Fig 6.

Dimensionality reduction for posture visualization. MAV from eight channels reduced to two discriminant dimensions found using LDA. Large outlined points indicate trial averages, while small dots represent individual timesteps. Dashed boxes magnify separation of relatively close movements. (a) MAV for P1 was processed in non-overlapping 50 ms time bins. (b) MAV for P2 was processed in a sliding 200 ms window updated every 50ms. (c) Cumulative variance explained by additional dimensions.

Fig. 11.

MAV traces from P1’s 1 of 10 calibration session used for offline dimensionality reduction (Fig. 6). EMG was rectified and averaged in non-overlapping 50ms time bins. Trials were time aligned so t = 0 is the start of the posture hold and then averaged (mean±s.t.d. n = 5–6 trials).

Fig. 12.

MAV traces from P2’s 1 of 10 from the calibration session used for offline dimensionality reduction (Fig. 6). EMG was rectified and averaged in 200ms time bins with a 50ms update rate. Trials were time aligned so t = 0 is the start of the posture hold and then averaged (mean±s.t.d. n = five trials).

In addition to high specificity, the implanted electrodes also recorded large amplitude responses with a low noise floor (Fig. 7). For P2, we compared the SNR of five intramuscular channels to simultaneously recorded surface signals. Bipolar surface recordings were acquired using adhesive gelled electrodes (Biopac, Goleta, CA) connected to the same signal processing equipment. For each targeted muscle, five primary movement repetitions were averaged together to calculate SNR (Table IV). Across the compared channels, the average SNR for implanted electrodes was 105.4±82.6 and 152.5±138.6 (mean±s.t.d) gain for P1 and P2. P1 did not participate in a surface session, while P2’s surface signals averaged an SNR of 8.5±6.6. A high surface SNR of 19.6 was observed during finger abduction which prompted a full splay of the hand. We recorded this movement with surface electrodes targeting extensor pollicis longus (EPL). However, it is possible we also recorded activity from extensor digitorum communis (EDC) or other nearby muscles. Deep muscles that control individual finger movements were more difficult to capture from the surface. Gelled electrodes targeting flexor digitorum profundus of the index finger (FDPI) recorded signals with an SNR of 2.26 during index finger flexion. The SNR for simultaneously recorded implanted electrodes was 244 for EPL and 66 for FDPI. We also measured SNR for electrodes implanted into RPNIs, however attempts to accurately target and record matching surface EMG were unsuccessful. By nature, RPNIs also do not have a direct functional correspondent, so residual muscles that control related movements were selected for comparison. Across all the comparisons conducted with P2, implanted electrodes provided a 6 to 30-fold SNR improvement. The Appendix further details the methods for this comparison and the below classifier simulations.

Fig. 7.

(a) Example SNR (gain) for EPL while P2 performed finger abduction. The individual trial with highest surface SNR is shown. (b) Same for FDPI during index flexion. Intramuscular (IM) and surface EMG were simultaneously recorded such that the movement cues for each row occurred at the same point in time.

TABLE IV.

Signal-to-Noise Ratios

Movement	Thumb Flexion	Small Flexion	Index Flexion	Wrist Flexion	Finger Abduction
IM Channel	Median RPNI	Ulnar RPNI	FDPI	FCR	EPL
P1 IM SNR	251	54.1	56.4	88.5	77.2
P2 IM SNR	23.0	79.7	66.0	350	244
Surface Target	FPL	FDPS	FDPI	FCR	EPL
P2 Surface SNR	5.33	6.65	2.26	8.82	19.6

SNRs were calculated for implanted channels by averaging the active and rest RMS voltages across five trials of the individual finger movement most relevant to the targeted muscle. For P2, simultaneous surface recordings were obtained by individually targeting implanted residual muscles. For RPNI comparisons, appropriate residual muscles were selected based on the cued movement.

E. Alternate Classifier Simulations

The HMM-NB models underlying state transitions to rapidly issue accurate predictions. Classification stability and accuracy can also be improved by adding time history and additional informative features. Fig. 8 simulates the ability of the HMM-NB, a standard Naïve Bayes (NB) classifier, and three alternate classifiers to distinguish the 1 of 10 posture set with different window lengths. The alternate classifiers used 5-time domain features and 6th order auto-regressive coefficients to characterize EMG from each channel. Increasing window length did not create multiple time points for input, preventing alternate classifiers from modelling EMG patterns over time.

Fig 8.

Offline performance of different algorithms and window lengths on the 1 of 10 posture set. The HMM-NB and a standard sNaïve Bayes classifier (NB) used only mean absolute value (MAV) as an input feature. An alternate NB implementation, linear discriminant analysis (LDA), and a support vector machine (SVM) used five time domain features and 6th order auto-regressive coefficients per channel. Classifier accuracy was evaluated across individual timesteps during hold and rest periods (5-fold cross validation). Dashed lines for P2 show performance using eight bipolar surface electrodes. Performance with P2’s implanted electrodes (mean±s.e.m) was evaluated across simultaneously recorded intramuscular EMG and three calibration sessions.

P1’s alternate classifiers improved performance with longer windows, however this difference was not robust for P2 across four datasets. Additional features proved beneficial when applied to P2’s surface EMG, and increased NB performance by 17.8±10% (mean±s.t.d.) across all window lengths. For implanted electrodes, the performance difference between NB using only MAV and each alternate classifier was not significant (p > 0.2, paired t-test, n = 35 window lengths across 5 datasets). The HMM-NB consistently improved simulated performance over NB, and overall outperformed each alternate classifier (p < 0.01, paired t-test, n = 42 window lengths across 6 datasets). The HMM-NB most noticeably improved performance for smaller processing windows, which can increase responsiveness. An online comparison between the NB classifier and the HMM-NB is detailed in Appendix. Both participants used the NB classifier to complete the 1 of 10 posture switching task with an average success rate of 97.3% and trial latency of 328 ms. These results indicate that a single-state classifier can still yield good performance.

VI. DISCUSSION

In this study, implanted electrodes recorded high-quality EMG signals from RPNIs and residual innervated forearm muscles in two persons with transradial amputations. The implant procedure targeted the same individual finger movements in both participants who were able to control a virtual hand to distinguish the same individual finger, intrinsic, and grasp postures. The posture switching task tested real-time control by prompting users to directly switch between postures and rest at a fast pace. The speed and accuracy of our pattern recognition system, to distinguish finger movements, exceeds earlier work which quantified real-time performance in virtual environments (Table II). In a controlled environment, the HMM-NB also distinguished a smaller set of functional postures in novel static arm positions. P2’s performance was consistent across most of the workspace, which we attribute to the stability of signals from the implanted electrodes. Finally, participants used the high speed pattern recognition system to control advanced robotic prostheses, eliminating the need for time consuming grip triggers or selection schemes.

A limitation of the functional prosthetic controller implemented here is a lack of intuitive and continuous speed modulation across flexion and extension movements. This capability allows for fine adjustments is likely necessary for more realistic activities. One approach is to proportionally map the collective EMG activation across indwelling channels to motor speed [37], [38]. Although approaches that automatically weight individual muscle and nerve signals may provide more accurate estimates of intended kinematics [32], [39]. For P2, our pattern recognition system was adapted to work with commercially available grip selection software. Enforcing the grip entry/exit structure can reduce error occurrences, but increases the effort required for corrections. The ability to move directly between hand postures allows for object manipulations. For example, P1 handled the SHAP sphere with the pinch and fist grasps. This additional dexterity may improve outcomes in more realistic activities as well [21].

Pattern recognition is a practical approach for intuitive grasp control because it is computationally inexpensive and can predict an increasing number of movements relative to input channels [8], [17], [18]. For example, in Vu et al., P1 was able to activate rest and four finger postures using only two EMG channels [32]. However, it can be challenging for classifiers to generalize to new contexts. In both participants, arm movements occasionally produced unintentional extensor activity and changing EMG patterns during object interactions led to some misclassifications. Ignoring unreliable channels was effective for this study, but could limit the number of predictable grasps compared to more robust classifiers [16]. Techniques, such as osseointegration, that improve mechanical coupling of the prostheses can reduce the effects of prostheses weight for more reliable control. Osseointegration has mostly been used in transhumeral cases, with recent investigation of transradial configurations [10], [40].

Our signal processing and feature extraction technique provides a stable proxy for local motor unit activity of residual muscles and peripheral nerves via RPNIs [30], [32]. Despite being sufficient for prosthetic control, our feature extraction and decoding latency exceeds intact neuromuscular dynamics [11], [41]. Decomposition techniques have been shown to extract individual motor neuron activity from EMG with less than two ms delay [42], [43]. Instead of pattern recognition, regression algorithms or biomechanical models can translate motor activity to simultaneous estimates of intended joint kinematics or torques [22], [25], [44]. These controllers can provide intuitive control of multiple DOF and may be more robust to different contexts [45]. The capabilities of our approach and these other methods should also be studied in patients with amputations above the elbow. These cases account for 30% of major upper-limb amputations and little information is available from remaining muscles to control active wrists and/or elbows in addition to multiple hand functions [46].

The peripheral nervous system contains well separated signal pathways to facilitate human-robot interactions [41]. Methods that reliably read motor activity from the nervous system may increase functionality of other prosthetic and assistive devices. Myoelectric control of lower limb prostheses has shown potential in various pilot studies, but few have investigated higher resolution neural interfaces to augment motor control [47]. Similar interfaces may one day benefit patients with functional weakness by allowing them to control assistive exoskeletons [48], [49]. In these applications, neural inputs may be integrated with mechanical sensors as part of an overarching control architecture, instead of directly decoding motor unit activity to control prosthetic joints [47], [50]. Developing implantable electronics to translate invasive technologies to clinical use is becoming increasingly practical. The signal processing chain presented here is amenable to a low-power architecture [51]. For persons with amputations, osseointegration provides a percutaneous feed-through for intramuscular electrodes [10]. Implantable wireless devices have also gained traction for neuromodulation therapies [52], [53]. Modifying these approved devices may be a cost-effective method to deploy fully implanted systems in the near future.

For effective human-robot interactions sensory feedback can be just as important as the forward motor path [54]. For example, in this study even though thumb opposition and finger adduction were accurately predicted offline, P2 had difficulty activating these postures during real-time control. P2 reported limitations on proprioceptive feedback in her phantom limb, notably including her thumb metacarpal phalangeal joint. Additional practice could resolve this issue [55]. However, a bi-directional interface with improved feedback mechanisms [56]–[59] may be necessary for some patients to realize the potential of improved motor control. Overall, increasing prostheses responsiveness, precision, and feedback can increase confidence, improve device embodiment, or reduce phantom limb pain in some users [10], [60]. The stable and highly specific EMG afforded by indwelling electrodes can play a significant role in improving prosthetic control and user satisfaction in the coming years.

Biographies

Alex K Vaskov is a graduate student at the University of Michigan. He received a B.S. in mechanical engineering from the Massachusetts Institute of Technology, Boston, MA in 2012. From 2012 to 2016, he worked as a software developer and consultant for Accenture. In 2018, he received his M.S. in Robotics from the University of Michigan, Ann Arbor, MI. His research focuses on using peripheral interfaces to control robotic hands.

Philip P. Vu (S’ 17) received a B.S. degree in biomedical engineering from the University of Florida, Gainesville, FL, USA, a M.S.E degree in biomedical engineering from the University of Michigan, Ann Arbor, MI, USA, and is currently working toward the Ph.D. degree in biomedical engineering at the University of Michigan. Mr. Vu is a recipient of the National Science Foundation Fellowship, and his research focuses on extracting peripheral neuro-prosthetic control signals to improve fine motor skills and finger movements.

Naia North is a certified prosthetist and was previously the residency program director and lead. She is currently a research and development engineer at Medtronic. She received her B.S. in mechanical engineering from the University of Michigan, Ann Arbor, MI in 2020.

Alicia J. Davis has been a clinical prosthetist and orthotist since 1991 and was the residency program director and lead prosthetist in the upper extremity prosthetic program at University of Michigan Orthotics and Prosthetics Center. She taught at Eastern Michigan University’s Master of Prosthetics and Orthotics Program. Her research interests are focused on upper extremity prosthetics. She has also served as the president of the American Academy of Orthotics and Prosthetics and as board member of the National Commission of Orthotic and Prosthetic education.

Theodore A. Kung is a Clinical Assistant Professor in the Section of Plastic Surgery at the University of Michigan. Dr. Kung received his medical degree from Case Western Reserve University. He then completed an integrated plastic surgery residency at the University of Michigan. In addition, he also pursued fellowship training at the University of Washington in Seattle to specialize in microsurgical reconstruction, breast reconstruction, and lymphedema surgery.

Deanna H. Gates is an associate professor of Movement Science and director of the Rehabilitation Biomechanics Laboratory (RBL) at the University of Michigan School of Kinesiology. She is also an associate professor of Biomedical Engineering in the College of Engineering and Medical School. Prior to her time at U-M, Dr. Gates was a research biomechanist, then site supervisor at the Brooke Army Medical Center in Fort Sam Houston, TX. She completed her PhD in Biomedical Engineering at the University of Texas at Austin. She is an associate editor for IEEE Transactions on Neural Systems and Rehabilitation Engineering. Her research interests include biomechanics, rehabilitation, prosthetic and orthotics, control of repetitive movements, and nonlinear dynamics.

Paul S. Cederna is the Robert Oneal Professor of Plastic Surgery, Chief of the Section of Plastic Surgery, and Professor in the Department of Biomedical Engineering at the University of Michigan, Ann Arbor, MI, USA. By combining his clinical training in general surgery, microsurgery, and plastic surgery and background in biomedical engineering, he is able to incorporate creative solutions to solve the most difficult clinical problems. He directs the Neuromuscular Research Laboratory at the University of Michigan and has been the Chairman of the Plastic Surgery Research Council, President of the American Society for Peripheral Nerve, is on the Board of Directors of the American Board of Plastic Surgery, and is currently President-Elect of the Plastic Surgery Foundation.

Cynthia A. Chestek (S’04-M’10) was born in Erie, PA, USA. She received the B.S. and M.S. degrees in electrical engineering from Case Western Reserve University, Cleveland, OH, USA, in 2005, and the Ph.D. degree in electrical engineering from Stanford University, Stanford, CA, USA, in 2010., From 2010 to 2012, she was a Research Associate at the Stanford Department of Neurosurgery. In 2012 she became an Assistant Professor of biomedical engineering at the University of Michigan, Ann Arbor, MI, USA. Her research interests include high-density interfaces to the nervous system for the control of multiple degree of freedom hand and finger movements.

APPENDIX

A. Hidden Markov Model Performance Comparison

The HMM-NB was compared to a standard Naïve Bayes (NB) classifier using the virtual task to switch between postures for the 1 of 10 and grasps sets (Table I). Table V shows the parameters for each decoder. Parameters were selected ad-hoc by completing preliminary control tests, decreasing the window size and filter length until instabilities prevented task completion. Both the NB and HMM used the same calibration data for each session. P1 completed one A-B session with the NB (A) and HMM-NB (B) classifiers for the 1 of 10 and grasps posture sets. P2 completed three sessions for posture set: one A-B, one A-B-A, and one B-A-B session for 1 of 10 and three A-B-A sessions for grasps. Fig. 9 and Fig. 10 show the results of the NB classifier which can be compared to the HMM-NB from Fig. 2 and Fig. 3. P1 completed the 1 of 10 posture set faster with the HMM-NB, achieving a latency of 159±237 ms compared to 258±313 ms (median±i.q.r.). However, the NB was more stable evidenced by a higher online accuracy, 95.9% compared to 93.0%, and fewer trials with latency greater than one second. NB achieved a high offline grasp accuracy for P1 but a lower online accuracy of 83.7% compared to 99.5% for the HMM-NB. The increase in transition errors contributed to a higher latency of 280±251 ms compared to 96±30 ms for the HMM-NB. P2’s NB classifier had a median latency of 398±1246 ms compared to 344±924 ms with the HMM-NB on the 1 of 10 posture set. Transition errors were worse as she achieved an online accuracy of 75.7% with NB compared to 80.0%. However, the NB classifier was able to return to rest more effectively than the HMM-NB. For her grasps posture set, the NB classifier was far less responsive with a latency of 355±1184 ms compared to 173±151 ms. Transition errors were also much worse with an online accuracy of 81.5% compared to 96.3% with the HMM-NB.

TABLE V.

Decoding Parameters for Real-Time Control Comparison

Participant	Task	Decoder	Time Hist. (ms)	Update Rate (ms)	Filter Length
P1	1 of 10	HMM	50	10	5
		NB	50	50	3
	Grasps	HMM	50	10	5
		NB	50	50	3
P2	1 of 10	HMM	50	50	1
		NB	200	50	4
	Grasps	HMM	50	50	1
		NB	200	50	4

Time history refers to the length of the processing window to extract MAV from all eight bipolar electrode pairs, while update rate refers to the timestep features and decoders were updated. The filter length is the number of consecutive decodes required for a change to be sent to the virtual prostheses.

Fig. 9.

Performance of the Naive Bayes (NB) classifier on the 1 of 10 posture set. (a) Simulated offline performance during rest and hold periods of P1’s training data (5-fold cross validation). (b) An output filter of three consecutive decodes was used for real-time control and P1 achieved an online accuracy of 95.9%. (c) Cumulative latency lines were drawn so the y-axis indicates the percentage of trials with latency less than values on the x-axis (n = 30 trials, 27 shown). Results compared to the HMM-NB from Fig. 2. (d,e) P2’s decoder used larger processing windows and a filter length of four consecutive decodes for real-time control. (f) Cumulative latency comparison for P2 (n = 168 trials, 114 shown).

Fig. 10.

Performance of the Naive Bayes (NB) classifier on the grasps posture set. (a) Offline classifier accuracy simulated on P1’s training data. (b) An output filter of three consecutive decodes was again used for real-time control. (c) Cumulative latency comparison for P1 (n = 26 trials, 20 shown). Results compared to the HMM from Fig 3. (d,e) P2’s decoder again used larger processing windows and a filter length of four consecutive decodes for real-time control. (f) Cumulative latency comparison for P2 (n = 144 trials, 102 shown).

The HMM-NB particularly excelled in distinguishing the grasps posture set. The ability to represent a posture as multiple states could be a greater advantage for predicting compound finger movements. The incomplete selection of latent states for output was problematic for P2’s 1 of 10 decoders which sometimes got stuck in transition states while returning to rest. Alternate output mappings could be explored to mitigate this issue. The Naive Bayes assumption of independence within the HMM-NB meant that individual states could not represent complex phenomena, but also meant the model was not prone to over-fitting [61]. However, expectation-maximization algorithms can settle into local minima. The HMM-NB was sensitive to the initialized state structure. Therefore, we cannot say our specific implementation was the best model, rather one that worked well. The number and structure of latent states per posture was driven by the computational requirements of the real-time system, ease of initialization, and success in prior applications [34], [62]. Relaxing the initialization routine could allow the HMM-NB to find to more optimal solutions. Although this may require more training data. As noted elsewhere, assembling large historical data-sets is feasible due to the stability of implanted electrodes over time [25], [32].

B. Methods for Alternate Classifier and SNR Comparisons

The 1 of 10 posture set was chosen for alternate classifier simulations because it required distinction between the most movements, some of which were well represented in electrode placement and some of which were not. The algorithms used for real-time control were compared offline to a Naive Bayes (NB), linear discriminant analysis (LDA), and a multi-class support vector machine (SVM) using five time domain features (mean absolute value, waveform length, variance, slope sign changes, and zero crossings) along with coefficients from a 6th order auto-regressive model. Alternate classifiers assumed equal prior probabilities for each posture and used default Matlab 2018 built-ins for training and evaluation. LDA used a diagonal-regularized pooled covariance matrix. The multi-class SVM used a one-vs-one architecture with a linear kernel. Performance was evaluated on calibration data with different sized processing windows containing 10, 25, 50, 100, 150, 200, and 250ms of EMG history. P1’s HMM-NB was updated in 10ms timesteps, to match his online implementation. All other classifiers were updated every 50ms.

P2 completed additional sessions to compare the performance between implanted and surface EMG. For the classifier simulation, eight pairs of adhesive electrodes were placed on P2’s residual limb. Surface muscles corresponding to implanted electrode functions were targeted by feeling P2’s forearm while asking her to perform movements with her phantom limb. The size of adhesive electrodes resulted in most of her medial forearm being covered. EMG from both the surface and implanted electrodes was simultaneously recorded while P2 performed a calibration run for the 1 of 10 posture set. In a separate session we more precisely targeted FCR, FDPS, FPL and EPL using established techniques [63]. For targeted sessions, muscles were recorded individually to avoid space constraints. Before each recording, P2’s forearm was cleaned with alcohol wipes and allowed to dry before applying the gelled electrodes. Signals from the corresponding implanted electrode pair were recorded for a simultaneous comparison. The same calibration routine was used to instruct movement cues that corresponded to the muscles’ motor functions. SNR’s were calculated by averaging the RMS voltage of active periods and dividing by the averaged RMS of rest periods. Rest periods sometimes began with EMG settling activity from the previous trial. This was particularly noticeable for some of P2’s surface channels as well as her FDPI and EPL implanted electrodes. Settling activity was manually removed for SNR calculation, but not for classifier training because it is important to the characterization of a rest intention. SNRs were calculated for both the targeted and classifier simulation datasets. For each movement and muscle pair, the session with the better surface SNR was presented in Table IV. For FCR and FPDS, the targeted sessions yielded better results. EPL, FDPI, and FPL comparisons were taken from the classifier simulation dataset. Examples of P1 and P2’s calibration EMG are available here: https://deepblue.lib.umich.edu/data/concern/data_sets/nv9353227.