Application of EMG feedback for hand prosthesis control in high-level amputation: a case study

Abstract

EMG feedback improves force control of a myoelectric hand prosthesis by conveying the magnitude of the myoelectric signal back to the users via tactile stimulation. The present study aimed to test if this method can be used by a participant with a high-level amputation, and whose muscle used for prosthesis control (pectoralis major) was not intuitively related to hand function. Vibrotactile feedback was delivered to the participant’s torso, while the control was tested using EMG from three different muscles. The participant completed four experimental sessions of a force-matching task with a prosthesis. The performance was evaluated by computing the target force success rate. The results of session 1 showed that the participant could effectively employ EMG feedback after only brief training. Session 2 demonstrated that EMG feedback benefited force control, increasing the success rate by approx. 30%. Finally, after proper training (sessions 3 and 4), the participant’s performance when using the muscle on the amputated side was similar to that achieved with the muscles on the contralateral side. Overall, the study results indicate that EMG feedback can be used in high-level amputations, despite the extent of the injury and non-intuitive control.

Introduction

Traumatic loss of an upper limb has a significant impact on the quality of life¹. The hindered ability to perform daily tasks² is coupled with negative body image, reduced social participation³, and, commonly, phantom limb pain⁴. Modern myoelectric prostheses enable the restoration of some of the lost motor functions. Such devices are controlled using the electrical activity of a person’s muscles to generate commands that drive the device motors.

Prostheses intended for transradial amputees typically exploit the activity of residual forearm muscles, which were normally used for hand and wrist control, to command a hand prosthesis. A different approach must be adopted, however, in the case of higher-level amputees, such as those who have undergone a transhumeral amputation or shoulder disarticulation. In the latter case, for instance, arm muscles are absent; therefore, the muscles of the thoracic region and/or back need to be used for prosthesis control. In such a setup, the mapping between the muscles and prosthesis functions that they control is no longer intuitive. A surgical procedure known as targeted muscle re-innervation (TMR) can be used to improve the intuitiveness of control scheme by increasing the number of signal sources; however, this approach is not accessible or applicable in all cases⁵. For instance, the procedure is not recommended for patients also suffering from peripheral vascular diseases or those that are on anti-coagulant medication, due to a higher risk for complications⁶. Additionally, TMR may require multiple surgeries before the transportation of the nerves is successful⁷, while it has been reported that some amputees are unwilling to undergo invasive procedures and would rather opt for non-invasive approaches⁸.

The restoration of missing somatosensory feedback is another challenge when attempting to replace the function of the lost biological limb. Most often, this is achieved by reading information from the sensors embedded in the prosthesis, measuring, for instance, grasping force that the hand applies⁹. This information is then conveyed to the user by stimulating remaining sensory structures electrically or mechanically^10–12. Besides force, other variables, such as closing velocity, aperture, and touch onset have been conveyed to improve performance and user experience¹³. However, despite promising results, providing effective feedback that will be useful in realistic conditions when the user also receives incidental information (e.g., motor sound and visual cues) is still an open challenge¹⁴.

To address this challenge, an alternative feedback method named “EMG feedback” was proposed¹⁵. In this approach, tactile stimulation is used to convey the strength of the user’s own muscle contraction rather than the data measured by prosthesis sensors to characterize its physical state. The same signal, namely the myoelectric command, is therefore employed to both drive the prosthesis and generate the feedback. This approach can be viewed as a way to re-calibrate as well as amplify the natural proprioceptive feedback from the muscles. EMG feedback assists the users in modulating their muscle contraction to a precise and desired level, which leads to the application of the desired grasping force. Contrary to the more commonly used force feedback, which is only activated after the prosthesis applies force on an object, EMG feedback is available as soon as the users activate their muscles. Therefore, the key advantage of this method is that it enables predictive control of the grasping force^15,16.

Most studies have investigated artificial sensory feedback for transradial amputees, while only a few tests have been conducted on participants with high-level amputations. Mann and Reimers presented one of the first attempts where information about the elbow angle of a prosthetic arm was conveyed by modulating the intensity of two vibration motors, thereby creating a sensation moving along the residual limb of a transhumeral amputee¹⁷. Similarly, a recent study¹⁸ used a multichannel vibrotactile interface with spatial encoding to convey elbow angle to transhumeral amputees controlling a virtual arm prosthesis. A bidirectional direct neural interface for the control of an osseointegrated transhumeral prosthesis was described in which the feedback was provided invasively by electrically stimulating peripheral nerves¹⁹. Vibrotactile feedback was also applied in participants with shoulder disarticulation to produce tactile sensations corresponding to the contact and grasping force as well as to create a realistic kinesthetic illusion by activating muscle afferents²⁰.

In all previous studies investigating EMG feedback^15,16,21,22, the setup was designed for transradial applications. The EMG was recorded from the forearm muscles to generate the prosthesis commands, while the feedback was delivered to the lower or upper arm using electro- or vibro-tactile stimulation. As noted before, this setup is simple and intuitive because the forearm muscles are normally used to grasp and modulate the force when handling objects in daily life using an intact hand. However, it has not been tested before and is, therefore, unknown if EMG feedback can be successfully translated to a case of high-level amputation, where the mapping between the muscle (e.g., the pectoralis major) and the function it restores (hand opening/closing and grasping force) is not natural. In this case, the stimulation to implement the feedback needs to be delivered to body areas even further away from the end effector.

The present case study, therefore, explores for the first time the application of proportional myoelectric control with vibrotactile EMG feedback on a participant with a very proximal (high-level) transhumeral amputation with an osseointegrated implant (Fig. 1). We tested the prosthesis control performance while the participant received vibrotactile stimulation on the anterior and posterior side of his right shoulder using four vibrotactors. The feedback conveyed five distinct spatial activation patterns which corresponded to five discrete myoelectric signal ranges. The participant used EMG feedback to adjust his muscle contraction to one of the five levels already during the prosthesis closing. Due to the proportional response of a myoelectric prosthesis (higher EMG leads to higher grasping force), the specified level of EMG resulted in the corresponding level of grasping force.

Fig. 1.

The participant with a high-level amputation used EMG feedback for closed-loop control of prosthesis grasping force across four experimental sessions. EMG was recorded from the thoracic muscles on the amputated side (sessions 1 and 2) and the contra-lateral thoracic and forearm muscles (sessions 3 and 4). The recorded and processed EMG was transmitted as the command for proportional control of a myoelectric prosthesis as well as conveyed to the participant via tactile stimulation (EMG feedback). Two vibrotactors placed on the anterior and two on the posterior side of his right shoulder conveyed five distinct levels of EMG that, when produced by the participant, translated into five corresponding levels of grasping force.

To evaluate the efficacy of the system in this particular type of prosthesis user, we conducted four experimental sessions over the span of four months (Fig. 1). Session 1 introduced the participant to the control and feedback interface and assessed the immediate adoption and use of the closed-loop control. The EMG electrodes were placed on the participant’s pectoralis major on the side of the amputation. Session 2 used the same setup to investigate if the EMG feedback improved the control of prosthesis grasping force compared to the condition in which the feedback was not provided. Finally, sessions 3 and 4 compared the performance of EMG feedback when using other muscles for prosthesis control to investigate the impact of the amputation (pectoralis on the amputated side versus pectoralis on the intact side) and the intuitiveness of the mapping (thoracic muscles versus hand/wrist flexors of the intact left arm). In all sessions, the participant was required to close the hand and produce one of the five target force levels indicated on the computer screen (force-matching task). The percentage of successful force applications (i.e., a generated force equal to the indicated target force) was used to evaluate the participant’s performance and, by extension, the applicability of the closed-loop control interface.

Results

Generated myoelectric signals

Some examples of myoelectric signals generated by the participant are shown in Fig. 2 to illustrate the use of EMG feedback. The signals were recorded in session 2 where the participant performed 3 blocks of the force-matching task. The EMG electrodes were placed on the pectoralis major on the side of the amputation and the feedback was provided only in the second block. The EMG feedback conveyed five levels of muscle activation and the participant was asked to close the hand and produce the indicated level of grasping force. Three intermediate levels of force were used as the targets in the experimental task, namely, levels 2, 3, and 4, hereafter also referred to as low, medium, and high. Figure 2 shows the trials with level 3 as the target force. When the feedback was provided (Fig. 2B), the participant-generated myoelectric signals that were mostly contained within the range corresponding to level 3 (shaded area). Conversely, when grasping without feedback, in the first and third blocks (Fig. 2A,C), he often produced contractions that were too strong, resulting in the myoelectric signal shooting over level 3. Notably, the myoelectric signals show that the participant’s approach when using EMG feedback was to rapidly modulate his contractions to the target level. The participant successfully exploited the unique ability offered by the EMG feedback, which is to predictively control the desired force during prosthesis closing. Finally, the generated force traces are not shown because in most cases they closely followed the upward modulation of the myoelectric signals due to proportional control, as explained in the previous section.

Fig. 2.

Example myoelectric signals produced by the participant in Session 2 when the task was to close the hand and generate level 3 of target force. Panels A and C are the trials from the first and last blocks performed without feedback while B shows the trials from the second block executed with feedback. The myoelectric signals display several overshoots when the subject did not receive feedback (plots A and C), while the signals were contained within the target range (EMG level 3, shaded area) when feedback was active (plot B). The myoelectric signals are shown from the moment they surpassed the dead zone, until the moment the maximum force was applied in each trial, and they are aligned to the touch onset (vertical red line). The EMG traces presented were generated with the right (amputation) side pectoralis major of the participant.

Participant performance over sessions

Figure 3 displays the performance achieved in the force-matching task across sessions, overall and for each target force level. The bars represent all trials (100%), which are then segmented into those that were successful (blue) and those in which the participant generated higher (orange) or lower (yellow) force compared to the target. Immediately after being introduced to the concept of EMG feedback in session 1, the participant achieved a high success rate (87% overall). With a success rate of 67%, the medium force level was the most difficult for the subject, while he achieved a perfect score (100%) when reaching the high force level.

Fig. 3.

The performance achieved in the four experimental sessions. The success (blue), overshoot (orange), and undershoot (yellow) rates are displayed, overall and for each force target. The black dashed line connects the overall success rates across conditions (blocks) in sessions 2, 3, and 4 to emphasize the trends. In plots C and D, the “Right Pec” indicates pectoralis major on the amputated side. Low, medium, and high force targets in the panels A-D correspond to levels 2, 3, and 4, respectively.

The aim of session 2 was to establish if the feedback was beneficial for prosthesis control. To assess the baseline, the participant first performed a block of trials with feedback deactivated. The feedback was provided in the second block to assess its immediate impact, and then again deactivated in the third block, to evaluate the effect of feedback on short-term learning. The success rate without feedback in the first block was 42% and it did not greatly improve in the last block, where the overall success rate was 47%. When feedback was provided in the second block, the performance improved substantially for all levels, with the overall success rate reaching 73%. Across target levels, the trend was similar to that obtained in session 1, with the worst and best performance achieved for levels 3 and 4, respectively. Without feedback, however, level 2 was the most difficult to generate correctly. Interestingly, the performance for level 3 in the last block (53.3%) was higher compared to that achieved in the first block (33.3%) and similar to the success rate obtained in the feedback block (56.7%). There was a clear tendency for the participant to overshoot the target levels in the first block with no feedback and this did not change when the feedback was activated. However, when the feedback was again deactivated (last block), most failed trials at level 4 were due to undershoots rather than overshoots.

The goal of session 3 was to assess if the quality of closed-loop control was affected by the choice of the muscle to generate the control signal. We compared the amputated (right) and intact (left) side as well as intuitive (forearm muscles) versus non-intuitive (thoracic muscles) control. The participant displayed a similar performance when controlling the interface with the left and right pectoral muscles. However, the success rate was approximately 12% lower than the one obtained in the feedback block of session 2. Surprisingly, the performance dropped to only 40% when using the forearm muscles. In all conditions in session 3, the performance achieved at level 2 was low, particularly with the forearm muscles, when all level 2 trials were unsuccessful. The EMG traces in Fig. 4 show that this was at least partially due to an unusual pattern of muscle activation. Initially, the subject successfully adjusted the muscle contraction within the desired level but then abruptly increased the signal just before relaxing the muscles. This unexpected peak would then generate an increase in force, thereby producing the force overshoot (failed trial).

Fig. 4.

Example traces recorded in session 3, when reaching for target force levels 2 (plot A) and 3 (plot B), illustrating unusual behavior of the participant in this session. The subject correctly achieved the target EMG level but then inadvertently executed a contraction “impulse” (black arrow) while attempting to relax his muscles. The myoelectric signals are shown from the moment they surpassed the dead zone, until the moment the maximum force was applied in each trial, and they are aligned to the touch onset (vertical red line).

The protocol was similar in session 4, but this time, we paid special attention to correcting such behavior during the training runs and, consequently, the performance was considerably higher than in session 3. Nevertheless, there was a drop in the success rate in the last run, when using the right pectoralis for control (amputated side). Interestingly, the obtained success rate with this muscle is similar to the one achieved when using the same muscle in session 2 in the feedback condition. The mistakes in session 4 were solely due to overshoots, as seen in Fig. 3D.

Discussion

EMG feedback for prosthesis control was originally proposed and validated as a viable feedback strategy for users of transradial prostheses, who employ forearm muscles to generate command signals. As explained in the introduction, the myoelectric control interface is, in this case, intuitive since the muscles from which the EMG is recorded are naturally used for hand control. More specifically, those muscles are often used for precise control of grasping and fine force modulation and, hence, it can be assumed that they can be readily used in combination with online EMG feedback. The present case study explored the feasibility of translating this method to the case of an amputee with a high-level amputation. Overall, the obtained results are encouraging and indicate that this approach to closing the control loop enables improved control of prosthesis grasping force, even when used with muscle groups other than those of the forearm.

Experimental session 1 demonstrated that the participant was able to effectively exploit the EMG feedback for precise force application after only a brief familiarization. He could rapidly associate his muscle contractions with the feedback and, by extension, with the force applied by the prosthesis. The participant achieved excellent performance, which is indeed an encouraging outcome, especially given the extent of the participant’s injury and the placement of the setup on a body area that is not naturally associated with hand closing. This highlights the intuitiveness of the approach and its ease of use. In fact, the participant achieved the overall highest success rate in session 1. This result is promising but also biased by the smaller number of trials in session 1 compared to subsequent sessions, and the fact that more trials were performed with target level 3, which was the easiest to generate correctly.

The feedback patterns were easy to discriminate and interpret, despite the feedback being delivered to the thoracic region and the upper back, the latter being a skin area characterized by limited tactile acuity²³. Good spatial discrimination was enabled by placing the tactors both far apart and on different anatomical regions (anterior and posterior) to facilitate recognition. Namely, the stimulation on the front was associated with lower levels, while that on the back indicated higher levels. It is likely that a more compact placement (e.g., placing all tactors anteriorly) could be used without the loss of performance. However, this would require testing different configurations as well as training the participant to discriminate closely-placed tactors²⁴ but this was outside the scope of the present study.

Past studies have shown that EMG feedback outperformed the more conventional force feedback^15,16,22 when using forearm muscles for prosthesis control. The results of session 2 demonstrated that the participant with a high-level amputation could also exploit EMG feedback to improve the prosthesis force control compared to the condition in which no explicit feedback was provided (block 2 versus block 1). In block 1, the subject could rely only on his natural muscle proprioception (the sense of contraction strength) and incidental feedback from the prosthesis (motor sound, closing velocity, etc.) to generate the desired force levels. Although the utility of such sources is not to be underestimated^25–27, the participant’s performance was poor without feedback, likely due to inexperience and short exposure to the system (session 1). He consistently overestimated the muscle contraction strength he needed to produce, thus overshooting the force in the majority of the trials in the first run of session 2. In the second block, the feedback helped the subject to downscale the contraction strength and maintain it within the desired range, leading to the generation of the correct target force, hence increasing the performance in all target levels, especially in level 2.

It has been shown before that the feedback can be used as an effective instrument for prosthesis control training, as the feedback allows the participant to acquire an internal model of prosthesis behavior^28–30. After the participants form such internal models, the feedback becomes less relevant, and the subject can gradually switch to feed-forward control. This has been demonstrated for the conventional force feedback³¹, which, however, is activated only after the prosthesis has applied grasping force. Therefore, during prosthesis closing, the subjects can solely focus on the sense of muscle contraction, and they can use the information about the achieved grasping force to adjust their contractions across trials. Over time, this will allow them to build a mental map between the desired force and the required muscle activation (internal model), which can then enable the successful execution of the task using feed-forward control (when the feedback is deactivated).

Indeed, it has been shown³¹ that a short single-session training with artificial force feedback was enough for the participants to develop a feed-forward model, which allowed them to control the grasping force with a high success rate even after the feedback was removed. Based on this result, we expected the same outcome in Session 2, which, however, was not the case. Importantly, contrary to force feedback, EMG feedback was activated as soon as the participant initiated a muscle contraction and the feedback was then available until the muscles were relaxed, providing continuous online information about the level of the generated myoelectric signal. To exploit this information, the participant focused on the feedback instead of, for instance, on monitoring the sense of his muscle contraction (as indicated in a study by Strbac et al.³¹). It has been shown that continuous reliance on feedback can, in fact, impair the creation of forward models³², which could explain the substantial drop in performance observed in Session 2, when the feedback was deactivated (see Figs. 2C and 3B). Nevertheless, we cannot fully exclude that the lack of learning was due to the status of the patient, which was very different compared to the amputee subjects in Strbac et al.³¹.

Nevertheless, the use of feedback appears to have at least some impact across the blocks of session 2. The failed trials targeting level 4 were mostly due to undershoots rather than overshoots in the final block. Hypothetically, thanks to the feedback, the participant realized that his muscle contractions were too strong, thus adopting a more conservative approach in the third block when the feedback had been deactivated. However, the lack of feedback prevented him from being aware of the undershoots, resulting in a low success rate. Finally, the performance in level 3 might indicate some learning, as the success rate for this level was the same as in the feedback block (block 2) and better than in block 1 (no feedback).

Sessions 3 and 4 were conducted to assess if the injury and comprehensive surgical intervention have affected the participant’s ability to effectively use the control interface. As explained in the “Participant” Section, the pectoralis muscle on the amputated side displayed significant differences in volume and contraction strength, while strong tremors were noticeable whenever the subject contracted it. Nevertheless, in session 3, the performance using the pectoralis on the amputated side was similar to that obtained with the contra-lateral pectoralis (intact side) and better than that of the forearm muscles. However, the success rates in this session were generally lower compared to the previous tests. This was due to atypical behavior when reaching level 2, as explained in the “Results” Section.

In session 4, this was corrected during the training and, hence, in this test, the performance was equally high for the muscles on the intact side and, while somewhat worse, still good with the pectoralis on the amputated side. Therefore, EMG feedback is robust with respect to the muscle used for the control of the prosthesis, even if the source muscle is not naturally related to hand control (left pectoralis versus left forearm). The lower performance of the right pectoralis in this session might be an effect of the amputation, but we cannot fully exclude cognitive fatigue (as this was the last muscle tested).

Observing the results across sessions, there is no visible trend of improvement in force control performance when using the pectoralis muscle on the amputated side (Right Pec), either overall or per level. The only exception was the drop and then the recovery of success rate for level 2 in sessions 3 and 4, but this was due to the specific reasons explained above. The stable performance is an interesting outcome considering that the participant was undergoing a rehabilitation program. However, as explained in Methods, the rehabilitation focused mostly on integrating the implant rather than muscle control. Also, the current experimental protocol was not conceptualized to facilitate training and a higher number of shorter, more closely spaced sessions would likely have been more effective in that sense. Interestingly, there is a clear increase in performance on the non-amputated side (Left Pec and Left forearm) from sessions 3 to 4. We could not identify a specific reason responsible for this. Still, it could be due to multiple factors, for instance, familiarity with the task, faster learning with intact muscles or the specific order of experimental conditions.

In principle, the encouraging results obtained in the present study imply that the proposed setup could be used for early training in prosthesis control. As explained before, the experimental sessions were conducted while the participant was undergoing the rehabilitation protocol comprising the gradual loading of the osseointegrated implant. At the time of the tests, he was still far from ready to mount and use a prosthetic arm. Nevertheless, by performing the force-matching task with the prosthesis on the table, the participant could use the source muscle in a functional way. In addition, EMG feedback facilitated the controlled modulation of muscle contractions and provided insight into how the contraction gets translated into prosthesis grasping force. In that sense, it would be worth investigating the effect of EMG feedback training in the pre-prosthetic phase (as in the present study) on the future use of the prosthesis. EMG feedback makes the association between muscle activation and the generated force explicit, which might be useful when controlling the prosthesis later on, even if the feedback is no longer available.

Interfaces intended for use by people with a transhumeral amputation are routinely evaluated in studies with a single participant (or a small number of participants)^33–36. Despite the purely observational nature of the results, such experiments can provide valuable insight into these systems, regarding their applicability, efficacy, and potential for improvement. Importantly, the participant in this study is rather unique in terms of the extent of his injury, the reconstructive surgery he had undergone, as well as the placement of the osseointegrated implant. To the best of our knowledge, this is the first study that evaluated a proportionally controlled myoelectric prosthesis interface with EMG feedback on a transhumeral amputee. Due to the participant’s status, the closed-loop control scheme could not be tested with a prosthesis mounted on him, which could be a setup closer to the envisioned daily use. Nevertheless, we have shown that the proposed approach can be employed for effective training of closed-loop control in an early phase of rehabilitation when the participant is not yet ready to wear a prosthesis.

Assessing a single subject, even across multiple sessions, is a clear limitation of the present work. Importantly, in our recent studies^37,38, EMG feedback was validated using a clinically relevant setup (feedback in the socket) in participants with transradial amputation. The tests showed that when integrated into the socket and used to perform a functional task, the feedback indeed improved force control. This is an encouraging result implying that the performance improvement obtained in the well-controlled conditions of the present study would likely translate into the clinical use of a prosthesis by this and possibly other participants with high-level amputations. This, however, needs to be tested in future studies.

It would be therefore relevant to test EMG feedback on the same participant with the prosthesis attached to the body, as well as on other persons with high-level amputation who have undergone more conventional surgeries, similar to a recent study on users with transradial amputations^37,38. Here, a potentially challenging aspect could be to find a proper placement and an easy mounting of the tactors for high-level amputations with conventional sockets and/or osseointegration. We recently developed a system that allows fast prototyping of feedback for clinical testing as well as integration into the socket for long-term use at home³⁹. Presently, this system is employed to assess the long-term use of EMG feedback in the daily activities of people with transradial amputation, but it can be also applied in the case of higher-level amputations such as the one presented in this study. This is an important future step to confirm that the improved force control demonstrated in the present experiment would translate into real benefits during daily life application.

In summary, this case study is the first application of proportional control of a hand prosthesis based on EMG feedback in the case of a high-level amputee. Case studies like this can provide important insight and are common in the field^5,40–42. The promising results invite the design of further experiments that will thoroughly evaluate the system. The authors hope that the proof of concept described in this study will allow for the application of prosthesis interfaces integrated with EMG feedback across a larger pool of users than initially envisioned.

Methods

Participant

In 2017, the participant (male, age 54) had a severe accident, where his right arm had to be amputated. The amputated humeral stump was very short, and the wound was addressed with a latissimus dorsi flap (LDF) without reconstructing the axilla (see Fig. 5A,B). After discharge, he was rehabilitated with a harness-suspended myoelectric socket prosthesis with a close fit around the shoulder. However, the electrodes often lost connection due to heat and sweat under the socket, rendering the prosthesis non-functional. The participant had a decreased range of motion in the right shoulder joint, while the muscles around the shoulder as well as the pectoralis major were atrophic due to 4 years of disuse. It was decided to offer him an osseointegrated prosthesis consisting of a threaded titanium implant (fixture), and a skin-penetrating implant (abutment), which can be connected to an external prosthesis.

Fig. 5.

The snapshots from left to right show: (A) The amputation site prior to stage-one surgery; (B) Radiograph of the amputation site prior to stage-one surgery; (C) Radiograph of the osseointegrated implant in the subject’s residual humerus after stage-one surgery; (D) Radiograph of the implant and the attached abutment in the subject’s residual humerus; and (E) the amputation site with the osseointegrated implant and the abutment.

In January and May 2022, the patient underwent stage one (Fig. 5C) and two (Fig. 5D–E) surgeries, that would result in the placement of an osseointegrated implant into his residual humeral bone, which penetrated the skin and terminated in an abutment. In order to improve the mobility of the shoulder, scar tissue was surgically removed and replaced by a latissimus dorsi flap. Importantly, the pectoralis major muscle was left untouched. A full prosthetic arm would be attached to this abutment once the integration of the implant into the surrounding bone, induced by the controlled loading of the abutment, was deemed complete. At the time of the experiments, the loading phase was still ongoing and the participant had not received a full prosthesis or control training yet.

The participant was recruited for the experiment at this time, because it allowed us to investigate the application of EMG feedback early in the rehabilitation process, when the participant could not even use a prosthesis (pre-prosthetic phase), highlighting the potential for this approach to be used as an early form of training. More specifically, if the participant can benefit from this method, EMG feedback can help him understand the relation between his muscle contraction and the produced prosthesis force.

The participant signed an informed consent before commencing the experiment. The experiment was performed according to the Declaration of Helsinki as well as national guidelines and regulations as approved by the Research Ethics Committee of the Nordjylland Region under the number N-20190036 (approval date: 7/8/2019).

Experimental setup

The experimental setup comprised the following: (1) a multifunctional myoelectric hand prosthesis (Michelangelo, OttoBock, Duderstadt, Germany), with its proprietary controller, one dry EMG electrode with an embedded amplifier (12E200, OttoBock, Duderstadt, Germany) and a USB Bluetooth dongle, (2) four C2 vibrotactors and their control unit (Engineering Acoustics Inc., Casselberry, FL, USA), (3) a standard laptop (Lenovo ThinkPad P52, Intel Core i7 @2.60 GHz, 32 GB RAM), running Windows 10 Professional and an 18” computer monitor. A thermoplastic splint (ORLIMAN) was used for the immobilization of the subject’s left wrist (intact arm) to enforce isometric muscle contractions in the last two experimental sessions. The prosthesis was connected to the laptop via Bluetooth and the tactor control unit was connected via a USB port. The program that controlled the setup (prosthesis control and generation of vibrotactile feedback) was developed in MATLAB Simulink 9.3, using the toolbox for closed-loop human-manual control⁴³.

The active electrode provided a linear envelope of the recorded EMG, which was then sampled by the prosthesis controller and transmitted to the laptop at 100 Hz. Depending on the experimental session, the EMG electrode was placed above the right (amputation side) or left (intact side) pectoralis major or the flexor carpi radialis muscle of the left forearm, as seen in Fig. 6. The EMG electrode was placed over the pectoralis muscles, targeting the approximate location of their clavicular and sternocostal heads, and then probing until a recording site that generated a clear EMG signal was identified. Such placement was selected because this muscle will be used for prosthesis control post-rehabilitation due to the poor condition and short length of the residual limb. Before placing the electrode, the skin was treated with a small amount of conductive and abrasive gel. To record EMG from the pectoralis muscles, the electrode was secured with medical tape. When using the flexor carpi radialis of the participant’s left forearm (sessions 3 and 4), the muscle was identified by palpation and the electrode was placed directly over it and held in place with an elastic band around the forearm.

Fig. 6.

(A) The subject, seated in front of the Michelangelo hand, wearing the C2 tactors. The tactors were placed over soft tissue, avoiding the clavicle, coracoid process, and acromion of the scapula. The different placements of the EMG electrode used in this study are also shown. In the first two sessions, the electrode was placed over the right pectoralis muscle, while in sessions 3 and 4, it was positioned over the right and left pectoralis and the left flexor carpi radialis muscle. (B) The EMG and force traces the participant could see on the monitor during training. The horizontal lines are the thresholds for the different levels. The area below level 1 of EMG is the dead zone (see text).

The placement of the C2 tactors is shown in Fig. 6. Two tactors were placed anteriorly in the thoracic region, above the EMG electrode, and two posteriorly on the participant’s back, approximately over the right-side of the trapezius. The tactors were not placed directly above bones (e.g., the clavicle or the scapula). They were also placed sufficiently far apart from each other (at least 5–10 cm) to ensure good spatial discrimination, especially in the posterior region. The vibrations produced by the tactors were perpendicular to the skin and could be adjusted in gain and frequency. The frequency was set to 230 Hz, which corresponds to the range of maximum sensitivity of the Pacinian corpuscles⁴⁴. The gain was set to 40% of the nominal dynamic range of the tactors, which was a value that elicited clear but non-intrusive vibrations.

The prosthesis was connected to a custom-made socket containing its batteries and controller. The socket was secured with a vice and placed on the table in front of the seated participant. The monitor was also placed on the table, behind the prosthesis, and approximately 50 cm away from the participant. The Michelangelo hand implements two-DoF velocity control, namely wrist rotation and hand opening/closing in palmar and lateral grasp types. The hand was configured to only execute palmar opening and closing in this experiment. The normalized myoelectric signal (ranging from 0 to 1) was mapped through a piecewise linear function to the closing velocity and the grasping force generated upon contact, as described in the next section.

After closure, increasing the magnitude of the myoelectric signal would proportionally increase the force, following the same piecewise linear relation. Notably, the generated force could only be modulated upwards, due to the non-backdrivability of the prosthesis (as further illustrated below). Therefore, after generating the desired force, the participant could relax his muscles while the hand remained closed and maintained the same amount of force. The prosthesis force was measured by the sensors embedded in the prosthesis, normalized to the maximum force, and transmitted to the laptop at 100 Hz. At the end of the trial, the experimenter opened the hand by manually sending an opening command.

Closed loop control

Figure 7A displays the control scheme implemented in this study. The myoelectric signal provided by the electrode was normalized to 40% of the maximum voluntary contraction (MVC) and low-pass filtered at 1 Hz with a 2nd-order Butterworth filter. The selection of the EMG processing parameters was based on the results of a recent study that systematically investigated the calibration of EMG feedback⁴⁵. As shown in that study, these values ensured smooth and well-controllable myoelectric signals while avoiding excessively high muscle contractions that could lead to the development of fatigue. This signal was then mapped to the prosthesis command input. The prosthesis command input sets the prosthesis closing velocity (0 - no movement and 1 - highest closing velocity) before contact. Therefore, stronger muscle contraction leads to faster closing and higher initial grasping force. After contact, a further increase in the prosthesis command input increases the prosthesis grasping force.

Fig. 7.

(A) The prosthesis control loop and the discretization of EMG feedback levels in the normalized scale. The processed EMG (myoelectric signal) was mapped to the desired prosthesis closing velocity through a piecewise linear function, while the discrete EMG level was conveyed to the user via vibrotactile stimulation. (B) Vibrotactile spatial activation patterns (EMG feedback) that indicated the five EMG levels.

More specifically, the normalized myoelectric signal, henceforth denoted simply as EMG, was discretized into six ranges (the dead zone and five levels of active muscle contraction). The dead zone was used to avoid the prosthesis reacting to any small and unintentional muscle activity. The thresholds that separated the levels were 0, 0.05, 0.2, 0.4, 0.65, 0.95, 1 in the normalized scale. They were selected so that the levels that the participant would be targeting (levels 2, 3, and 4) were of increasing size, to mitigate the effect of the higher signal variability that characterized stronger muscle contractions (see Fig. 7A). The last level was narrow, and its role was to indicate the overshoot to the participant. The EMG threshold values were mapped to the thresholds of the prosthesis command input 0, 0, 0.15, 0.4, 0.75, 0.95, 1, thus implementing the piecewise linear mapping between the EMG and the prosthesis command signal. The prosthesis command input corresponded to the closing velocity and, by extension to the grasping force that the prosthesis applied.

As the participant modulated the amplitude of the myoelectric signal, the vibrotactile stimulation (EMG feedback) indicated the momentary EMG level, using a simple spatial encoding scheme. When the myoelectric signal crossed the threshold of the dead zone to enter level 1, feedback level 1 was activated (see Fig. 8). As the myoelectric signal increased, feedback levels 2, 3, and 4 were activated in sequence, indicating that the myoelectric signal crossed into ranges 2, 3, and 4, respectively. The tactor activation patterns corresponding to each feedback level are shown in Fig. 7B. Discrete and spatially encoded feedback schemes for myoelectric prosthesis feedback are simple and easy to perceive and have been used extensively in literature^22,46–50. Importantly, although the feedback indicated a discrete range (EMG level), the myoelectric signal transmitted as the command input into the prosthesis was not discretized. This means that despite discrete feedback, the participant could modulate the closing velocity and grasping force throughout the whole working range of the prosthesis.

Fig. 8.

Demonstration of EMG feedback. The generated EMG is shown in blue, the prosthesis grasping force is shown in red, and the corresponding EMG feedback is represented with the dotted line. The vertical red lines represent the touch onset and the opening command.

Figure 8 illustrates the closed-loop control showing the generated EMG signal, feedback levels, and grasping force. The feedback is active from the start of the trial and indicates the momentary level of EMG generated by the participant. The EMG reaches level 3 just before contact, and the initial grasping force in the moment of contact is proportional to the EMG. After contact, the participant increases the EMG further to level 4 and the force abruptly jumps to a higher level. The participant then relaxes his muscles, and the EMG goes to zero, but the force is still maintained due to prosthesis non-backdrivability. The force drops to zero only after an “active” opening command was issued.

A unique feature of EMG feedback compared to other feedback approaches is that it facilitates predictive control of the grasping force¹⁵. Specifically, the participant can use the feedback to modulate their muscle contraction to a desired level while the hand is still in motion and has not yet applied any force. The piecewise linear correspondence between the EMG, velocity, and force implies that the force level exerted after closing would be equal to the EMG level conveyed to the subject through the feedback. More specifically, in the present study, maintaining the EMG signal within one of the five levels, as explained above, produced the grasping force within one of the five force levels. For instance, level 2 of EMG would lead to level 2 of the generated grasping force. The force levels were defined by the thresholds 0, 0.2, 0.4, 0.75, 0.95, 1, where 1 denoted the maximum force of the prosthesis, and the thresholds were established experimentally by commanding the prosthesis to close at different velocities and then measuring the resulting grasping force. More specifically, we generated a step command signal, waited for the prosthesis to close, and measured the generated grasping force. This was repeated with increasing amplitudes of the step input from 0 to 1 at increments of 0.05. Since during the experiment, the prosthesis command signal was the normalized EMG, this test allowed us to define the mapping between EMG and force levels, as described above. As can be seen from the obtained thresholds of the force levels, they closely correspond to those of the EMG. The small differences in some levels ensured that the EMG-to-force mapping was indeed reliable. Essentially, such fine-tuned mapping guaranteed that if the participant maintained his EMG within level $n=1,2,...5$, the prosthesis generated the grasping force of the same level upon closing. Importantly, the same piecewise linear relation between the EMG and force existed in the condition where no feedback was delivered; however, in this case, the subject did not receive explicit information about the magnitude of the myoelectric signal and, hence, to execute the task, he could only rely on incidental cues from the prosthesis and his natural muscle proprioceptive feedback.

Experimental protocol

Each experimental session started with the measurement of the participant’s baseline EMG and MVC. The participant was first asked to relax his muscles for 10 seconds and the mean value of the resulting EMG was adopted as the baseline and subtracted from subsequent recordings. This was followed by the recording of the MVC, where the subject was asked to maximally contract the target muscles in 3 windows of 5 seconds. The only visual feedback that the participant received in this phase was a cue indicating when to initiate and terminate the contractions. The average of the mean amplitude values from these windows was used as the MVC. The participant activated his left (intact side) pectoralis by performing a humeral adduction against his torso and his flexor carpi radialis by performing a wrist flexion against the thermoplastic splint.

A short training session followed, where the four C2 tactors were activated in sequence to convey all the feedback levels and the meaning of each activation pattern was explained to the subject. The patterns were then activated in a pseudorandom order, while the subject was asked to identify them, to ensure that he could recognize the different stimulation patterns. If the subject gave a wrong answer, the experimenter pointed out the correct one (reinforced learning). This procedure continued until the subject could reliably identify all EMG feedback patterns.

The experimental task was the same in all sessions. The participant performed a force-matching task where he was asked to close the prosthesis and generate the desired level of grasping force, which was indicated on the computer screen. The targets were force levels 2, 3, and 4, defined in section Closed-loop control, and henceforth referred to as low, medium, and high target force. The prosthesis was closing without an object in the hand and the grasping force was produced by the opposing fingers.

Each experimental run was preceded by 30 training trials. During the training, the participant could see the myoelectric signal (filtered and normalized EMG) and force traces as well as their respective thresholds on the monitor, as shown in Fig. 6B. He was initially allowed to freely modulate his muscle contractions and feel how the vibrotactile stimulation indicated the level of the signal magnitude. Guided by the experimenter, the participant was instructed to generate the three different EMG levels by relying on tactile feedback or by looking at the traces on the monitor in the condition when no feedback was being delivered (first and third block in session 2). The correspondence between the generated force and the EMG feedback was also explained and demonstrated to the participant. Notably, the applied force registered for each trial corresponded to the highest force achieved during the trial.

Session 1

The aim of the experimental session 1 was to introduce the participant to the prosthesis control with EMG feedback and to assess how well he could exploit the feedback right after a brief training. In session 1, the participant performed 60 trials of the force-matching task, using his right pectoralis major to control the prosthesis. The sequence of the targets was pseudorandomized, and the participant performed 18 trials targeting levels 2 and 3 and 24 trials targeting level 4. Compared to subsequent tests, the number of trials was lower and unbalanced across levels, since this was the very first assessment, and the preparation time was longer.

Session 2

The goal of session 2 was to investigate if the provision of EMG feedback would improve the participant’s performance in controlling the prosthesis grasping force compared to the no-feedback condition. A secondary aim was to assess if the feedback would have a positive training effect by facilitating short-term (within-session) learning to control the prosthesis. To that end, the participant performed three blocks of 90 trials (30 trials per target force level), and the EMG feedback was activated only during the second block. Therefore, the first block provided the baseline, the second indicated the performance with closed-loop control, and the third was used to assess the control immediately after “training” with the feedback. A 90-trial block was conducted in three batches of 30 trials, with a short break between them. There was a longer break after each block. The participant used his right-side pectoralis major to control the prosthesis in this session.

Sessions 3 and 4

Sessions 3 and 4 aimed at assessing the quality of closed-loop control with EMG feedback when using the muscle on the amputated side versus the same muscle on the intact side, as well as the forearm muscles. This was relevant to assess because of the particularly extensive surgical intervention and prolonged disuse of the pectoralis muscle on the amputated side, which could have affected the ability of the participant to reliably modulate the muscle activation. In session 3, therefore, the tests were conducted by placing the EMG electrodes on the right and left pectoralis major and left flexor carpi radialis, while the order of the muscle tests was reversed in session 4. In both sessions, the subject conducted 60 trials (20 trials per target force level) in two 30-trial blocks with each muscle. After testing with one muscle was complete, there was a break of approximately 5 min during which the participant rested. The electrodes were then repositioned onto the next muscle and the test was repeated.

Data analysis

Three outcome measures were used to evaluate the performance: the success rate, overshoot, and undershoot rates defined as the number of trials where the applied force level was equal, higher, or lower than the target force level, respectively, divided by the total number of performed trials and expressed in percent. The over- and undershoot rates can provide insight into the strategies and control behavior of the participant. Each of these measures was computed per level as well as overall. The success rate indicated the ability of the subject to successfully complete the task by applying the correct force, while the over- and undershoot rates offered insight into the pattern of errors during the task execution.

Author contributions

J.T., J.L.D, and S.D. conceived the experiment(s), R.L.H. and P.H.J. performed the surgery, followed up and monitored the participant, and J.T. conducted the experiments and analyzed the results. All authors reviewed the manuscript.

Data availibility

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Declarations

Competing interests

None of the authors have any competing interests to report.