Skip to main content
PMC home page
Scientific Reports logoLink to Scientific Reports
. 2018 Jun 4;8:8541. doi: 10.1038/s41598-018-26810-w

Audible Feedback Improves Internal Model Strength and Performance of Myoelectric Prosthesis Control

Ahmed W Shehata 1,2,✉, Erik J Scheme 1,2, Jonathon W Sensinger 1,2
PMCID: PMC5986794  PMID: 29867147

Abstract

Myoelectric prosthetic devices are commonly used to help upper limb amputees perform activities of daily living, however amputees still lack the sensory feedback required to facilitate reliable and precise control. Augmented feedback may play an important role in affecting both short-term performance, through real-time regulation, and long-term performance, through the development of stronger internal models. In this work, we investigate the potential tradeoff between controllers that enable better short-term performance and those that provide sufficient feedback to develop a strong internal model. We hypothesize that augmented feedback may be used to mitigate this tradeoff, ultimately improving both short and long-term control. We used psychometric measures to assess the internal model developed while using a filtered myoelectric controller with augmented audio feedback, imitating classification-based control but with augmented regression-based feedback. In addition, we evaluated the short-term performance using a multi degree-of-freedom constrained-time target acquisition task. Results obtained from 24 able-bodied subjects show that an augmented feedback control strategy using audio cues enables the development of a stronger internal model than the filtered control with filtered feedback, and significantly better path efficiency than both raw and filtered control strategies. These results suggest that the use of augmented feedback control strategies may improve both short-term and long-term performance.

Introduction

Recent advances in material design, micromachining, and the understanding of human neuromuscular systems have enabled the development of lightweight prosthetic devices that can be used to help amputees perform activities of daily living. One approach to controlling these devices is to use myoelectric signals sensed from contractions of the amputee’s remnant or congenitally different muscles1. Researchers have developed many signal processing techniques2, feature extraction methods3–5, and control strategies6 to enhance the performance of this approach. Despite these advancements in the field of myoelectric prostheses, many amputees still abandon their devices out of frustration7, due in part to insufficient precision in the control of prosthesis movements and a lack of adequate sensory feedback8,9.

Although invasive feedback, such as stimulation of sensory peripheral nerves10, has the potential to elicit close-to-natural tactile sensations, many prosthesis users prefer non-invasive feedback methods that do not require surgical intervention11. With this preference in mind, researchers have proposed using non-invasive sensory substitution methods to provide sensory information to prostheses users either through different sensory channels or using different modalities12. Vibro-tactile13, mechano-tactile14, electrotactile15–17, skin stretch18, and auditory19 are just some of the techniques that have been developed and used to provide prosthesis users with feedback. Although some studies (e.g.14,20,21) have shown that sensory feedback improves performance, others (c.f.14) have concluded that sensory feedback had no effect on performance. This lack of consensus arises, at least in part, because of an unclear understanding of how the incorporation of feedback relates to performance.

The role of feedback for real-time regulation and in improving human understanding of the control system and the task being performed is still unclear. Researchers have hypothesized that the human central nervous system implements control by estimating the current state of the musculoskeletal system and updating this estimate using sensory feedback22,23. This state estimation and prediction process is governed by a model formed in the central nervous system, which is known as internal model. This internal model holds properties of the arm, which are used to imitate its behavior24, predicts consequences of an action, and computes an action based on desired consequences25. Hence, this internal model is used in the feedforward control26 of the arm or residual limb, and affects the overall performance of a prosthesis27.

Humans use visual feedback along with other sensory and proprioceptive information to develop their internal model, however constant visual attention and the high level of concentration it requires may lead powered prosthesis users to reject their devices28. Conveniently, augmented feedback can be used to convey artificial proprioceptive and exteroceptive information21,29, which may help to develop strong internal models. Researchers have used audio augmented feedback in both robotic teleoperation30,31 and Brain Computer Interfaces (BCI)32 and have concluded that audio augmented feedback improves performance. Unlike visual feedback, audio requires less focus of attention and reduces distraction33,34. We hypothesize that audio augmented feedback may similarly improve the performance of myoelectric prosthesis control by enabling the development of stronger internal models.

In a recent study35, we assessed the performance and the strength of internal models developed by users when using two myoelectric control strategies that differed in control signals as well as level of feedback. The main aim of that study was to investigate whether the ability of users to adapt is influenced by the degree of sensory feedback inherent in different myoelectric controllers. Results from that study suggest that control strategies with raw control signals and high feedback level (Raw Control with Raw Feedback (RAW), such as with regression-based control) enable the development of a strong internal model, but at the expense of short-term performance. In addition, we found that control strategies with filtered control signals and reduced feedback (Filtered Control with Filtered Feedback (FLT), such as classification-based control) may enable better short-term performance, but hinder the development of internal models.

To mitigate this tradeoff, we have extended this work by decoupling the concepts of control and feedback through the use of augmented feedback. We combined the filtered control strategy that resulted in better short-term performance with audio augmented feedback from the raw control strategy that enables the development of stronger internal models. In this work, we assessed internal model strength and short-term performance for this audio augmented feedback control strategy known as Filtered Control with audio Augmented Feedback (AUG) and compared its results to the two commonly used myoelectric control strategies, RAW and FLT, that we assessed previously35. Our results show that the audio augmented feedback control strategy produces better short-term performance (as assessed using path efficiency and accuracy) than the feedback-rich control strategy, while enabling the development of a stronger internal model than the reduced feedback control strategy.

Results

To inform the development of myoelectric control strategies with better short- and long-term performance, we investigated whether augmented feedback could improve user’s internal model strength without reducing short-term performance of the control. Short-term performance and internal model strength (as a predictor for long-term performance36) were evaluated while using a filtered control strategy that was augmented by audio feedback in a virtual target acquisition constrained-time task.

Outcome Measures

To assess the internal model developed by the human central nervous system and evaluate performance of the audio-augmented controller, we employed the same five parameters that were used in our previous work35:

  • Adaptation rate was extracted by computing the rate of feedforward modification of the control signal from one trial to the next37,38. The first 100–230 msec window of activations that started on the mark of the visual rendering on the screen for each trial was used to ensure that only the subject’s feedforward intent was captured and to avoid the effect of feedback synchronization. The target control signal was the activation of the cursor movement to the right direction only i.e., wrist extension. Other activations were considered as self-generated errors, which subjects were instructed to minimize.

  • The just-noticeable-difference (JND) parameter was computed from a perception threshold test as the threshold value reached after 23 reversals calculated by an adaptive staircase39. This parameter is influenced by both controller and sensory noise.

  • Internal model uncertainty Pparam was calculated using sensory uncertainty, controller uncertainty, JND, and adaptation rate35.

  • Performance of the audio-augmented control strategy was evaluated using path efficiency, computed by comparing the path taken to reach a target to the shortest Manhattan path to that target. Targets were defined by their location with respect to the X and Y axes on the computer screen. Targets that were located on either the X-axis or the Y-axis were known as on-axis targets, which only required the activation of a single DOF to be acquired. By the same token, targets that were not located on either the X-axis or the Y-axis were known as off-axis targets and therefore requiring the activation of 2 DOFs to be acquired (Appendix Figure A1).

  • Accuracy of target acquisition was computed as the Manhattan distance between the center point of a target and the actual final point reached. The ratio between this value and the optimal Manhattan path was then used to compute normalized accuracy as a percentage.

Psychophysical tests

The adaptation rate serves as an indicator of how much humans modify their internal models when performing a task. A value of 1 indicates perfect adaptation; higher and lower values correspond to over- and under-compensation, respectively. Results for the adaptation rate test show that there was a statistically significant difference in adaptation rates to self-generated errors between subjects when using the RAW, FLT, and AUG controllers, as determined by a one-way ANOVA (F (2, 21) = 9.17, p = 0.02). A Tukey HSD post hoc test reveals that the adaptation rate for subjects using FLT (0.46 ± 0.19) was significantly lower than RAW (0.98 ± 0.3, p = 0.02) and AUG (0.86 ± 0.27, p = 0.021). In addition, no significant difference was found between adaptation rate data for subjects using the RAW and AUG control schemes (p = 0.65). These results suggest that audio-augmented feedback may enable similar adaptation behavior to feedback-rich control strategies (Fig. 1a).

Figure 1.

Figure 1

Open in a new tab

Overall psychophysical test results. (a) Results for adaptation rate across control strategies showing low adaptation rate to self-generated errors when using FLT. (b) JND results across control strategies showing low perceivable sensory threshold when using either RAW or AUG. (c) Internal model uncertainty reflected by both adaptation rate and JND results showing significantly less uncertain (more confident) internal models developed when using RAW and AUG than the one developed when using FLT.

The JND parameter is a measure of the smallest measure of stimulus that a subject is able to identify when using a certain controller. The lower this parameter, the better the ability of a subject to detect changes in the controller used. Like the adaptation rate results, analysis of JND test data using one-way ANOVA revealed a significant difference between subjects using RAW, FLT, and AUG (F (2, 21) = 5.17, p = 0.008). Upon running a Tukey HSD post hoc test on the data, we found that subjects who tested FLT had significantly higher JND values (59 ± 16 degrees) than other subjects who tested RAW (45 ± 20 degrees) and AUG (47 ± 13 degrees), but no significant difference in JND data between subjects who tested RAW or AUG (p = 0.92), which suggests that feedback in both RAW and AUG was sufficient to allow subjects to perceive a lower sensory threshold than the reduced feedback FLT (Fig. 1b).

Both adaptation rate and JND results were used to compute the internal model uncertainty for each of the tested control strategies. The lower the internal model uncertainty, the more confident a subject is in the control system used. Results showed significant difference in internal model uncertainty between the control strategies (robust Welch ANOVA (F (2, 13) = 9.3, p = 0.003)). Games-Howell post hoc analysis revealed a significant difference between internal model uncertainty for subjects who tested FLT (0.14 ± 0.1) and the subjects that tested RAW or AUG (0.055 ± 0.09, p = 0.035 and 0.041 ± 0.034, p = 0.004, respectively). On the other hand, there was no significant difference in the internal model uncertainty between subjects that used RAW or AUG (p = 0.56) (see Fig. 1c). These findings suggest that a strong internal model may be developed when using either the RAW or AUG control strategy, which confirms the first part of our hypothesis that audio-augmented feedback may enable the development of a stronger internal model than FLT alone.

Performance tests

For on-axis path efficiency, there was a significant difference between control strategies as determined by a one-way ANOVA (F (2, 21) = 5.8, p = 0.01). Bonferroni post hoc analysis showed that subjects who tested the AUG control strategy had significantly higher path efficiency than subjects who tested either the FLT or RAW strategies (p = 0.019, p = 0.02) (Fig. 2a). Analysis of the accuracy data collected from subjects using the three control strategies showed a significant difference between control strategies (one-way ANOVA (F (2, 21) = 4.5, p = 0.024)). Even though subjects that used FLT achieved high accuracy for on-axis targets (73 ± 7%), it was not significantly different (Bonferroni post hoc test) from subjects using who used RAW (65 ± 11%, p = 0.36) or AUG (80 ± 11%, p = 0.51). Conversely, subjects who tested AUG achieved significantly higher accuracies than those who tested RAW (p = 0.02) (Fig. 2b). These results further support our hypothesis that augmented feedback may also enable better short-term performance. For off-axis target performance tests, the null-hypothesis was not rejected when comparing path efficiency or accuracy data between the three control strategies (one-way ANOVA (F (2, 21) = 0.16, p = 0.86)) (Fig. 2c,d).

Figure 2.

Figure 2

Open in a new tab

Overall performance results. (a) and (b) Show results for path efficiency and accuracy, respectively, of on-axis targets across control strategies. (c) and (d) Show no significant difference between control strategies tested for off-axis targets.

Learning effect

We ran 28 dependent paired t-tests to investigate possible learning effects when testing with one control strategy and retesting using another one (summarized in Appendix). Unlike adaptation rate, a significant learning effect was observed for JND values of subjects who used FLT after being exposed to AUG, where the JND dropped from 74 ± 8.0 degrees to 43 ± 4.2 degrees (p = 0.029). This suggests that there may be possible enhancement in sensory perception threshold when using the FLT control strategy after first being exposed to the feedback-richer AUG control strategy. On the other hand, learning effect for JND values for subjects who tested FLT after AUG were not significantly reflected in the internal model uncertainty (t (3) = 1.41, p = 0.25), because internal model uncertainty is affected by several other parameters e.g., adaptation rate and controller noise. These results suggest that internal model uncertainty for FLT may be improved after being first exposed to AUG by improving sensory perception, and less so by adaptation. Interestingly, on-axis target path efficiency significantly increased when testing the AUG control strategy after testing the feedback-rich RAW control strategy (t (3) = −14, p = 0.005). Also, accuracy for on-axis targets when using FLT after first being exposed to AUG increased significantly from 84 ± 3.7% to 94 ± 3.1% (t (3) = 23, p = 0.028).

Discussion

Significant progress made in the fields of signal processing, sensory substitution and pattern recognition for myoelectric prostheses has allowed for improvements in the performance of myoelectric control. It stands to reason that such improvement is partially driven by feedback40, however contradictory results about the effect of feedback (in presence of vision) on performance oppose this belief41.

Several studies have investigated the effect of various feedback modalities on performance12. In particular, results from studies providing electromyography (EMG) biofeedback either through visual42 or electrotactile20 feedback have showed promising improvement in performance, but have not investigated the effect of this feedback on the internal model. Other researchers36 have investigated the effect of internal models on performance and found evidence of improvement in performance through improvement in the internal model. In a recent study43, researchers have found that feedback improves short and long-term performance and credited this improvement to internal models. However, they were not able to quantify the improvement in internal models that influenced the improvement in performance. To address this deficiency, in a previous study35 we used a psychophysical framework to quantify the strength of internal models developed for two commonly used myoelectric prosthesis controllers44, namely regression-based and classification-based controllers. In this study, we extended these previous studies by exploring the effect of relaying EMG biofeedback through the less-focus demanding audio feedback to augment myoelectric controllers that enable better short term performance. We hypothesized that augmented feedback may enable stronger internal model generation and better short-term performance. Our results showed that an audio-augmented feedback control strategy enables the development of a significantly stronger internal model than a classification-like filtered control with filtered feedback control strategy. This audio-augmented feedback controller also resulted in significantly better path efficiency for a one DOF task than both the raw feedback and the filtered feedback controllers.

Unlike classification-based control methods, the use of a regression-based control strategy enables the extraction of more information that may be used for augmented feedback. In this work, the concept of a hybrid control strategy that combines the robustness of classification-based control and the rich feedback of regression-based control was introduced. The concepts of feedback and control were effectively decoupled by employing classifier-like control, while providing the user with audio-augmented feedback derived from the feedback-rich regression control scheme.

Adaptation rate results showed that the controller with audio-augmented feedback enabled better adaptation behavior than FLT while maintaining similar adaptation behavior to the feedback-rich RAW. Hence, we may conclude that audio-augmented feedback may contain sufficient and comparable information as the RAW, therefore enabling better understanding of the myoelectric system than FLT. Equally as important, JND results for sensory perception showed that subjects who used either RAW or AUG achieved lower sensory thresholds than those who used FLT, which again supports the conclusion that feedback inherent in AUG was as effective as the feedback in the feedback rich RAW control strategy, enabling the perception of lower sensory thresholds. These results combined with results from the assessment of internal model uncertainty confirm our hypothesis that audio-augmented feedback enables the development of a strong internal model, possibly leading to better long-term myoelectric control performance.

On-axis performance results showed that audio-augmented feedback helped subjects perform better with respect to path efficiency and accuracy when compared to FLT. This finding suggests that subjects are able to integrate the audio feedback and use it for real time regulation as well as for internal model development. We also found that when subjects were first trained with AUG and then used FLT, they retained the internal model developed for AUG and were able to achieve better on-axis accuracy. This observation raises a new question to be investigated in future work: how long can this stronger internal model be retained before reverting completely to a model developed by the FLT control strategy?

Despite significant improvement in performance for on-axis targets when using the audio augmented controller, no significant improvement for off-axis targets accuracy or path efficiency was found. However, statistically significant results for accuracy and path efficiency for off-axis targets may be found with larger cohorts targeted towards these metrics. Future work informed by this study could include investigating the effect of augmented feedback on the internal model strength and short-term performance when using other controllers, assessing the impact of other modalities of augmented feedback (such as vibrotactile), exploring combinations of augmented feedback (such as audio + vibrotactile, which might enable an even stronger internal model), investigating how long the human central nervous system is able to retain the improved internal model strength (following a protocol similar to other studies43), and finally, assessing the benefits of using augmented feedback for limb-different individuals by performing psychophysical tests using prosthetic devices. Future studies can extend this work to accommodate the integration of feedback in real-time regulation using a framework such as the Smith predictor45,46 or the adaptive Smith predictor26,47.

Some limitations to this study were: the learning effect due to prolonged exposure to the feedback-rich control strategy was not investigated, no amputees were included in this study, and only a one DOF task was used for the assessment of the internal models.

The benefits of using audio-augmented feedback for myoelectric prosthesis control, highlighted in this study, call for inclusion of audio feedback in myoelectric control training procedures. For example, myoelectric prosthesis users may be provided with audio feedback when training with a myoelectric controller to improve their internal model strength and then prompted to use this control strategy without this audio feedback during regular use. One might argue that providing audio feedback using an earpiece or headphones to prostheses users might be impractical or uncomfortable, but emerging technologies, such as those that transmit sound using bone conduction, may soon enable users to benefit from other types of similarly augmented feedback. Furthermore, it is not yet known how frequently this augmented feedback is need. It is possible that users could occasionally enable the audio-augmented feedback to re-enforce their understanding.

Methods

Control Strategies

In order to conduct a fair comparison between the control strategy developed here and the ones assessed in35, Support Vector Regression (SVR) was selected as the machine learning algorithm for the AUG strategy48. Figure 3 shows a sample of raw signals sensed from muscles. These signals were processed using standard EMG processing techniques used in the field of prostheses49,50. Specifically, these signals were low-pass filtered at 450 Hz with a fifth-order Butterworth filter and high-pass filtered using a third-order filter at 20 Hz51. In addition, a notch filter was implemented using a 2nd-order Butterworth band-stop filter from 57 Hz to 63 Hz52. Data were sampled at a frequency of 1 KHz and time-domain features53,54 were extracted from these processed signals every 64 msec using 160 msec windows to train the SVR model.

Figure 3.

Figure 3

Open in a new tab

A sample of EMG signal recorded from 8 electrodes placed on subject’s forearm. Subjects were asked to follow a cursor on a screen moving to the left and right with isometric wrist extension/flexion. The velocity profile for the moving cursor is shown in the middle of this figure as a sinusoidal wave with a 1 second delay after which the cursor started moving to the right first.

For real-time control, the SVR model produced output every 16 msec using 160 msec windows51. The output from this model was directly mapped to the velocity of the cursor for simultaneous activation of 2 degrees-of-freedom (wrist extension/flexion and adduction/abduction), but only the highest activated DOF was used in FLT; therefore, only allowing sequential control. Similar to FLT, the AUG control strategy only allowed sequential control, but used audio feedback to convey simultaneous control information, making this controller a hybrid between classification-based control and regression feedback (Fig. 4).

Figure 4.

Figure 4

Open in a new tab

Control strategy development. Audio feedback mapped from raw control input and augmenting the filtered control input.

Audio Augmented Feedback

For the AUG control strategy, audio feedback maps the simultaneous activation of the 2 DOF motion using 4 audibly distinct frequencies i.e., one frequency per direction, (see Fig. 5a). The amplitude of the tone is directly proportional to the SVR output, requiring no knowledge of the intended task. For instance, a subject using AUG to reach a point to the right of the cursor while activating both the right and up directions, but with a higher activation in the right direction will see the cursor moving to the right accompanying to that cursor movement, the subject will hear two tones simultaneously with frequencies of 500 Hz and 800 Hz with the 500 Hz tone being louder than the 800 Hz tone.

Figure 5.

Figure 5

Open in a new tab

Experimental Set-up, with a subject controlling a cursor on the screen to acquire a set of targets during the training phase. The target shown on the f Subjects were allowed to make their choice during the just-noticeable-difference test by pressing keys on a keyboard. (a) Audio map showing the frequencies assigned for each direction, i.e., 500/400 Hz corresponded to wrist extension/flexion and 900/800 Hz corresponded to wrist adduction/abduction. The volume is directly proportional to the magnitude of the activation in each direction (b) A shrinking off-axis target.

General Experimental Procedures and Protocol

Each subject sat comfortably in a chair approximately 80 cm away from a computer screen with their right arm relaxed in a restraint and used their left hand to press keys on a keyboard placed on a desk in front of them. Subjects were asked to wear a 15 mW Cobra headset with the volume set to a no more than 52.5 ± 3 dB for the whole period of the experiment, but were allowed to remove it during scheduled breaks between the testing block. During the testing blocks, subjects controlled a cursor on a computer screen using isometric muscle contractions sensed by an electrode array placed on their forearm to acquire targets (Fig. 5b). Targets were crosshairs surrounded by an imploding red circle. Using the same experimental protocol implemented in35, which was approved by the University of New Brunswick’s Research and Ethics Board (file number 2014–019), subjects used each control strategy to complete a series of test blocks in a specific order after accomplishing a training block.

To determine the gain for mapping the control signals to the cursor velocity on the computer screen, each subject was asked to control a brush to paint a screen35. Their ability to maximize the area covered by paint determined the appropriate gains for each DOF, i.e., 20 pixels/s for wrist adduction/abduction DOF. The training block consisted of three sets of targets that appeared at predetermined, but randomly ordered, positions. Each set consisted of 16 targets, which subjects were asked to acquire in less than 12 seconds, after which it would disappear and another target would appear. The first test block was used to test the adaptation to self-generated errors. In this block, subjects were asked to repeatedly acquire a single target on the horizontal axis over a set of 80 trials.

Before the start of each test block, subjects were given a 2 minutes break, in which they were allowed to stand, remove the headset, and stretch if needed. The JND perception threshold test, the second test block, is a psychometric measure of sensory threshold for perception of a sensory stimulus. In this test, subjects were presented with two trials where one of these trials was perturbed with a stimulus. Subjects were then asked to select the trial which they thought had the added stimulus by pressing 1 for trial 1 and 2 for trial 2 on a keyboard39. The added stimulus was calculated using an adaptive staircase with accuracy set to 0.8455. Finally, the last test block was a performance test where subjects were asked to acquire targets, arranged in 2 sets of 16 targets each. For the first set of targets, subjects were tasked with acquiring each target in less than 1.7 s, whereas they were only allowed 1.4 s for each target in the second target set.

Participants

A total of twenty-four healthy right hand dominant subjects (15 male, and 9 female; mean and SD of age: 25 ± 6 years) participated in the study. All participants had either normal or corrected-to-normal vision. Informed consent was obtained from subjects before conducting the experiment according to the University of New Brunswick’s Research and Ethics Board (REB 2014-019). Subjects were randomly assigned to one of 3 main groups (eight subjects each). Group 1 subjects tested RAW and 4 subjects from this group were randomly selected to retest using AUG, referred to as subgroup 2. Group 2 subjects tested FLT and 4 subjects from that group were randomly selected to retest using AUG (subgroup 4). Finally, group 3 subjects tested AUG and 4 subjects were selected at random to redo the experiment using RAW (subgroup 5), while the remaining 4 subjects were asked to redo the experiment using FLT, and were referred to as subgroup 6.

Data from Groups 1, 2, and 3 were used to assess differences in outcome measures between the control strategies tested. Additionally, data from subgroups 2, 4, 5, and 6 were used to investigate possible learning effects between the commonly used control strategies and the audio-augmented feedback control strategy (Fig. 6). Learning effects between the raw and filtered controllers, subgroups 1 and 3, were investigated in a previous work35, which suggested significant improvements in internal model strength and adaptation rate when using the filtered controller after being exposed to the raw controller, but no significant improvements were observed when using the raw controller after being exposed to the filtered controller.

Figure 6.

Figure 6

Open in a new tab

Subject assignment to testing groups. (a) 3 groups of 8 subjects each were used to assess differences in outcome measures between the three control strategies tested. (b) Subgroups of 4 subjects each (in gray) were used to investigate possible learning effects on AUG after using RAW and FLT, and on both RAW and FLT after using AUG.

Statistical Analysis

The Statistical Package for the Social Science software SPSS (IBM Corp, Released 2016, IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp) was used to run Levene’s test on JND, adaptation rate, internal model uncertainty, and performance measure results to investigate homogeneity in variances of the data. If data variances were found to be homogenous, one-way ANOVAs were used to assess differences between outcome measures for the control strategies tested. If statistical significance was found, post-hoc analysis was performed using either the Bonferroni or Tukey HSD test and reported using the one with the highest confidence in the p-value56. Conversely, if data variances were found to be heterogeneous, robust Welch ANOVA was used instead, and followed by post-hoc analysis using Games-Howell test (Fig. 7). Paired two sample t-tests were used with data collected from subjects in the subgroups (equal sample size). All analyses used a significance criterion of α = 0.05 and error bars shown in data plots were based on the standard error of the mean (SEM)57.

Figure 7.

Figure 7

Open in a new tab

Statistical analysis decision tree followed throughout the analysis of the results in this study.

Data availability

Data is available upon request - email corresponding author.

Electronic supplementary material

Appendix (1.2MB, pdf)

Acknowledgements

The authors thank all subjects that volunteered to participate in this study. Many thanks to Dan Blustein for his support and valuable discussions.

Author Contributions

A.S., E.S., and J.S. planned the experiments and A.S. prepared and conducted the experiments. All authors analyzed the results, reviewed the manuscript, and approved the submitted version.

Competing Interests

The authors declare no competing interests.

Footnotes

Electronic supplementary material

Supplementary information accompanies this paper at 10.1038/s41598-018-26810-w.

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  • 1.Childress DS. Historical aspects of powered limb prostheses. Clin Prosthet Orthot. 1985;9:2–13. [Google Scholar]
  • 2.Parker PA, Stuller JA, Scott RN. Signal processing for the multistate myoelectric channel. Proceedings of the IEEE. 1977;65:662–674. doi: 10.1109/PROC.1977.10545. [DOI] [Google Scholar]
  • 3.Stango A, Negro F, Farina D. Spatial correlation of high density emg signals provides features robust to electrode number and shift in pattern recognition for myocontrol. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2015;23:189–198. doi: 10.1109/TNSRE.2014.2366752. [DOI] [PubMed] [Google Scholar]
  • 4.Englehart K, Hudgins B, Parker PA, Stevenson M. Classification of the myoelectric signal using time-frequency based representations. Medical engineering & physics. 1999;21:431–438. doi: 10.1016/S1350-4533(99)00066-1. [DOI] [PubMed] [Google Scholar]
  • 5.Phinyomark A, Phukpattaranont P, Limsakul C. Feature reduction and selection for emg signal classification. Expert Systems with Applications. 2012;39:7420–7431. doi: 10.1016/j.eswa.2012.01.102. [DOI] [Google Scholar]
  • 6.Erik Scheme P, Kevin Englehart P. Electromyogram pattern recognition for control of powered upper-limb prostheses: State of the art and challenges for clinical use. Journal of rehabilitation research and development. 2011;48:643. doi: 10.1682/JRRD.2010.09.0177. [DOI] [PubMed] [Google Scholar]
  • 7.Biddiss EA, Chau TT. Upper limb prosthesis use and abandonment: a survey of the last 25 years. Prosthetics and orthotics international. 2007;31:236–257. doi: 10.1080/03093640600994581. [DOI] [PubMed] [Google Scholar]
  • 8.Clancy EA, Morin EL, Merletti R. Sampling, noise-reduction and amplitude estimation issues in surface electromyography. Journal of Electromyography and Kinesiology. 2002;12:1–16. doi: 10.1016/S1050-6411(01)00033-5. [DOI] [PubMed] [Google Scholar]
  • 9.Doeringer JA, Hogan N. Performance of above elbow body-powered prostheses in visually guided unconstrained motion tasks. IEEE Transactions on Biomedical Engineering. 1995;42:621–631. doi: 10.1109/10.387202. [DOI] [PubMed] [Google Scholar]
  • 10.Raspopovic S, et al. Restoring natural sensory feedback in real-time bidirectional hand prostheses. Science translational medicine. 2014;6:222ra19–222ra19. doi: 10.1126/scitranslmed.3006820. [DOI] [PubMed] [Google Scholar]
  • 11.Clemente F, D’Alonzo M, Controzzi M, Edin BB, Cipriani C. Non-invasive, temporally discrete feedback of object contact and release improves grasp control of closed-loop myoelectric transradial prostheses. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2016;24:1314–1322. doi: 10.1109/TNSRE.2015.2500586. [DOI] [PubMed] [Google Scholar]
  • 12.Antfolk C, et al. Sensory feedback in upper limb prosthetics. Expert review of medical devices. 2013;10:45–54. doi: 10.1586/erd.12.68. [DOI] [PubMed] [Google Scholar]
  • 13.D’Alonzo M, Cipriani C. Vibrotactile sensory substitution elicits feeling of ownership of an alien hand. PloS one. 2012;7:e50756. doi: 10.1371/journal.pone.0050756. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Antfolk C, et al. Artificial redirection of sensation from prosthetic fingers to the phantom hand map on transradial amputees: vibrotactile versus mechanotactile sensory feedback. IEEE transactions on neural systems and rehabilitation engineering. 2013;21:112–120. doi: 10.1109/TNSRE.2012.2217989. [DOI] [PubMed] [Google Scholar]
  • 15.Kaczmarek KA, Webster JG, Bach-y Rita P, Tompkins WJ. Electrotactile and vibrotactile displays for sensory substitution systems. IEEE Transactions on Biomedical Engineering. 1991;38:1–16. doi: 10.1109/10.68204. [DOI] [PubMed] [Google Scholar]
  • 16.Green, A., Chapman, C., Kalaska, J. & Lepore, F. Sensory feedback for upper limb prostheses. Enhancing Performance for Action and Perception: Multisensory Integration, Neuroplasticity and Neuroprosthetics 69 (2011). [DOI] [PubMed]
  • 17.Gonzalez-Vargas J, Dosen S, Amsuess S, Yu W, Farina D. Human-machine interface for the control of multi-function systems based on electrocutaneous menu: application to multi-grasp prosthetic hands. PloS one. 2015;10:e0127528. doi: 10.1371/journal.pone.0127528. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Wheeler J, Bark K, Savall J, Cutkosky M. Investigation of rotational skin stretch for proprioceptive feedback with application to myoelectric systems. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2010;18:58–66. doi: 10.1109/TNSRE.2009.2039602. [DOI] [PubMed] [Google Scholar]
  • 19.Gonzalez J, Soma H, Sekine M, Yu W. Psycho-physiological assessment of a prosthetic hand sensory feedback system based on an auditory display: a preliminary study. Journal of neuroengineering and rehabilitation. 2012;9:33. doi: 10.1186/1743-0003-9-33. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Schweisfurth MA, et al. Electrotactile emg feedback improves the control of prosthesis grasping force. Journal of neural engineering. 2016;13:056010. doi: 10.1088/1741-2560/13/5/056010. [DOI] [PubMed] [Google Scholar]
  • 21.Markovic M, Karnal H, Graimann B, Farina D, Dosen S. Glimpse: Google glass interface for sensory feedback in myoelectric hand prostheses. Journal of Neural Engineering. 2017;14:036007. doi: 10.1088/1741-2552/aa620a. [DOI] [PubMed] [Google Scholar]
  • 22.Wolpert, D. M., Ghahramani, Z. &Jordan, M. I. An internal model for sensorimotor integration. Science269, 1880–1882 http://www.learning.eng.cam.ac.uk/zoubin/papers/WolGhaJor95.pdf10.1126/science.7569931 (1995). [DOI] [PubMed]
  • 23.Gazzaniga, M. S. The cognitive neurosciences (MIT press, 2004).
  • 24.Jordan MI. Computational aspects of motor control and motor learning. Handbook of perception and action. 1996;2:71–120. doi: 10.1016/S1874-5822(06)80005-8. [DOI] [Google Scholar]
  • 25.Cisek, P. Internal models. In Encyclopedia of Neuroscience (Springer, 2009).
  • 26.Jordan, M. I. Chapter 2 computational aspects of motor control and motor learning. In Heuer, H. & Keele, S. W. (eds.) Motor skills, vol. 2 ofHandbook of Perception and Action, 71–120 (Academic Press, 1996).
  • 27.Imamizu H, et al. Human cerebellar activity reflecting an acquired internal model of a new tool. Nature. 2000;403:192. doi: 10.1038/35003194. [DOI] [PubMed] [Google Scholar]
  • 28.Atkins DJ, Heard DC, Donovan WH. Epidemiologic Overview of Individuals with Upper-Limb Loss and Their Reported Research Priorities. JPO Journal of Prosthetics and Orthotics. 1996;8:2–11. doi: 10.1097/00008526-199600810-00003. [DOI] [Google Scholar]
  • 29.González, J. & Yu, W. Multichannel audio aided dynamical perception for prosthetic hand biofeedback. In Rehabilitation Robotics, 2009. ICORR 2009. IEEE International Conference on, 240–245 (IEEE, 2009).
  • 30.Apostolos, M. K., Zak, H., Das, H. & Schenker, P. S. Multisensory feedback in advanced teleoperations: benefits of auditory cues. In Applications in Optical Science and Engineering, 98–105 (International Society for Optics and Photonics, 1992).
  • 31.Liu, P. R. & Meng, M.-H. Acoustic display for navigation in internet-based teleoperation. In Intelligent Robots and Systems, 2005.(IROS 2005). 2005 IEEE/RSJ International Conference on, 4161–4165 (IEEE, 2005).
  • 32.Tidoni, E., Gergondet, P., Kheddar, A. & Aglioti, S. M. Audio-visual feedback improves the bci performance in the navigational control of a humanoid robot. Frontiers in neurorobotics8 (2014). [DOI] [PMC free article] [PubMed]
  • 33.Sigrist R, Rauter G, Riener R, Wolf P. Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review. Psychonomic Bulletin & Review. 2013;20:21–53. doi: 10.3758/s13423-012-0333-8. [DOI] [PubMed] [Google Scholar]
  • 34.Eldridge A. Issues in auditory display. Artificial Life. 2006;12:259–274. doi: 10.1162/artl.2006.12.2.259. [DOI] [PubMed] [Google Scholar]
  • 35.Shehata, A. W., Scheme, E. J. & Sensinger, J. W. Evaluating internal model strength and performance of myoelectric prosthesis control strategies. IEEE Transactions on Neural Systems and Rehabilitation Engineering 1–1, 10.1109/TNSRE.2018.2826981 (2018). [DOI] [PubMed]
  • 36.Lum PS, Black I, Holley RJ, Barth J, Dromerick AW. Internal models of upper limb prosthesis users when grasping and lifting a fragile object with their prosthetic limb. Experimental Brain Research. 2014;232:3785–3795. doi: 10.1007/s00221-014-4071-1. [DOI] [PubMed] [Google Scholar]
  • 37.Johnson RE, Kording KP, Hargrove LJ, Sensinger JW. Adaptation to random and systematic errors: Comparison of amputee and non-amputee control interfaces with varying levels of process noise. PloS one. 2017;12:e0170473. doi: 10.1371/journal.pone.0170473. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Shehata, A. W., Scheme, E. J. &Sensinger, J. W. The effect of myoelectric prosthesis control strategies and feedback level on adaptation rate for a target acquisition task. In Rehabilitation Robotics (ICORR), 2017International Conference on, 200–204 (IEEE, 2017). [DOI] [PubMed]
  • 39.Faes L, et al. Small-sample characterization of stochastic approximation staircases in forced-choice adaptive threshold estimation. Attention, Perception, & Psychophysics. 2007;69:254–262. doi: 10.3758/BF03193747. [DOI] [PubMed] [Google Scholar]
  • 40.Childress DS. Closed-loop control in prosthetic systems: historical perspective. Annals of biomedical engineering. 1980;8:293–303. doi: 10.1007/BF02363433. [DOI] [PubMed] [Google Scholar]
  • 41.Cipriani C, Zaccone F, Micera S, Carrozza MC. On the Shared Control of an EMG-Controlled Prosthetic Hand: Analysis of User–Prosthesis Interaction. IEEE Transactions on Robotics. 2008;24:170–184. doi: 10.1109/TRO.2007.910708. [DOI] [Google Scholar]
  • 42.Dosen S, Markovic M, Somer K, Graimann B, Farina D. EMG Biofeedback for online predictive control of grasping force in a myoelectric prosthesis. Journal of NeuroEngineering and Rehabilitation. 2015;12:55. doi: 10.1186/s12984-015-0047-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Štrbac M, et al. Short-and long-term learning of feedforward control of a myoelectric prosthesis with sensory feedback by amputees. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2017;25:2133–2145. doi: 10.1109/TNSRE.2017.2712287. [DOI] [PubMed] [Google Scholar]
  • 44.Hahne, J. M., Markovic, M. &Farina, D. User adaptation in Myoelectric Man-Machine Interfaces. Scientific Reports 1–10, 10.1038/s41598-017-04255-x (2017). [DOI] [PMC free article] [PubMed]
  • 45.SMITH OJM. A controller to overcome dead time. ISA J. 1959;6:28–33. [Google Scholar]
  • 46.Tan K, Lee T, Leu F. Predictive pi versus smith control for dead-time compensation. ISA Transactions. 2001;40:17–29. doi: 10.1016/S0019-0578(00)00035-5. [DOI] [Google Scholar]
  • 47.Miall R, Weir DJ, Wolpert DM, Stein J. Is the cerebellum a smith predictor? Journal of motor behavior. 1993;25:203–216. doi: 10.1080/00222895.1993.9942050. [DOI] [PubMed] [Google Scholar]
  • 48.Ameri A, Kamavuako EN, Scheme EJ, Englehart KB, Parker PA. Support vector regression for improved real-time, simultaneous myoelectric control. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2014;22:1198–1209. doi: 10.1109/TNSRE.2014.2323576. [DOI] [PubMed] [Google Scholar]
  • 49.Parker, P., Englehart, K. &Hudgins, B. Myoelectric signal processing for control of powered limb prostheses. Journal of Electromyography and Kinesiology16, 541–548, 10.1016/j.jelekin.2006.08.006, Special Section (pp. 541–610): 2006 ISEK Congress (2006). [DOI] [PubMed]
  • 50.Ortiz-Catalan M, Hakansson B, Branemark R. Real-time and simultaneous control of artificial limbs based on pattern recognition algorithms. IEEE Trans Neural Syst Rehabil Eng. 2014;22:756–764. doi: 10.1109/TNSRE.2014.2305097. [DOI] [PubMed] [Google Scholar]
  • 51.Scheme E, et al. Motion normalized proportional control for improved pattern recognition-based myoelectric control. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2014;22:149–157. doi: 10.1109/TNSRE.2013.2247421. [DOI] [PubMed] [Google Scholar]
  • 52.Wilson AW, Losier YG, Parker PA, Lovely DF. A bus-based smart myoelectric electrode/amplifier: System requirements. IEEE Transactions on Instrumentation and Measurement. 2011;60:3290–3299. doi: 10.1109/TIM.2011.2159316. [DOI] [Google Scholar]
  • 53.Scheme EJ, Englehart KB. Validation of a selective ensemble-based classification scheme for myoelectric control using a three-dimensional fitts’ law test. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2013;21:616–623. doi: 10.1109/TNSRE.2012.2226189. [DOI] [PubMed] [Google Scholar]
  • 54.Farrell TR, Weir RF. The optimal controller delay for myoelectric prostheses. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2007;15:111–118. doi: 10.1109/TNSRE.2007.891391. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Ernst MO, Banks MS. Humans integrate visual and haptic information in a statistically optimal fashion. Nature. 2002;415:429–433. doi: 10.1038/415429a. [DOI] [PubMed] [Google Scholar]
  • 56.Engineering Statistics Handbook, http://www.itl.nist.gov/div898/handbook/prc/section4/prc473.htm (2003).
  • 57.Krzywinski M, Altman N. Points of significance: error bars. Nature methods. 2013;10:921–922. doi: 10.1038/nmeth.2659. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix (1.2MB, pdf)

Data Availability Statement

Data is available upon request - email corresponding author.

Articles from Scientific Reports are provided here courtesy of Nature Publishing Group

RESOURCES