Electrotactile Feedback Improves Performance and Facilitates Learning in the Routine Grasping Task

Abstract

Aim of this study was to investigate the feasibility of electrotactile feedback in closed loop training of force control during the routine grasping task. The feedback was provided using an array electrode and a simple six-level spatial coding, and the experiment was conducted in three amputee subjects. The psychometric tests confirmed that the subjects could perceive and interpret the electrotactile feedback with a high success rate. The subjects performed the routine grasping task comprising 4 blocks of 60 grasping trials. In each trial, the subjects employed feedforward control to close the hand and produce the desired grasping force (four levels). First (baseline) and the last (validation) session were performed in open loop, while the second and the third session (training) included electrotactile feedback. The obtained results confirmed that using the feedback improved the accuracy and precision of the force control. In addition, the subjects performed significantly better in the validation vs. baseline session, therefore suggesting that electrotactile feedback can be used for learning and training of myoelectric control.

After hand amputation, the myoelectric prostheses can be used to partially restore motor functions. In these systems, the electrical activity is recorded from the wrist flexor and extensor muscles and converted into the control signals for the prosthesis.¹ The main drawback of the commercially available prostheses is the lack of somatosensory feedback. The only exception is a recently presented device (VINCENTevolution 2, Vincent Systems GmbH, DE) integrating a simple force feedback delivered through a vibration motor. Providing the users with artificial somatosensory feedback could improve the control, facilitate the feeling of embodiment and the prosthesis ownership, and reduce the phantom limb pain.^2,3

Sensory substitution,⁴ an approach that can be employed for restoring the missing somatosensory information, is based on collecting the data from the prosthesis sensors and delivering them to the user by activating remaining sensory structures. The most commonly used non-invasive methods include delivering feedback information by stimulating the skin over the residual limb using vibrotactile,^5,6 electrotactile stimulation, ^6,7 or even the combination of the two.⁸ In case of electrotactile stimulation, tactile sensations are elicited by delivering low-level electrical current pulses to the skin to depolarize the skin afferents. The feedback information can be transmitted by modulating the quality and intensity of the elicited sensations, which can be achieved by changing the stimulation parameters (i.e., pulse width, amplitude, and frequency modulation) and/or location of the active channel (spatial modulation). The latter demands the use of a multichannel interface.⁷ If in addition the stimulation parameters can be independently modulated, this would allow the implementation of high-resolution multichannel interfaces with flexible information coding. Most closed loop systems presented in prevous studies considered grasping force as a variable to be sent to the user.⁹ To grasp an object securely, the forces need to be high enough to prevent slipping but also not excessive, in order not to damage the object. Furthremore, the grasping force cannot be directly determined by visual inspection when grasping stiff objects. The role of the feedback is to assist the grasping process by improving the consistency of the grasping action and force generation, but it can also be considered as an instrument for learning through repeated practice.¹⁰

In the current study, a multichannel electrotactile interface was used to feedback the applied grasping force of a state of the art myoelectric prosthesis. The understanding of electrotactile feedback and the quality of force control during the routine grasping task under two conditions (open and closed loop control) were evaluated on three subjects with transradial amputation. The main goal of the study was to assess the effects of electrotactile feedback on the control of the grasping force. More specifically, the aim was to investigate the immediate impact of feedback on performance as well as the utility of feedback (closed loop training) in learning the feedforward control of the prosthesis. The latter aspect was also investigated in a recent study,¹⁰ but the protocol included visual feedback, able-bodied subjects and a single level of force.

Materials and Methods

A. System setup

The system setup comprised the following (Fig. 1): 1) Michelangelo hand prosthesis with a dry active surface EMG electrode (Otto Bock Healthcare GmbH, Vienna, AT), 2) multichannel electrotactile stimulation system (MaxSens, Tecnalia, San Sebastian, ES), and 3) a laptop PC (Intel(R) Core(TM) i5-4210U CPU at 1.70GHz, 6GB RAM).

Fig 1.

The system setup comprising Michelangelo hand (1), electrotactile stimulation system (2), and the laptop PC (3). The laptop PC ran the control loop, received sensor data (EMG signal and grasping force) from the prosthesis while sending back normalized commands, and sent feedback mapping to the stimulator unit via Bluetooth (blue arrows). EMG signals were recorded from wrist flexor muscles of the residual limb using one EMG electrode connected to the prosthesis (dashed line). The electrotactile feedback was presented to the user through an array electrode placed on the intact forearm

The Michelangelo hand is a two degree-of-freedom myoelectric prosthesis capable of producing two types of grasps: palmar (used in this study) and lateral. The prosthesis is equipped with an embedded sensor measuring grasping force (maximum ~100 N). One active electrode was used for recording and preprocessing of a bipolar EMG signal from the wrist flexor muscle, which was subsequently low-pass filtered at 1.5 Hz.

The stimulation system was a fully-programmable and integrated electrotactile interface comprising a stimulation unit and a flexible array electrode. The stimulation unit generates biphasic, current-controlled pulses with adjustable stimulation parameters (50 – 1000 µs pulse width with 10 µs step, 1 – 400 Hz pulse rate with 1 Hz step, and 0.1 – 5 mA amplitude with 0.1 mA step). Using the multiplexing unit, pulses generated by a single stimulation channel were distributed to 16 independent pads of the array electrode. The stimulation array electrode, which consisted of 16 circular cathodes and a single adjacent anode, was 200 mm long and designed to be placed circumferentially around the forearm (Fig. 2). The size of the electrode was chosen in accordance with an average forearm circumference and the inter-pad distance satisfies the two-point discrimination threshold for electrical stimulation on the forearm.¹¹ It was made of a polyester layer, and Ag/AgCl conductive layer, and an insulation coating covering the conductive leads. In order to improve the electrode-skin contact, the pads were covered with conductive hydrogel pads (AG702, Axelgaard, DK).

Fig 2.

The stimulation array electrode, consists of 16 circular cathodes and a single anode stretching alongside, used for electrotactile feedback. The pads used for feedback mapping are marked with black rectangles (Levels 1 – 5) and red circles (Level 6).

The control loop with sensory feedback was implemented at the laptop PC using a flexible test bench for the evaluation of the human manual control systems, developed using Matlab Simulink and Real-Time Windows Target toolbox.¹² Both the prosthesis and the stimulation system were connected to the laptop PC via Bluetooth. Sensor data (EMG signal and normalized grasping force) was received from the prosthesis at 100 Hz. The normalized linear envelope of the EMG signal was used as a control input for the prosthesis, proportionately setting closing velocity and grasping force. These two variables are directly dependent – the force applied by the prosthesis at the moment of the initial contact with an object is proportional to the hand closing velocity. Opening of the prosthesis was automatically triggered when the subject relaxed the muscles. Normalized grasping force was the feedback variable transmitted to the stimulation interface, thus closing the loop.

B. Feedback mapping

The feedback information was delivered through the electrotactile interface using spatial coding configuration. The frequency was set to 20 Hz and the pulse width to 220 µs, while the amplitude for each pad was individually calibrated for each subject to obtain clear and comfortable sensations. The full range of grasping force (0 – 100%) was divided in 6 subranges (force levels). Due to nonlinearity of the prosthesis’ response, which results in an offset force even when closed at the minimum velocity, the first subrange covered 0 – 25%, and the remaining force range of 25 – 100% was divided in 5 equal subranges. Feedback mapping is presented in Fig. 2. Five groups of 2 neighboring pads were selected for coding the first 5 levels of the feedback variable: level 1 – pads 1 and 2, level 2 – pads 5 and 6, level 3 – pads 8 and 9, level 4 – pads 11 and 12, and level 5 – 15 and 16. The last subrange (85 – 100%) was coded by the simultaneous activation of 6 pads spreading around the forearm (pads 1, 5, 8, 9, 12 and 16). Therefore, an increase in the feedback variable was perceived by the subject as electrotactile stimuli moving along the forearm in mediolateral direction, to be concluded with the stimuli spreading around the full forearm circumference, when the force exceeded 85%.

C. Experimental task and protocol

Three transradial amputees (2 males, 1 female, 48 ± 9 yrs.) participated in the experiment. The experimental protocol was approved by the Ethics Committee of the Specialized Hospital for Rehabilitation and Orthopedic Prosthetics Belgrade, Serbia. All subjects gave their informed consent for participating in the study.

The subjects were comfortably seated in front of the table. The prosthesis and a solid object, set to be grasped by Michelangelo hand, as well as the laptop PC, were positioned on the table in front of the subjects. The EMG electrode was placed on the wrist flexor muscle of the subject’s residual limb, and the stimulation electrode was positioned on the forearm of the intact arm. Each subject was asked to perform wrist flexion using maximum voluntary contraction (MVC), and the gain of the EMG electrode was set so that the generated signal can reach the saturation level. The range from 10% to 60% of MVC was mapped as the normalized prosthesis input (0 – 100 %).

Since the subjects had no prior experience in using a myoelectric prosthesis or receiving electrotactile stimulation, they were presented with an explanation of working principles of the prosthesis and the stimulation. In the same manner as in 13, the subjects first went through a short training of myoelectric control. The control of the grasping force was evaluated using the routine grasping task. In this, the subjects were asked to activate the muscles to an appropriate contraction level (feedforward command) so that the prosthesis closes and grasps the object with the indicated desired grasping force. They were instructed not to steer the force after contact but rather to use the information on the generated force (electrotactile feedback) to adjust the feedforward command in the next trial. Out of the six force levels coded through the feedback, the four levels (levels 2-5) were used as the target forces. The levels 1 and 6 were left out since these forces could be achieved trivially by performing minimal or maximal contraction, respectively.

The experimental session was organized in four blocks of grasping trials and the feedback training with psychometric evaluation. The first and last block of grasping trials were performed in open loop, and the second and third block in closed loop with electrotactile feedback. Each block comprised 60 grasping trials, with target forces (levels 2-5) pseudorandomized so that each of the target forces appeared 15 times in total. In summary, the first block evaluated the baseline performance of force control, i.e., the precision and accuracy that the subjects could achieve with no explicit feedback on the generated force. The second and third block aimed at assessing the impact of feedback and in addition served as the training. In these blocks, as explained before, the subjects could exploit feedback in order to adjust their feedforward myoelectric commands across trials. The last block (no feedback) evaluated the extent of acute learning, i.e., the potential retention of the learned feedforward control.

Feedback training during which the subjects were trained to correctly interpret the stimuli, followed by the psychometric evaluation, was conducted before the start of the second block of grasping trials (force control with feedback). All six electrotactile levels were first presented to the subject from the lowest to the highest. After that, 30 trials of reinforced learning were performed. The six stimulation codes were randomly delivered, the subjects were asked to identify the code (i.e., corresponding force range), and the experimenter provided the correct answer. Finally, the psychometric evaluation included 60 trials of randomly ordered electrotactile codes with no feedback about the correct answer.

D. Data analysis

The outcome measure in the psychometric test was success rate (SR) of correctly identified stimulation codes (feedback levels). In the routine grasping task, the mean absolute error (MAE) and the standard deviation of the absolute force error (STDAE) were calculated for each subject and each session to assess the accuracy and precision of the force control, respectively. The error was computed for each grasping trial as a difference between the normalized target and applied force (both expressed as a percentage of maximum force). The latter was determined as the maximal force reached during the first 200 ms after the touch onset.

Results

The results of the psychometric evaluation are presented in Table 1. Individual and averaged SR for all three subjects are presented for each force level and the whole evaluation session. The subjects were able to perceive and identify six electrotactile codes with an average SR of 94 ± 4%. While the inner four levels (levels 2 – 5) were occasionally misclassified, all subjects were able to recognize levels 1 and 6 with the SR of 100%.

Table 1.

Success rate when discriminating six spatially coded electrotactile feedback levels.

	Success Rate [%]
	Subject 1	Subject 2	Subject 3	Averaged across subjects
Level 1	100	100	100	100
Level 2	83	75	100	86 ± 13
Level 3	83	85	100	89 ± 9
Level 4	80	100	100	93 ± 11
Level 5	100	100	92	97 ± 4
Level 6	100	100	100	100
Averaged across levels	91 ± 10	93 ±11	99 ± 3	94 ± 4

Figure 3 depicts MAE (left) and STDAE (right) for the four blocks of the grasping trials, averaged across the three subjects. As expected, MAE was lower when the subjects were provided with the electrotactile feedback – 12.9% and 12.6% for Blocks 2 and 3, respectively, compared to 24.4% in the first block. Moreover, when the feedback was abolished in the fourth block, the MAE increased but still remained lower than in the first block with no feedback (15.6% vs. 24.4%). The trend was similar for the precision of the grasping force control, since the STDAE decreased from 14.0% in Block 1 to 9.3% in Block 2 and 9.7% in Block 3. After the feedback was removed in Block 4, the STDAE remained similar (9.6%) to that in the blocks with feedback.

Fig 3.

Mean absolute error (left) and standard deviation of absolute error (right) for four blocks of the routine grasping task, averaged across all subjects. The results for individual subjects are marked with different symbols (Subject 1 – triangle, Subject 2 – circle, Subject 3 - square).

Discussion

The ability of the subjects to understand multichannel electrotactile feedback and exploit it in the closed loop myoelectric control during the routine grasping task was evaluated in three amputated subjects. Psychometric tests demonstrated that the subjects were able to identify a set of six electrotactile spatial codes with a high success rate (94 ± 4%). In the previous study conducted in 10 able-bodied subjects with the same electrode, the SR in recognizing six equally spaced electrotactile stimulation channels was 88 ± 11%.¹⁴ The results indicate that the novel method of spatial coding can improve the recognition of the electrotactile feedback. Instead of six levels with equally distributed pads, feedback was coded with five groups of two neighboring pads and one distinct level, associated with the maximal subrange of force, which relied on simultaneous activation of multiple stimulation sites. In this manner, the spatial discrimination requirements were reduced and the feedback understanding was increased. Level 1 and level 6 were always correctly recognized by all the subjects. However, it is important to emphasize that the feedback mapping was chosen due to its simplicity, in order to ensure the high recognition rate and understanding of stimuli in psychometric tests. Different mapping, which would rely on physiological modeling and the intuitivism of the feedback, could possibly lead to a similar performance in closed loop force control, independent of the results of psychometric test.

In routine grasping task, MAE and STDAE decreased in the blocks with feedback with respect to the first block (baseline open loop performance). Therefore, the provision of feedback improved the quality of force control in both accuracy (MAE) and precision (STDAE). Importantly, contrary to previous studies,^10,13 the present experiment included a realistic task, in which the subjects needed to generate four levels of force in a random order (as when grasping a sequence of objects in real life). At the same time, the feedback was transmitted using a practically relevant method, i.e., a multichannel electrotactile interface implementing a low-resolution feedback (6 discrete levels). This is an encouraging result implying the benefits of closing the loop in myoelectric prosthetics. In addition, at all times, the subjects had access to indirect feedback sources (e.g., motor sound and visual assessment of hand closing velocity). Although these alternative forms of feedback could partially contribute to faster learning of force control, noticeable difference in performance between block 1 (baseline) and block 2 (in which the electrotactile feedback was introduced) suggests that electrotactile feedback had a major influence.

Furthermore, the results demonstrated that the short training of force control with electrotactile feedback (120 trials of blocks 2 and 3) was enough to improve the performance of open loop grasping (block 1 vs. block 4, both without feedback). The subjects used the feedback to adjust and learn the myoelectric commands required for producing the four force levels, and they were able to retain and reproduce those feedforward commands once the feedback was removed. This finding suggests that electrotactile feedback can be exploited for learning and training of myoelectric control. We believe that extensive training with electrotactile feedback could result in further improvement in accuracy and precision, eventually leading to an equal performance with and without feedback. However, this is a hypothesis that needs to be tested in the future experiments.

In conclusion, the presented study demonstrated that the control of myoelectric prosthesis, both in terms of precision and accuracy, improved when the subjects were provided with electrotactile force feedback. Furthermore, the study revealed the learning process, since all the subjects performed better in the final block without feedback compared to the first one. This is an encouraging outcome, suggesting that electrotactile feedback not only improves the performance in myoelectric control, but can also allow short-term learning. The next step is the evaluation of the results in longitudinal study to explore the long-term effects of the closed loop control.