Object stiffness recognition using haptic feedback delivered through transcutaneous proximal nerve stimulation

Abstract

Objective

Haptic feedback is crucial when we manipulate objects. Information pertaining to an object’s stiffness in particular can help facilitate fine motor control. In this study, we seek to determine whether objects of different stiffness levels can be recognized using haptic feedback provided by transcutaneous electrical stimulation of peripheral nerves.

Approach

Using a stimulation electrode grid placed along the medial side of the upper arm, the median and ulnar nerve bundles were targeted to evoke haptic sensation on the palmar side of the hand. Stimulation current amplitude was modulated in real-time with the fingertip force recorded from a sensorized prosthetic hand. In order to evaluate which stimulation pattern was more critical, object stiffness was encoded either by the rate of change of the stimulus amplitude or the level of peak stimulus amplitude, as the prosthesis grasped the objects.

Main results

Both encoding methods allowed the subjects to differentiate objects of different stiffness levels with >90% accuracy. No significant difference was observed between the two encoding methods, which indicated that both the rate of change of the stimulation amplitude and the peak stimulation amplitude could effectively provide stiffness information of the objects.

Significance

The outcomes suggest that it is possible to elicit haptic sensations describing various object stiffness levels using transcutaneous nerve stimulation. The haptic feedback associated with object stiffness can facilitate object manipulation/interactions. It may also improve user experience during human-machine interactions, when object stiffness information is incorporated.

Introduction

Tactile feedback plays a critical role when we interact with objects, and a lack of such feedback can limit motor performance and limit control of assistive devices, such as prosthetic arms or remotely operated manipulators [1,2]. For example, advanced prosthetic arms now allow the users to produce complex grasp patterns; however, dexterous control of these devices is still limited, partly because of the lack of sensory feedback [3–6]. Additionally, providing feedback associated with prosthetic joint angles or grasp forces can effectively improve prosthesis users’ ability to interact with objects of different properties and to effectively perceive various object properties, such as size, stiffness, or shape, with or without visual feedback [7–10].

Among the different object properties, stiffness is an important feature that characterizes an object’s resistance to imposed forces. Stiffness perception allows us to perform fine motor control, such as interacting and manipulating delicate objects [2, 11]. Unfortunately, identifying object stiffness only through visual perception can be unreliable [12–16]. We typically sense tactile information through a series of mechanoreceptors embedded in our skin [17–19]. The different types of mechanoreceptors respond to different stimuli, which can transmit different types of tactile information, such as force dynamics and the area or location of skin contact. Regarding stiffness perception, when we grasp a deformable object, the rate of change of the force imposed on the skin surface carries stiffness information. For example, a higher rate of change represents a stiffer object. Meanwhile, with a given degree of object deformation, the peak force sensed by the skin also carries stiffness information, with a higher peak force representing a more stiff object. However, it is not clear which information is more critical for stiffness perception [ 14, 17–20].

Previous studies have evaluated stiffness recognition using various stimulation modalities such as mechanical indentation or vibration [21, 22], or electrical stimulation [23–26]. For example, to restore haptic sensation of individuals with arm amputation, Raspopovic et al [25] elicited haptic sensation using proximal peripheral nerve stimulation via an intrafascicular electrode, and showed that the object stiffness, encoded by the rate of change of the stimulation intensity, could be recognized by a prosthetic hand user. However, during stiffness recognition, the peak stimulation intensities were not constant for different objects in this study. As a result, it was not clear whether stiffness recognition was based on the rate of change in the stimulation intensity or on the peak stimulation intensity.

To overcome these limitations, the purpose of this study was to determine whether objects with different stiffness levels can be differentiated using transcutaneous electrical stimulation of the proximal segments of the peripheral nerves. Previously, a 2 × 8 electrode array was placed along a subject’s upper arm. The electrode array stimulation has been shown to activate selective sets of afferent fibers in the proximal segments of the median and ulnar nerves [27, 28]. Spatially and amplitude modulated sensations can be perceived by the participants at distinct regions of their hands [27]. Based on this stimulation approach, the stiffness level was encoded using two separate methods: (1) the rate of change of stimulation intensity with an identical finger flexion velocity and (2) the peak stimulation intensity with an identical level of object deformation and identical velocity as well. Isolation of these two encoding methods, through the use of an experimenter-controlled prosthetic hand, allowed us to evaluate which method provided information that was more critical for stiffness recognition. During the testing, the subjects associated the stimulation patterns with the object stiffness, and all other sensory (visual, auditory, and proprioceptive) information was not available to the subject. The performance of these two stiffness encoding methods were compared using either index-thumb pinch or middle-thumb pinch of a sensorized prosthetic hand. Our results demonstrated that the stiffness recognition performance was comparable between the two encoding methods. Our findings suggest that object stiffness can be identified using different encoding methods (i.e. the rate of change of the stimulation amplitude and peak stimulation amplitude). The outcomes can allow us to readily evaluate the sensorimotor integration processes in prosthetic control in order to improve dexterity and promote user confidence.

Methods

Subjects

Ten neurologically intact subjects (seven males, three females, 20–35 years of age) were recruited for this study. All recruited individuals had no known neurological disorder, and each gave informed consent via protocols approved by the Institutional Review Board of the University of North Carolina at Chapel Hill.

Experimental setup

Each subject was asked to be seated with one arm comfortably placed on a table in front of him or her. The experimenter palpated the medial side of the resting upper limb directly below the short head of the biceps brachii, in order to identify the location of the brachial artery. This artery, running parallel to the median and ulnar nerve bundles, was used as a biological landmark for the placement of a 2 × 8 electrode grid. The grid was placed parallel to the vector that connects the medial epicondyle of the humerus and the center of the axilla (figure 1(A)), after the skin was cleaned with alcohol pads. This location maximizes the access to the median and ulnar nerve bundles from skin surface, which are responsible for sensation in the palmar side of the hand. The median nerve innervates the index, middle, and a portion of the ring finger, while the ulnar nerve innervates the remainder of the ring finger and the pinky. Once the electrode grid placement was complete, the electrodes were secured with the application of a mild inward pressure through a custom vice. Subjects were asked to report any discomfort or occurrence of restrictive blood flow throughout the experiment. The selection of distinct electrode pairs in the stimulation grid allows for the generation of unique electric field distributions, which can activate different sets of axons in the nerve bundles, innervating different regions of the hand.

Figure 1.

Diagram illustrating the placement of the 2 × 8 electrode grid along the upper arm (A) and flowchart describing how the fingertip forces recorded by the sensorized hand modulates stimulation amplitude. (B) Flowchart includes image of the prosthetic hand, example sigmoidal transfer function that transforms force to stimulation current amplitude, and stimulation train with constant frequency/pulse width and modifiable amplitude. This force-current amplitude transformation was utilized for both encoding methods.

The electrode pair selection was performed using a custom MATLAB (v2016b, MathWorks Inc, Natick, MA) interface. The interface controlled the selection of sixteen Ag/AgCl gel electrodes (1 cm in diameter) using a switch matrix (Agilent Technologies, Santa Clara, CA). The matrix linked one of the 16 electrodes among the 2 × 8 grid to either the anode or the cathode of the stimulator, leading to bipolar stimulations. When different electrode pairs were selected, perceived haptic sensation can be altered due to the change in the recruited sensory axons [27, 28].

A multi-channel fully programmable stimulator (STG4008, Multichannel Systems, Reutlingen, Germany) was used to deliver a customized single-channel electrical stimulus to each subject. The stimulation parameters were controlled using a custom MATLAB interface, which could control the output current of a charge-balanced biphasic square wave stimulus in real time (figure 1(B)). A pulse width of 200 μs and a pulse frequency of 150 Hz were selected based on earlier studies [27, 28].

To capture the stiffness of different objects, a sensorized prosthetic hand (The LUKE DEKA RC ARM, Modius Bionics, Manchester, New Hampshire) was used to interact with different objects using a pinch grip. The DEKA hand has sensors in each finger. The sensors could capture the forces applied at each of the prosthetic’s fingertips, as determined by a calibration using a force transducer (LCM201–100N, Omega Engineering Inc., Stamford, CT, USA). Real-time acquisition of the force and joint position of the finger was performed. The index finger could be controlled independently; however, the middle, ring, and pinky could only be actuated concurrently. The robotic hand was controlled using the same MATLAB interface, which implemented a joint-position control scheme. This control scheme could adjust the speed and the final position of the finger joints. Prosthetic control was performed by the experimenter to ensure that the stiffness encoding methods were properly separated. This also ensured that subjects did not receive confounding information from proprioceptive or other tactile feedback when active control of the prosthetic hand was performed. Subjects were also visually and auditorily blinded to ensure differentiation could not be made based on those sources of sensory feedback.

To encode the object stiffness, the stimulation current amplitude was altered based on the force readings from the prosthetic’ s index or middle finger (figure 1(B)). As the individual prosthetic index or middle finger contacts an object, fingertip force was recorded by the embedded sensors. A sigmodal transfer function was used to convert the recorded finger force to the stimulation current amplitude delivered to the subject. A sigmoidal function was selected as it is common in psychometric testing for human tactile perception [29, 30]. The sigmoidal function was constructed based on the allowable stimulation current range, the minimum and maximum force readings, and the desired steepness of the function. The allowable current range for a given subject was based on the difference between the minimum current (i.e. the sensory threshold) and the maximum current (i.e. just below the motor threshold). The sensory and motor thresholds were subject specific, and were identified by adjusting the stimulation amplitude in steps of 0.1 mA until finger sensation or finger motion first occurs, respectively. The steepness and the force minimum and maximum (N) were kept consistent across subjects with values of 1, 0.5, and 2.75 for the index finger pinch, and 1, 0.5, and 6 for the middle finger pinch, respectively. These force values and corresponding finger forces fall within the range most commonly utilized by neurologically-intact individuals during activities of daily living [31]. The function used to calculate the necessary stimulation current level for a given force is shown in equation (1), where I, I_Max, and I_Min each represent the actual delivered, maximum, and minimum stimulation current, respectively. Steepness, actual force, maximum force, and minimum force were implemented using the variables k, F, F_Max, and F_Mi_n, respectively. A non-zero minimum force of 0.5 N was used due to a non-zero force reading when the finger flexed.

$$I(F)=\frac{\left(I_{Max}-I_{Min}\right)}{1+e^{\left(-k\ast \left(F-\frac{F_{Max}+F_{Min}}{2}\right)\right)}}+I_{Min}.$$ (1)

Procedure

The experiment began by delivering electrical stimuli to various electrode pairs until clearly evoked sensation in either the index or middle finger was reported by the subject. Index finger sensation was elicited during the index pinch grip, while middle finger sensation was elicited during the middle pinch grip. Stimulus intensity during single (index or middle) finger contact was regulated based on its corresponding sensors.

For the main experiment, object stiffness recognition was performed using two methods to determine which sensory encoding provided a more accurate gauging of different stiffness levels. The first method was based on the rate of change of the force/sensation with a fixed finger closing speed and a fixed finger peak force. In this setting, an object with a higher stiffness corresponded to a higher rate of change of the force. Three cubes with similar dimensions (5 cm × 5 cm × 4 cm) but with varying stiffness levels were used: a wooden block with a high stiffness (minimal deformation), a stiff foam with a moderate stiffness (2.9 N mm⁻¹), and a soft foam with a low stiffness (1.7 N mm⁻¹). The index pinch had a peak finger force of 2.5 N, while a peak force of 5 N was selected for middle finger pinch. Peak forces were selected based on the maximum consistent force achievable with the lowest stiffness object during respective pinch grasp. Figure 2 shows exemplar curves of the index finger force and current amplitudes for each object.

Figure 2.

Hand grip on objects of varying different stiffness with stiffness decreasing from left to right. (A) The graphs correspond to the finger force, (B) joint angle, (C) and the associated stimulation current (D) when using rate of change of stimulation amplitude as the stiffness encoding method.

The second method was based on the amplitude of force/sensation with a fixed flexion end-position. In this setting, an object with a higher stiffness corresponded to a higher peak force. During the testing of this encoding method, the subjects were instructed to differentiate object stiffness based on the peak stimulation amplitude. Because the wooden block was not deformable, a styrofoam cube was used in its place to represent the highest stiffness level (6.5 N mm⁻¹). The same stiff and soft foam cubes from the first method were used here. The final position of each finger was selected to be the maximum deformation of the styrofoam cube. The input and output curves are shown in figure 3. All the peak forces and final positions were determined based on preliminary evaluations and the force and measurement capabilities of each joint. These values were kept consistent across subjects.

Figure 3.

Hand grip on objects of varying different stiffness with stiffness decreasing from left to right. (A) The graphs correspond to the finger force, (B) joint angle, (C) and the associated stimulation current (D) when using peak stimulation amplitude as the stiffness encoding method.

For each stiffness encoding method, stiffness recognition was tested using three comparison techniques tested in three blocks. The three conditions were designed based on the uncertainty of how well the subjects could successfully identify the response to a given test, which can provide insight on how they were able to differentiate the various percepts elicited. The first condition ‘Ordering of 2 Objects’ evaluated the stiffness relation during the binary evaluation of two objects. The subjects were asked to identify if the first was stiffer, the second was stiffer, or if they had the same stiffness. This condition tested subjects’ ability to recognize differences between two objects, including the possibility that the objects possess the same stiffness. A total of 18 trials involving all the possible object combinations were evaluated. The second condition ‘Ordering of 3 Objects’ evaluated the identification of the stiffness order of three objects. The subjects were asked to rank order the stiffness levels after the three objects were offered sequentially. A total of 12 trials involving the three different objects sequentially in all possible random orders were evaluated. The final condition ‘Identification of Random Object’ evaluated the stiffness perception when identifying a random object during a single trial. The subjects were asked to identify the stiffness level (high, moderate, or low) of the object. This condition required the subjects to memorize the stiffness levels initially. The subjects then used that information to perceive the stiffness level of random objects. A total of 24 trials were tested. For each block, 10 s of rest time were provided between consecutive trials. The order of the testing blocks was randomized between subjects. A flowchart displaying the experimental protocols is shown in figure 4.

Figure 4.

Flowchart illustrating the experimental protocol.

Data analyses

To determine the accuracy of the stiffness identification for each combination of the encoding method and comparison technique, six confusion matrices were created that compared the actual relative stiffness level to the perceived stiffness level across all subjects. In each figure, the high, moderate, and low stiffness were presented by the numbers 1,2, and 3, respectively.

Statistical analyses

One sample f-tests were performed to determine if the recognition accuracy was significantly greater than chance values. A logit transformation was applied to the proportions prior to the statistical analysis due to the rightly skewed distribution of the data, since the percentage of accuracy is close to the upper bound of 1. This transformation can help correct this skewed distribution and lead to normal distribution of the residual [32]. For the three comparison techniques, the random chance of accurately identifying the stiffness relation, rank order, or individual stiffness level was 0.33, 0.16, and 0.33, respectively. Wilcoxon signed-ranked tests were also performed for each combination of encoding method and evaluation technique to ensure that there was no significant difference between the middle and index pinch tasks. Additionally, paired t-tests were performed for each technique to determine if one encoding method was significantly better than the other.

Results

Stiffness recognition based on ordering of 2 objects

We first evaluated if the subjects could identify the relative stiffness level between two given objects. Using binary comparisons, each subject was asked to report if the first object was stiffer, the second was stiffer, or if they had the same stiffness. Figure 5(A) shows the confusion matrices illustrating the actual stiffness pair presented and the perceived stiffness relation across all subjects, when the prosthetic hand pinched the objects at a fixed closing speed and stopped when a fixed force was reached. The high, moderate, and low stiffness were presented by the numbers 1,2, and 3, respectively. The results showed that all the subjects could identify the relative stiffness based on the rate of change in the fingertip force (stimulation amplitude) with an average accuracy of 91.7% ± 2.9%. During the second method (figure 5(B)), the amount of object deformation was fixed, and the object stiffness was associated with the peak force (peak stimulation amplitude). The subjects had similar success by correctly identifying 163 out of the 180 object pairs, resulting in an average accuracy of 90.6% ± 6.4% across subjects. Performance of both encoding methods were found to be significantly greater than the chance value (p < 0.001). In contrast, when comparing the two methods, no statistical difference was found between the two encoding methods (t = 0.49;p > 0.05).

Figure 5.

Confusion matrices quantifying the instances when comparing the perceived relative stiffness with the ground truth using the rate of change of stimulation amplitude (A) or the peak stimulation amplitude (B) as stiffness encoding. The average accuracy and the standard deviation (C) between subjects are displayed as well for both encoding strategies with an asterisk indicating significance of P < 0.001. ‘First’ means the first object was stiffer. ‘Second’ means the second object was stiffer. ‘Same’ means both objects are the same stiffness. ‘1’ corresponds to the stiffest object, ‘2’ corresponds to the moderate stiff object, and ‘3’ corresponds to the least stiff object. There was a maximum of 20 trials or instances for each stiffness pair across all subjects.

In addition, the type of recognition errors was summarized as well. Specifically, most of the recognition errors arose from the trials where two objects with different stiffness levels were recognized as the same. This situation occurred in 9 out of 15 and 10 out of 17 errors in the two encoding methods. It is important to note, however, that no object pairs with the stiffest and the least stiff object were incorrectly identified.

Stiffness recognition based on ordering of 3 objects

We then evaluated if the subjects could identify the relative stiffness level among three objects of varying stiffness. The confusion matrices presented in figure 6 illustrate all possible three object sequences and their respective level of stiffness recognition across subjects. The results showed that both stiffness encoding methods led to similar performance in stiffness recognition. Specifically, with a fixed peak force, the subjects were able to correctly identify the stiffness order in 114 out of 120 trials with an average accuracy across subjects of 95.0% ± 10.5%, based on the rate of change in the fingertip force (stimulation amplitude). With the peak fingertip force as the stiffness encoding, the subjects were able to correctly identify the stiffness order with an accuracy of 91.7% ± 11.1%, or 110 out of 120 accurate trials. The confusion matrices also showed that the types of errors were largely composed of incorrectly identifying adjacent stiffness levels. Both encoding methods were found to be significantly greater than the chance value (p < 0.001), with no statistical difference between the two encoding methods (t = 0.25; p > 0.05).

Figure 6.

Confusion matrices quantifying the instances when comparing the perceived relative stiffness order with the ground truth using the rate of change of stimulation amplitude (A) or the peak stimulation amplitude (B) as stiffness encoding. The average accuracy and the standard deviation (C) between subjects are displayed as well for both encoding methods with an asterisk indicating significance of P < 0.001. ‘1’ corresponds to the stiffest object, ‘2’ corresponds to the moderate stiff object, and ‘3’ corresponds to the least stiff object. There was a maximum of 20 trials or instances for each condition across all subjects.

Stiffness recognition based on identification of random object

Lastly, we evaluated if the subjects could identify the stiffness level of a single object given randomly during each trial. The subjects were asked to report whether the perceived stiffness corresponded to a high, moderate, or low stiffness object. The confusion matrices of stiffness recognition across subjects are shown in figure 7.

Figure 7.

Confusion matrices quantifying the instances when comparing the perceived stiffness with the ground truth using the rate of change of stimulation amplitude (A) or the peak stimulation amplitude (B) as stiffness encoding. The average accuracy with the standard deviation as error bars (C) between subjects are displayed as well for both encoding strategies with an asterisk indicating significance of P < 0.001. There was a maximum of 80 trials or instances for each condition across all subjects.

The results showed that the majority of the perceived stiffness levels were correctly identified for both encoding methods. Specifically, with the rate of change of stimulation amplitude as stiffness encoding, 222 out of 240 trials were correctly identified, resulting in an accuracy of 92.5% ± 7%, based on the rate of change in the fingertip force. Similarly, 220 out of the 240 trials were correctly identified with peak stimulation amplitude, resulting in an accuracy of 91.7% ± 7.9%. Both encoding methods were significantly greater than the chance value (p < 0.001). Statistical analysis showed that the results were similar between the two encoding methods (t = 0.68; p > 0.05). For all combinations of encoding methods and comparison technique, Wilcoxon sign ranked tests found no significant difference across middle and index pinch tasks as well.

Discussion

This study sought to identify if stiffness recognition could be performed using transcutaneous nerve stimulation delivered to the proximal segments of the median and ulnar nerves. Two stiffness encoding methods were evaluated to determine which method was more critical for stiffness recognition. Specifically, using an electrode grid, different electrical field distributions can be induced, which can then activate different afferent nerve fibers producing distinct haptic sensations in the subject’s hand. Electrode pairs evoking either middle or index finger haptic sensation were utilized in this study, with the current amplitude being modulated by the fingertip force of the DEKA hand. Our results demonstrated that both encoding methods allowed the subjects to accurately (>90%) identify the stiffness levels of different objects. The results suggest that it is possible to elicit haptic sensations describing various object stiffness levels using transcutaneous nerve stimulation. The haptic feedback associated with object stiffness can help to improve the performance of dexterous movement, ensure successful object manipulation/interactions, and potentially promote user embodiment when using prosthetic arms/hands.

The first stiffness encoding method allowed the subjects to differentiate stiffness levels based on the rate of change of the sensation intensity when the peak force (stimulation amplitude) was maintained constant across different objects. The fixed peak force ensured that the judgment was not assisted based on varying levels of peak sensation intensity. Our results demonstrated that different object stiffness levels could be correctly identified during all the three evaluation techniques with accuracies >90%. The subjects did not require any training before the actual experiment, and each subject’s decisions were made immediately following the end of each trial, suggesting that the nerve stimulation provided was highly informative as well.

Previous studies have shown that stiffness discrimination could be performed using similar encoding methods with accuracies being slightly lower [23, 24] or comparable with our results after multiple sessions [25]. Differences in discrimination accuracy may be caused by multiple factors. First, the experimenter controlled the prosthesis in our current study. Subject’s stiffness discrimination was exclusively based on sensation without any sensorimotor integration process. A direct control of the prosthetic hand can provide the user with additional information during sensorimotor integration. Alternatively, as subjects control the prosthetic hand, differences in grasp trajectory caused by the user can lead to altered perception during discrimination. For example, variations in grasp force and aperture could likely affect the rate of change of the force profile, which can affect the individual’s perception of the stiffness. Further studies need to evaluate the role of sensorimotor integration in sensory perception. Second, the accuracy of stiffness recognition may also be affected by the range of allowable stimulation parameters for a given individual and stimulation technique. The just-noticeable difference (JND) describes the smallest change of a stimulation parameter that can be perceived by the subject. As the range of allowable stimulation parameters decreases, the number of distinct percepts, determined by the JND, decreases as well, thereby limiting the distinguishability of objects with similar stiffness levels. Our results support this notion as the highest recognition accuracy was observed in the two individuals with the widest range of stimulation amplitudes. Lastly, different objects were used across studies, which can also affect the accuracy of the stiffness recognition. Differences in object stiffness will affect the distinguishability between objects. It is expected that objects of similar stiffness levels would be harder to differentiate compared to those with large differences. Overall, our results revealed that the non-invasive stimulation targeting major nerve bundles can reach comparable accuracy of object stiffness recognition, compared with implantable nerve interface techniques in earlier work [24, 25]. Our non-invasive approach can reduce the concerns with stability and post-surgery care that come with invasive approaches, and also reduce the potential for myoelectric control interference of distal transcutaneous nerve stimulation.

The second stiffness encoding method allowed individuals to differentiate stiffness based on the peak force (stimulation intensity) with identical levels of deformation. This encoding method for stiffness identification has never been evaluated using elicited sensory feedback. Compared with the previous encoding method (rate of change of sensation), similar recognition accuracy was observed, which indicates that each subject can discriminate between force levels. Future work will need to be conducted to identify the potential JND to determine the discrimination resolution of varying stiffness levels. As one would expect, a limitation to this encoding method is that the level of deformation needs to be kept consistent in order to properly evaluate various object stiffness levels.

Throughout the experiment, visual feedback was blocked to strictly evaluate subjects’ ability to discern stiffness based on the delivered tactile sensation. Integration of vision with this tactile sensation can potentially allow for improved discrimination of stiffness, as information pertaining to finger position/movement will be available. Information received through visual stimulus has been shown to assist in discriminating stiffness [12]; however, a lack of visual cues or in some cases altered/incorrect perception can result in errors as detailed by multiple studies [13–16, 33]. For this reason, subjects were blinded in this study to examine the extent in which stiffness could be determined using both encoding methods. In addition, with active control of the robot, proprioceptive feedback can also be integrated with the tactile information, which could allow for a better estimate of object stiffness. Horch et al and D’Anna et al [24, 34] have shown that the addition of proprioceptive feedback, delivered using direct nerve stimulation, can improve stiffness identification with objects of varying sizes and levels of stiffness. Additionally, if kinesthetic feedback cannot be evoked with our technique, vibrotactile or electrotactile devices can be employed as a substitution to provide information associated with joint angles. Multiple studies have demonstrated that these tactors are simple to implement and can be effective when used to deliver proprioceptive stimulus to an individual [26, 35–37].

For example, a study utilizing current amplitude-modulated electrotactile feedback demonstrated that object stiffness, along with weight and size, could be correctly identified when combining two sources of stimuli corresponding to finger aperture and force [26]. The success of this study suggests that providing useful non-invasive multisensory feedback might improve control as well as provide additional insight about various object properties. In this earlier study, the feedback was delivered to the skin under the electrode. In contrast, the nerve stimulation used in our current study elicited haptic sensation in the referred fingers directly, which could provide more intuitive feedback, which may require less training and cognitive processing due to locational similarities [38]. Similarly, transcutaneous nerve stimulation on the forearm of the amputees can activate the afferent fibers in the distal branches of the median and ulnar nerves [23]. With varying stimulation parameters corresponding to different object stiffness, the amputees can recognize object stiffness with high accuracy. However, stimulation of distal nerve branch can potentially induce interference on myoelectric signal recordings for future studies.

Limitations and future work

Limitations of this study include the lack of user-control of the prosthesis, the lack of amputee subjects, potential for confounding factors during stiffness recognition, and the absence of stimulation adjustment to potential sensory adaptation after continuous stimulation. As previously mentioned, the lack of user-control may result in varied recognition accuracies. In this study, control was delegated to the experimenter in order to ensure the two encoding methods were properly separated. Additionally, this decision ensured that confounding factors, resulting from individual’s proprioceptive feedback during muscle activation and finger movements, could provide either converging or interfering information to the users, which could bias the results. Studies have also shown that sensory perception may be affected during the active control of hand motion due to added cognitive load [39]; however, minimum difference has been reported when comparing the discriminative ability between these two scenarios [40, 41]. Additionally, simply replaying predefined stimulation profiles could still have addressed our current research question. Utilization of the robotic hand accounted for the potential variability that may arise from the placement of the objects, sensor recordings from the robotic hand, or the delay in delivering the real-time stimulation. As a result, an experimenter-controlled approach appeared to be the appropriate decision for this study. Clearly, further studies are necessary to evaluate the performance of stiffness recognition when users directly control the prostheses, especially for clinical applications. Given that both encoding methods showed similar performance, the users may actively control the prosthetic hand using either strategies by slowly adjusting the grip aperture to produce different force profiles. Earlier work has shown that the somatosensory cortex representation of the phantom hand is largely stable [42] even years after amputation. In a previous study, we have also shown that evoked sensory perceptions through transcutaneous nerve stimulation are similar between intact individuals and arm amputees [27], which suggests that results demonstrated in intact individuals can be translatable and characteristic to those expected in amputees. Nonetheless, further studies including upper limb amputees are necessary to evaluate the performance of stiffness recognition.

The discrimination of stiffness was performed based on specific stimulation patterns. In other words, the subjects associated the stimulation patterns with the object stiffness. Indeed, the stimulation patterns matched the physical stiffness parameters, rather than random stimulation patterns. The elicited haptic feedback was not natural compared with the biological feedback. But the subject was able to correctly identify the patterns immediately after the stimulation without any prior training, which suggested that the information was more intuitive than substituted sensory information, which typically requires training and the recognition process was also slow. During the experiment subjects were asked to differentiate the stiffness of the three objects based on the rate of stimulus change or the peak stimulation amplitude. During either process, it was emphasized that discriminations of the percepts should be performed by solely the rate of change or the peak stimulation amplitude. Despite the explicit instructions to the subjects, there is still a possibility that other factors may have been utilized, such as stimulation duration for the first method or the rate of change for the second method. As the peak deformation was held constant for the second method, the rate of change of the stimulus would vary with different object stiffness levels. If the flexion velocity of the finger was altered to ensure identical rate of stimulus change, then stimulation duration would be a confounding factor instead. Future studies are necessary to isolate these confounding variables and further evaluate user performance. Lastly, sensory adaptation over time can alter the perception of various elicited sensations with continuous stimulation. Investigation on sensory adaptation have been performed with implanted electrode nerve stimulation, which suggested that sensory adaptation may be partly mediated by a desensitization of the mechanotransduction sites [43]. Additionally, sensory adaptation has been evaluated using skin surface electrotactile stimulation [44–46], with one study [46] showing that adaptation can be delayed with intermittent stimulation. Future studies are needed to understand the occurrence of sensory adaptation, which can help develop necessary compensatory stimulation paradigms to accommodate the expected changes.

Conclusion

In summary, our findings demonstrated that stiffness recognition could be performed via transcutaneous nerve stimulation targeting the proximal segment of the nerve bundles. Two stiffness encoding methods, the rate of change of stimulation amplitude and the peak stimulation amplitude, showed similar performance, suggesting that either could be implemented individually or in combination with other feedback mechanisms when using a sensorized prosthesis or remotely controlling a robotic device. Implementation of haptic feedback allowing stiffness recognition can better replicate the sensations perceived during natural touch. It can also lead to increased user intuition and embodiment, which can help improving current prosthesis acceptance rate.