Vibrotactile Sensory Substitution for Object Manipulation: Amplitude versus Pulse Train Frequency Modulation

Abstract

Incorporating sensory feedback with prosthetic devices is now possible, but the optimal methods of providing such feedback are still unknown. The relative utility of amplitude and pulse train frequency modulated stimulation paradigms for providing vibrotactile feedback for object manipulation was assessed in 10 participants. The two approaches were studied during virtual object manipulation using a robotic interface as a function of presentation order and a simultaneous cognitive load. Despite the potential pragmatic benefits associated with pulse train frequency modulated vibrotactile stimulation, comparison of the approach with amplitude modulation indicates that amplitude modulation vibrotactile stimulation provides superior feedback for object manipulation.

Introduction

Due to advances in sensing technology, it is possible to sense fingertip forces in real-time [1–5], with many groups successfully integrating these technologies into prosthetic hands, e.g., the cybernetic hand [3, 4]. Providing this information to users of prosthetic hands has the potential to improve control [6, 7] and quality of life [8, 9]. However, it is not yet clear how to provide this sensed feedback to users to optimally integrate the information for sensorimotor control.

In order to quantitatively examine and compare possible methods of delivery, we have designed a simple robotic and virtual interface in which visual and haptic feedback can be experimentally controlled [10]. Vibrotactile stimulation is cheap, non-invasive, and could be easily implemented into existing prosthetic technologies as augmentative sensory feedback [11, 12]. However, previous work has shown inconsistent performance of users [2, 13–16]. This inconsistency could be, in part, due to differences in stimulation methodologies. Previous work utilizing augmentative vibrotactile feedback for users of prosthetic hands has focused on two main stimulation paradigms, which will be referred to as “amplitude modulated” and “pulse train frequency modulated.”

The modulation of stimulation amplitude is perhaps the most obvious paradigm, and has accordingly received the most attention. In a case study utilizing the Boston Arm in 1970, Mann & Reimers used vibrotactile stimulation on the participant’s stump to signal tactual limb angle, combined with force feedback at the stump during an arm positioning task [13]. The stimulation was a 100 Hz square wave from two stimulators, each of which produced stimulation amplitudes proportional to the angle of the limb. The vibrotactile stimulation was found to improve the accuracy of the participant’s ability to position his arm. Pylatiuk et al. [2] presented 5 users of myoelectric prosthetic hands with vibrotactile stimulation directly on the prosthesis or on the skin of the residual limb during a simple object grasp task. The stimulation was provided through a motor in which vibration amplitude and, to some degree, stimulation frequency were linked to the level of force produced during their grasp. Stimulation frequency ranged from 50 Hz to 80 Hz. Their work showed that users were able to decrease their contact forces during grasping when this feedback was available [2]. Similarly, in our past work, we have found that unimpaired individuals were able to use the combination of amplitude modulated vibrotactile stimulation and vision to perform virtual object manipulation with increased performance than with vision alone [16].

On the other hand, Patterson & Katz did not find amplitude modulated vibrotactile feedback to be effective [15]. In their study, 25 unimpaired participants broken into five groups received one of five types of feedback related to the applied voltage of a robotic arm: 1) a pressure cuff placed on the upper arm, 2) vibrotactile stimulation of the upper arm, 3) vision, 4) pressure and vision, and 5) vibrotactile stimulation and vision. During vibrotactile feedback, the amplitude of vibration was modulated proportionally to the applied voltage of the robotic arm; the frequency of vibration was not reported. Participants performed gripping trials. These consisted of a reference grip in which they chose the force, followed by a replication grip in which they attempted to match the force from the previous grip. Results did not show a difference in performance between the group that received vibrotactile and vision and the group that received vision alone [15].

Chatterjee et al. performed the only research to date using pulse train frequency modulated stimulation [14]. Eight unimpaired individuals were provided with vibrotactile stimulation on the upper arm during use of a myoelectric prosthesis simulator to complete an interactive force-matching task. The stimulus paradigm represented force by modulating the pulse rate of a square wave, using a 200-Hz carrier frequency. In their scheme, increases in pulse rate indicated an increase in force. A “low” grasping force was indicated by a pulse rate of approximately 1 Hz, whereas a “high” grasping force was represented by a pulse rate of approximately 3 Hz. Unfortunately, use of this feedback did not result in a consistent overall reduction in force-matching error [14]. However, Chatterjee et al. argued the potential pragmatic merits of pulse train frequency modulated stimulation, citing inconsistency of amplitude modulation schemes due to differences in contactor area and mounting pressure [14]. In addition, amplitude modulation by definition requires some stimulation at low amplitudes. These low amplitudes may be beneath the detectable threshold for some users due inter-operator differences, such as body fat level [17]. Thus, determining the maximal dynamic range of amplitude stimulation may require the feedback system to be calibrated for individual users. Using pulse train modulated stimulation allows the use of stimulation amplitudes that are easily felt by all users, since the desired feedback is encoded using frequency-based measures.

The focus of this paper is to examine the relative utility of amplitude modulated and pulse train frequency modulation stimulation paradigms for providing augmentative vibrotactile feedback for object manipulation. We compare the two approaches using a robotic interface with which both visual and direct haptic feedback can be experimentally controlled [16]. We compared virtual object manipulation using the two different stimulation paradigms as a function of training block (presentation order) and a simultaneous cognitive load. We hypothesized that performance would be decreased during a simultaneous cognitive task, and that participants would show increases in performance throughout training with either presentation scheme. We also hypothesized that the cognitive load could show differential effects on the task performance, with greater reduction in performance during the theoretically less natural pulse train frequency stimulation paradigm than in the amplitude modulation paradigm.

Methods

Participants

Participants were 10 adults (9 right-handed, 1 left-handed; 8 male, 2 female; mean age = 21.9 years, SD = 4.1 years). The individuals reported normal hand function, with no complaints related to their hands. Informed consent was obtained from all participants in compliance with the Institutional Review Board of the University of Washington.

Virtual Environment and Experimental Design

Participants interacted with a virtual environment by placing their right index finger into a custom splint attached to the end effector of a PHANTOM Premium 1.0 robotic device (Sensable Technologies, Inc., Woburn, MA). Their goal was to apply appropriate normal force to a virtual object to allow for translation, and to drag it to a target without breaking it as quickly as possible. This task was specifically chosen due to the known difficulties of prosthetic hand users with appropriately applying normal force to delicate objects, such as picking up and manipulating a disposable plastic cup [18]. The task was implemented to be difficult for participants so that any differences based on feedback modality would be apparent.

The PHANTOM was used to measure the position of the tip of the finger, and was located inside a projection system. This system consisted of a frame above the PHANTOM, supporting an inverted video monitor. The video monitor was positioned at 45º toward the participant, and a mirror was placed between the virtual environment and the monitor to permit reflection of images from the monitor to the user (see Fig. 1). Participants sat in front of the projection system and their hand was free to move about the 3D workspace.

Fig. 1.

Methodology. The left panel shows a schematic of the physical set-up of the experiment. An inverted video monitor was positioned at 45º toward the participant, and a mirror was placed between the virtual environment and the monitor to reflect visual feedback to the user. The right panel shows a screenshot of the virtual environment used during experimentation. Participants were asked apply appropriate normal force to a virtual object (box) to drag it to a target at the right of the screen as quickly as possible without breaking it. Finger location was shown as a small white ball.

The virtual environment was programmed in C++, with graphics driven by OpenGL. During interaction, one of two possible virtual objects was located at the left end of the workspace (see Fig. 1). The upper surface of these objects (referred to as boxes) had distinct stiffness characteristics. Box stiffness functions were scaled versions of a fit to the force-displacement curve acquired empirically by pushing on a disposable plastic cup. The difference between the two boxes was signaled to the participant by box color, red or blue. The stiffness characteristics of each box were defined as:

$$F_{\mathit{blue}}=\{\begin{array}{cc}(0.34\text{N}/\text{cm})x,& \text{if}x<1.7\text{cm}\\ (4.65\text{N}/{\text{cm}}^2)x^2-(14.33\text{N}/\text{cm})x+11.55,& \text{if}x>1.7\text{cm}\end{array}$$

$$F_{\mathit{red}}=\{\begin{array}{cc}(0.68\text{N}/\text{cm})x,& \text{if}x<1.7\text{cm}\\ (9.59\text{N}/{\text{cm}}^2)x^2-(30.67\text{N}/\text{cm})x+25.56,& \text{if}x>1.7\text{cm}\end{array}$$

The virtual normal force of the blue box, F_blue, and the virtual normal force of the red box, F_red, were defined with x as the displacement of the finger into the box in the normal direction, in centimeters. The stiffness characteristics were decreased for the blue relative to the red box.

The virtual force required to overcome friction to translate each box, F_move, was arbitrarily defined as 1.2 times the virtual force at the displacement of 1.7cm. The virtual force threshold to “break” each box, F_break, was defined as 0.75N greater than F_move. Virtual normal force applied to the box between F_move and F_break allowed the participant to slide the object to a target located 30 cm to the right of the workspace. The difference in stiffness between the two boxes resulted in different allowable displacements of the finger during motion (1.6 mm for the red box and 2.7 mm for the blue box) in the direction of the virtual normal force. Thus, for most users, the red box was more difficult to move successfully.

During the task, users received visual feedback consisting of a real time depiction of the location of the finger in the virtual environment, and the current position of the box (see Fig. 1). Finger position was indicated by a small sphere. The finger location was occluded during penetration of the box, and deformations of the box were not shown. This level of feedback was provided to the user to approximate the real-life visual feedback of task performance available to users of prosthetic limbs during object manipulation.

Vibrotactile Feedback

In both conditions, vibrotactile stimulation at 250 Hz was provided using a C2 tactor (Engineering Acoustics, Inc.) mounted to the right lateral upper arm and secured with an elasticized cloth bandage. A 250-Hz carrier frequency was used since human glabrous skin has been shown to be maximally sensitive to vibrotactile stimulation at this frequency [19, 20]. During interaction with the virtual environment, increases in virtual normal force were translated to one of two types of vibrotactile feedback. In the first paradigm (amplitude modulated), increases in virtual normal force were translated into increases in the amplitude of continuous vibrotactile stimulation. The maximum amplitude of vibration used was approximately 400 μm. The second paradigm (pulse train frequency modulated) translated increases in virtual normal force into increased pulse train frequencies. During the pulse train frequency modulated paradigm, stimulation was at a constant amplitude that was easily felt by all participants (approximately 340 μm). The stimulation was applied as pulse trains, with periods ranging from 430 ms (lowest virtual force) to 146 ms (highest virtual force) and a 50% duty cycle (2.3 – 6.8 Hz). Figure 2 shows a schematic of the two feedback paradigms.

Fig. 2.

Vibrotactile Feedback Paradigms. The schematic shows the vibrotactile response to virtual force during the two vibrotactile feedback paradigms. The carrier frequency in both paradigms was a 250 Hz sine wave.

Cognitive Load

In order to determine the differential effects of a simultaneous load during the use of the two types of vibrotactile feedback, an auditory 2-back test was used [21]. During this test, participants listened to random 16 digit strings and responded verbally to identify any numbers repeated with only one intervening number. Each string was 16 s in length. To ensure that participants understood the task, participants practiced 20 sets of this task without the motor task prior to experimentation. During interaction with the virtual environment, participants were asked to complete the cognitive task while simultaneously completing the motor task. Although the length of each trial was dependent on task performance, each number string was of a specific finite length. Therefore, completion of the entire 16 digits of the cognitive task was not achieved if the box was broken in less than 16 s.

Experimental Design

Over approximately 2 – 3.5 hours (including breaks), each participant completed 160 trials of interaction with the virtual system. Each participant completed 2 blocks of 80 trials. Each block utilized one of the two stimulation paradigms. The order of the stimulation paradigm was varied, with half of the participants (N = 5) receiving amplitude modulated stimulation in the first block, followed by the pulse train frequency modulated stimulation in the second block. The presentation order was reversed for the other 5 participants.

Within the 80 trials for block, trials were presented in 20 segments, randomized within segment by box (blue, red), and cognitive load (ON, OFF). Trials ended when the box reached the target or was broken. At the end of each trial, the participant was asked to rate the difficulty of completing the motor task on an ordinal scale 1 – 5, in which 1 was very easy and 5 very difficult. Participants were required to take 5 minute breaks every 40 trials, and were encouraged to take breaks between any trials to avoid fatigue. Participants generally took between 4 – 6 breaks during the experiment.

Participants wore noise-canceling headphones (Bose, Framingham, MA) during experimentation to present the stimuli for the cognitive task, and to provide low-level masking noise. Vibrotactile feedback was provided at 250Hz, which is in the range of human hearing. The masking noise and noise-canceling headphones were used to ensure that participants were not using any auditory feedback from the tactor to complete the motor task.

At the end of the experiment, each participant rated which of the two vibrotactile stimulation paradigms they preferred.

Analysis

Performance variables used were box displacement (total distance toward the target that participants were able to translate the box during the trial), average box velocity (box displacement normalized by trial duration), and difficulty ratings.

Data analysis to determine the performance variables for each trial was performed using MATLAB (Mathworks, Natick MA), and statistical analysis was performed using Minitab Statistical Software (Minitab Inc., State College, PA). A 4 factor analysis of variance (ANOVA) was performed to assess the effects of stimulation paradigm (feedback), cognitive task, presentation order (block), and box, as well as the interactions of feedback × block, cognitive task × feedback, and block × cognitive task on the performance variables. Post hoc two-sided Tukey’s Simultaneous tests were used when appropriate. A Chi-Square Goodness-of-Fit Test was performed on the ratings of user preference for the two stimulation paradigms, against the null hypothesis that the two paradigms were equally preferred. All statistical analyses were performed using an alpha level of 0.05 for significance.

Results

Overall, out of 1600 combined trials, participants were able to successfully move the box to the target 136 times (8.5% of attempts). During these successful attempts, the average distance achieved was the full range of the task (30 cm), the average velocity was 0.63 cm/s (SE = 0.029 cm/s), and the average difficulty rating was 1.9 (SE = 0.07). Conversely, during unsuccessful attempts, the average distance achieved was 4.07 cm (SE = 0.17 cm), the average velocity was 0.21 cm/s (SE = 0.01 cm/s), and the average difficulty rating was 3.7 (SE = 0.03). Figure 3 shows the effects of feedback, block, and cognitive task on box displacement, box velocity, and difficulty ratings.

Fig. 3.

Effect of feedback, block, and cognitive task on performance variables. Panel A shows results for box displacement, Panel B for box velocity, and Panel C for difficulty ratings. Error bars show the 95% confidence interval of the mean. Statistically significant (p < 0.05) increases in box displacement and velocity and decreases in difficulty between blocks 1 and 2 were seen for amplitude modulated feedback, but not for pulse train frequency modulated feedback.

ANOVA of box displacement showed a significant effect of box, feedback, block, and the interaction feedback × block, but did not show a significant effect of the cognitive task, or the interactions cognitive task × feedback and block × cognitive task (see Table 1). Post hoc testing indicated that participants could move the blue box with significantly larger displacements than the red box with a mean of 9.3 cm (SE = 0.4 cm) versus a mean of 3.3 cm (SE = 0.2 cm). In addition, amplitude modulated feedback enabled significantly greater box displacements, with a mean of 7.8 cm (SE = 0.4 cm) versus a mean of 4.7 cm (SE = 0.3 cm) using pulse train frequency modulated feedback. Participants were able to produce significantly greater box displacements during the second block of the experiment (mean = 7.1 cm, SE = 0.4 cm) versus the first block (mean = 5.4 cm, SE = 0.3 cm). In addition, post hoc testing based on the significant interaction between feedback and block showed that although box displacement resulting from amplitude modulated feedback during block 2 (mean = 9.6 cm, SE = 0.6 cm) was significantly greater than that resulting from amplitude modulated feedback during block 1 (mean = 6.1 cm, SE = 0.5 cm), pulse train frequency modulated feedback did not show a significant increase in displacement between block 1 (mean = 4.8 cm, SE = 0.4 cm) and block 2 (mean = 4.6 cm, SE = 0.4 cm).

TABLE 1.

Results of ANOVA of Box Displacement

Source	DF	F	p
Box	1	182.4	<0.001
Cognitive Task	1	0.7	0.39
Feedback	1	49.4	<0.001
Block	1	14.4	<0.001
Feedback × Block	1	16.9	<0.001
Cognitive Task × Feedback	1	1.3	0.25
Cognitive Task × Block	1	1.6	1.21

ANOVA of box velocity showed a significant effect of box, feedback, block, cognitive task, and the interaction feedback × block, but did not show a significant effect of the interactions cognitive task × feedback and block × cognitive task (see Table 2). Post hoc testing indicated that participants could move the blue box significantly faster than the red box with a mean of 0.34 cm/s (SE = 0.012 cm/s) versus a mean of 0.15 cm/s (SE = 0.006 cm/s). Box velocity during amplitude modulated feedback was significantly greater, with a mean of 0.30 cm/s (SE = 0.011 cm/s) versus a mean of 0.19 cm/s (SE = 0.009 cm/s) using pulse train frequency modulated feedback. Participants were able to move the box with significantly greater velocities during the second block of the experiment (mean = 0.30 cm/s, SE = 0.01 cm/s) versus the first block (mean = 0.19 cm/s, SE = 0.01 cm/s). During the simultaneous cognitive task, participants moved the box with significantly smaller velocities (mean = 0.23 cm/s, SE = 0.01 cm/s) than when performing the motor task in isolation (mean = 0.26 cm/s, SE = 0.01 cm/s). Post hoc tests indicated that box velocities during amplitude modulated feedback were significantly greater during block 2 (mean = 0.39 cm/s, SE = 0.02 cm/s) than during block 1 (mean = 0.21 cm/s, SE = 0.01 cm/s). However, pulse train frequency modulated feedback did not show a significant difference between block 1 (mean = 0.17 cm/s, SE = 0.01 cm/s) and block 2 (mean = 0.21 cm/s, SE = 0.01 cm/s).

TABLE 2.

Results of ANOVA of Box Velocity

Source	DF	F	p
Box	1	227.4	<0.001
Cognitive Task	1	4.1	0.04
Feedback	1	70.0	<0.001
Block	1	68.6	<0.001
Feedback × Block	1	25.6	<0.001
Cognitive Task × Feedback	1	0.0	0.96
Cognitive Task × Block	1	0.0	0.91

ANOVA of difficulty ratings showed a significant effect of box, feedback, block, and the interaction feedback × block, but did not show a significant effect of the cognitive task, or the interactions cognitive task × feedback and block × cognitive task (see Table 3). Post hoc testing indicated that participants could move the blue box with significantly less difficulty than the red box with a mean of 3.3 (SE = 0.04) versus a mean of 3.8 (SE = 0.03). Difficulty was significantly reduced during amplitude modulated feedback (mean = 3.3, SE = 0.04) than during pulse train frequency modulated feedback (mean = 3.8, SE = 0.03). Difficulty ratings were significantly lower during the second block of the experiment (mean = 3.4, SE = 0.04) versus the first block (mean = 3.7, SE = 0.03). Further, post hoc testing based on the significant interaction between feedback and block indicated that although difficulty ratings resulting from amplitude modulated feedback during block 2 (mean = 2.9, SE = 0.06) were significantly lower than those resulting from amplitude modulated feedback during block 1 (mean = 3.6, SE = 0.04 cm), pulse train frequency modulated feedback did not show a significant difference in difficulty ratings between block 1 (mean = 3.8, SE = 0.05) and block 2 (mean = 3.9, SE = 0.04).

TABLE 3.

Results of ANOVA of Difficulty Ratings

Source	DF	F	p
Box	1	111.3	<0.001
Cognitive Task	1	1.1	0.30
Feedback	1	118.3	<0.001
Block	1	43.9	<0.001
Feedback × Block	1	61.5	<0.001
Cognitive Task × Feedback	1	0.6	0.42
Cognitive Task × Block	1	0.8	0.37

Out of the 10 participants, 6 users preferred amplitude modulated stimulation versus 4 users who preferred pulse train frequency modulated stimulation. This difference was not found to be statistically significant based on the results of a Chi-square test (N=10, DF=1, χ²=0.4, p=0.53).

Discussion

Overall results show a strong effect of both stimulation paradigm and training block. With training, users were better able to utilize amplitude modulated vibrotactile stimulation to perform object manipulation. On average, participants were only able to successfully move the box to the target in 8.5% of attempts, which is likely a direct result of our intent to create a difficult task so that potential effects of stimulation paradigm would not be masked. Although this is the first study to directly compare these two methods, our results coincide well with the literature. Chatterjee et al. found that pulse train frequency modulated vibrotactile feedback to perform a force-matching task did not result in a consistent overall reduction in error [14], and we have previously found that amplitude modulated vibrotactile feedback could be utilized by individuals to improve performance on an object manipulation task [16].

Effects of the Stimulus Paradigm

Encoding force through changes in pulse train frequency has potential advantages due to the possible inconsistency of amplitude modulation schemes in real-world application due to differences in contactor area and mounting pressure. However, our results found that individuals were able to achieve greater box displacements and velocities with decreased difficulty during amplitude modulated feedback relative to pulse train modulated feedback. It is not entirely clear why participants failed several seconds more quickly while using the pulse modulation scheme than during amplitude modulation. The outcome measures of distance and velocity were not perfectly normal, but had a positive skewness, with many trials ending with very short distances. During pulse train frequency modulation the distribution of distances were more positively skewed than during amplitude modulation, with more extremely small distance values.

One possible reason for differences in the ability to use the different stimulation for object manipulation could be related to differences in the perceptual dynamic range of the two paradigms. If too small of a dynamic range was utilized in the pulse train frequency stimulation paradigm, that might lead to poor differential perception and lower performance of the motor task. Difference limens for pulse train frequencies in the range used in the present study (2.3 – 6.8 Hz) have been shown to vary in a near linear fashion as a function of frequency [22] when presented to the finger. The average just noticeable difference (JND) over this range is approximately 0.13 Hz [22], leading to approximately 32 discriminable steps in pulse rate. Conversely, vibrotactile amplitude modulation from threshold to approximately 400 μm applied on the chest is thought to provide approximately 15 discriminable steps [23]. Although differences in carrier frequency and body location may affect the number of discriminable steps, there is no reason to suspect that differences in dynamic range are to blame for the poor performance found in our study of individuals utilizing the pulse train frequency modulated stimulation paradigm.

One possible weakness of pulse train frequency modulated stimulation is the loss of precise timing information. Feedback corresponding to changes in virtual force is delayed by the length of the previous stimulation period. In the current paradigm, the minimum delay to the user is 146 ms (at the highest virtual forces), and the maximum delay is 430ms (at the smallest virtual forces).

Lastly, differences in the ability of users to successfully integrate the vibrotactile feedback may be due to differences in the perceived naturalness of the stimulation paradigms. Amplitude modulated stimulation provides a graded stimulus to replace the expected graded force feedback. Conversely, pulse train frequency modulation provides a discrete stimulation pattern that is quite different temporally from the expected force feedback. During amplitude modulation, the graded amplitude of the vibrotactile information is more like the original signal, so may be integrated more effectively for the object manipulation task. This theory is congruent with the past work of Patterson and Katz, who found that a more natural feedback modality provided better sensory feedback information for grasping [15], and with previous research in cross-modal transfer (CMT; skills acquired during learning of a motor task with specific stimulus information being transferred to a new stimulus condition [e.g., 24, 25]), which increases in CMT when the two stimuli shared the same temporal pattern [24].

The effects of stimulation paradigm produced clear effects on all performance variables, with amplitude modulation producing better performance than pulse train frequency modulation. However, when users were asked to rate their preference for stimulation, only 6 users preferred amplitude modulated stimulation to pulse train frequency modulated stimulation. We have previously noted a lack of correlation between quantitative measures of motor task performance (e.g., box velocity and displacement) and user ratings of difficulty [16]. This difference in objective outcomes and user preference highlights the importance of collecting both types of measures in future work.

Effects of the Cognitive Task

The addition of a simultaneous cognitive task led to decreased box velocities and increased difficulty ratings, but did not show a significant effect on box displacements. These results are consistent with previous work [16]. The intent of including a simultaneous cognitive task to the study design was to unmask potential differences between seemingly similar feedback paradigms so that their merit for use in daily life could be discriminated. Specifically, we hypothesized that the addition of a cognitive load would lead to a greater reduction in performance during pulse train frequency modulated feedback than during amplitude modulated feedback. However, none of the performance variables showed a statistically significant effect of cognitive task × feedback (see Tables 1–3). Thus, although differential effects of the cognitive task were not seen as a function of the feedback paradigm, the results of the present study indicate a clear difference in the gross utility of the two paradigms.

Effects of Training

A strong effect of the order of presentation was seen in all performance variables, indicating increased performance as a function of participation. However, the degree of learning was dependent upon the order of stimulus paradigm presentation. Specifically, training effects were seen to generalize from the pulse train frequency modulated training to the amplitude-based training, but less improvement was seen when individuals moved from amplitude modulated to pulse train frequency modulated feedback. This is likely a reflection of the overall superiority of the amplitude modulated stimulation paradigm. Regardless of the stimulation method, some training is required to increase performance. However, even with training, degraded feedback such as the pulse train frequency modulated stimulation does not enable increased learning, at least for the limited training times explored. Based on the training periods in this study (2 – 3.5 hours), amplitude modulated feedback may offer increased potential over pulse train frequency modulated feedback as a method of continuous real-time augmentative feedback for users of prosthetic hands.

Although training offers the potential to increase the viability of vibratory feedback, long-term sensory substitution using this modality also has the potential for habituation to the stimulus. Adaptation of sensory afferents to vibrotactile stimuli occurs during continuous vibrotactile stimulation with time constants ranging from 10 – 40 s, with recovery time constants ranging from 10 – 30 s and can be both centrally-mediated and a result of sensory peripheral adaptation [26]. The results of our work do not show evidence of desensitization, with increases in box displacement and velocity occurring throughout experimentation with amplitude modulated feedback. In fact, we have shown that individuals continue to learn to use amplitude modulated vibratory feedback in experiments over 4 sequential days [27]. However, additional future work to study the long-term habituation to vibrotactile stimulation for object manipulation in individuals with a history of limb amputation to integrate vibrotactile stimulation is necessary.

Summary

Despite the pragmatic benefits associated with pulse train frequency modulated vibrotactile stimulation, comparison of the approach with amplitude modulation using a robotic interface with which both visual and direct haptic feedback can be experimentally controlled indicates that amplitude modulation provides superior feedback for object manipulation. Future work will determine the effects of longer-term training in prosthetic hand users and design practical methods for implementation.

Biographies

Cara E. Stepp (M’10) received the S.B. in engineering science from Smith College, Northampton in 2004, the S.M. in electrical engineering and computer science from the Massachusetts Institute of Technology, Cambridge in 2008, and the Ph.D. in biomedical engineering from the Harvard-MIT Division of Health Sciences & Technology, Cambridge in 2009.

She is currently a Postdoctoral Research Associate at the University of Washington in the Departments of Computer Science & Engineering and Rehabilitation Medicine, Seattle, WA USA.

Yoky Matsuoka (M’03) received the B.S. degree from the University of California, Berkeley, in 1993, and the M.S. and Ph.D. degrees from Massachusetts Institute of Technology, Cambridge, in 1995 and 1998, respectively, all in electrical engineering and computer science. She is currently an Associate Professor of computer science and engineering with the University of Washington, Seattle, WA USA.

Dr. Matsuoka received the Presidential Early Career Award for Scientists and Engineers in 2004, the Anna Loomis McCandless Chair in 2004, the IEEE Robotics and Automation Society Early Academic Career Award in 2005, and the MacArthur Fellowship in 2007.