Evaluation on Human Perception of Various Vibrotactile Encoding Methods Through a High Density Haptic Feedback Interface

Abstract

High density (HD) haptic interfaces have become increasingly common for entertainment thanks to advancements in virtual reality technology, however their flexibility may make them a useful sensory substitution interface for motor rehabilitation. Yet little research has explored how users interpret different haptic feedback encoding methods. Therefore, this study’s objective was to evaluate the effectiveness of various encoding methods for conveying information based on existing sensory substitution strategies, one being a line motion tracking task and the other a direction tracking task. The first encoding method was Perceived Position Encoding (PPE), where information was encoded into the perceived position of stimulation. The second was Perceived Intensity Encoding (PIE), encoded information into the perceived amplitude of the stimuli. Twenty-one participants performed tracking tasks using both the PIE and PPE methods. The results showed similar performance in line motion tracking between the PIE and PPE methods, although the extra motors used in the PPE method appear to introduce uncertainty in users. Nevertheless, users were significantly more accurate with direction tracking when using PPE. These findings highlight the need for task-specific encoding methods, and showcase the versatility of the HD haptic vest as a tool for augmented feedback in motor rehabilitation.

I. Introduction

Augmented sensory feedback has been widely used in rehabilitation to strengthen sensorimotor integration for improved motor performance in individuals with neuromotor deficits [1]. One common feedback paradigm is to provide spatial information related to body motion or interactions with the environments in the context of a given task. For example, feedback can be given on the position of the Center of Pressure (CoP) to train balance for patients with sensorimotor impairments. Feedback of endpoint movement during therapy has been used to train smooth and coordinated limb motions [2]. The feedback systems in this paradigm depends on an effective sensory interface to convey the movement state of body to humans in real-time.

There are two approaches for delivering sensory feedback: sensory restoration [3] and sensory substitution [4]. Sensory restoration is a method where stimulation is applied to sensory neural pathways to restore somatotopic-matched natural sensations. For example, when individuals with limb loss interact with the external environment through their prosthesis, haptic sensation to the missing limb can be restored by stimulating peripheral nerves [5] [6] or by stimulation of dorsal root ganglia via epidural spinal cord stimulator [7]. However, these interfaces are invasive and require surgeries, limiting their accessibility. Non-invasive high-density surface electrodes have also been used to stimulate nerve bundles that come close to the surface of the skin to restore natural haptic sensation [8]. However, due to the relative motion of electrodes on skin to the peripheral nerve, this approach has not been tested through functional tasks.

A different approach is sensory substitution, which delivers sensory feedback through other sensory modalities. Researchers have used various sensory modalities, such as visual displays [9], auditory feedback [10], and haptic feedback [11]. Visual and auditory feedback have been studied in lab and clinical settings, but their practicality in everyday environments may be limited. These limitations arise as visual and auditory attention are both required for numerous daily tasks [12] [13] and feedback through these mediums can be a pervasive distraction both for the users and those around them. Haptic feedback, however, is a promising method of using sensory substitution in real world situations as the sensation can be delivered through wearable devices that pose minimal distraction in daily tasks outside a clinical setting.

Haptic sensation can be created with a variety of different actuators such as balloons to create pressure [14], electrodes to provide electrotactile sensation [15], actuators to stretch the skin [16], or vibrotactile motors to create vibrations [17]. These vibrotactile-based interfaces are commonly used in rehabilitation [18] as they are designed to be compact, wearable, and lightweight [19]. Often, 2–8 motors are attached to the skin to deliver haptic or movement related feedback through an encoded stimulation pattern achieved by modulating vibration magnitude and frequency [20] [21]. The regions for motor attachment include the arm [22], waist [23], forehead [24], or the residual limb for amputees [25] as sensory information from such areas are generally not critical for other daily tasks. In general, sensory substitution is less intuitive than sensory restoration approaches because it requires additional mental processes from users to interpret the feedback as their motor state. However, this method is non-invasive and has shown the potential to reinforce the sensory motor integration as long as the feedback interface, referring to the hardware, and encoding method, referring to the software, are setup to make the haptic feedback intuitive to understand [18].

There have been many designs of vibrotactile-based sensory substitution systems with different motor configuration and encoding methods. The feedback interface design and encoding method are dependent on both the biomechanical measure used for the feedback and the task required for rehabilitation. Some common examples include providing feedback on the motion of anteroposterior (AP) center of pressure (CoP) beneath the prosthetic foot of an amputee [26], where the patient needs to associate the haptic feedback with the line motion of the CoP. A different example is giving feedback to patients with multiple sclerosis on the direction of their trunk sway during standing [27] to give a better sense of how they are balanced.

For providing line motion feedback on the AP CoP, one typical design uses 2 vibrotactile motors, often with one on the front and one on the back of the participant. The vibration amplitude of the front motor increases the further forward the CoP moves from the mid-point; and similarly the magnitude of the back motor increases the further back the CoP moves [25] [28]. The user can interpret the CoP location from the perceived amplitude of the activated motor. This method is referred to as Perceived Intensity Encoding (PIE), as the user must sense the amplitude of the activated motors. Another reported approach is done with 3 motors placed in a row. In this setup the activation of 3 motors will correspond to the position of the CoP in the toe area, middle of the foot, and heel, respectively [29] [30]. In this method the user interprets the CoP through only the perceived position of the stimulus, so we will refer to this method as Perceived Position Encoding (PPE).

When the feedback signal needs to be conveyed in 2 dimensions, whether it be CoP in the AP and mediolateral (ML) directions [31] or direction of trunk sway as previously mentioned [27], the feedback strategy changes from conveying motion along a line to conveying directional information. This is often done with the PPE method where a ring of motors is placed around the head [32] or waist [31] of the participant and the activated motor corresponds to the direction of trunk sway or CoP drift. The PIE method has also been used in this way to convey directional information. One study used a wrist mounted guidance system, where 4 motors were placed equidistant around the wrist. To communicate direction, the two nearest motors were activated with their amplitudes being proportional to the proximity of the angular direction of a target the user needed to reach for [33].

The majority of existing studies that use vibrotactile feedback need to design a feedback interface specifically tailored for the experimental task being studied. In addition, the limited number of vibrotactile motors used in current literature acts as a restriction on how sensory feedback can be encoded for easy comprehension. These factors limit the functional use of existing vibrotactile sensory feedback systems in phyisical rehabilitation. For example, ideally in balance training, it is desirable for a sensory feedback system to not only render CoP movement during walking exercises but also display trunk sway direction in standing postural training. Such a requirement needs a sensory interface that is flexible to program. High-density (HD) haptic interfaces can be a promising solution that have become increasingly common thanks to recent advancements in virtual reality. While the traditional haptic feedback strategies discussed thus far tend to use anywhere from 2–8 motors to deliver feedback, some of these commercial devices can contain up to 80 individually programmable vibrotactile motors [34]. This greatly increases the flexibility in how encoding methods can be designed to deliver intuitive feedback for more than one task context. In addition, such an haptic jacket is comfortable and easy to wear. However, to our knowledge, this technology has yet to be applied for rehabilitation purposes. In addition, since previous research focuses on one type of design, there is very little research to help understand how different encoding methods influences the communication of knowledge of motor performance to the user.

Hence the goals of this study are (1) to design different feedback encoding methods utilizing different encoding methods and quantities of vibrotactile motors on a HD haptic interface and (2) evaluate whether humans can interpret both continuous line motion and directional information conveyed through a HD haptic interface. To quantify the participants’ ability to interpret this spatial information, participants were asked to trace the perceived path or direction of a target on a touch screen tablet as conveyed through the haptic interface with both PIE and PPE methods. The outcome of this study may inform the future designs and use of HD haptics vest in physical rehabilitation to improve the motor function in different patient populations with sensorimotor deficits.

II. Materials & Methods

To evaluate the different sensory substitution methods, two different manual tracking tasks were devised to test for both the line motion and directional information cases. For these tasks the participants would wear a haptic feedback vest and sit in front of a touch screen tablet displaying the grid seen in Figure 1B and try to trace the path of a moving target based on the haptic sensation delivered through the vest. In the line motion tracking task the participant would track the target in real time as it continuously moved up and down on the y-axis. For the directional tracking task the target always started in the center and then suddenly moved in a random direction to the edge of the screen, and participants would need to determine the direction it moved by tracing the path on the touchscreen.

Fig. 1.

Block diagram depicting experimental setup. A.) The participant responded to the haptic stimuli by tracing on the touchscreen tablet with a digital pen. B.) The grid displayed to the user through the Unity App, which recorded the user’s input on where they drew and acted as a visual representation of the array on the back of the vest. C.) The experiment was run through MATLAB, which sent trial info to the Unity app, which controlled the haptic interface, all of which ran on the Lenovo tablet. D.) The haptic interface used was the bHaptics Tactsuit x40, haptic sensation was delivered to the array on the back of the vest.

For each task two different encoding methods were used. One based on the PIE method, where the target location or direction was communicated through the perceived amplitudes of the activated motors. The other was based on the PPE method, where the perceived position of the stimuli was used to communicate the target location or direction.

A. Materials

The bHaptics Tactsuit x40 (bHaptics Inc., Daejeon, South Korea) was used as our feedback interface. It was selected as it is a commercially available haptic interface with a 4×5 array of individually programmable vibrotactile motors on both the front and back of the vest. Each motor can produce frequencies ranging from 0–120 Hz and can be individually programmed to vibrate from 0% to 100% of its maximum amplitude, with a maximum output of 2.6 G. We used a touchscreen Lenovo Thinkpad X1 Yoga G7 (Lenovo, Hong Kong, China) to run the experiment and control the vest using Bluetooth 5.0. The laptop had a 14” touch display which users could interact with by drawing on the screen with a Lenovo digital 2 pen (Lenovo,Hong Kong, China).

To control the haptic vest, we used bHaptic’s open source C# library available through the Unity asset store to create an app in Unity that can read predefined motor activation patterns from an excel sheet to activate the motors on the vest. The Unity app also acted as the interface participants used during the experiments. The app recorded the participant’s cursor position at a sampling rate of 100Hz. The diagram of the full experimental setup can be seen in Figure 1.

B. Participants

21 participants were recruited (12 Male, 9 Female, ages (19 – 54). All participants provided written consent through the NC State University Institutional Review Board (IRB-20647). Each participant completed 1 experimental session with a duration of approximately 1.5 hours.

C. Line Motion Tracking Task

Participants completed two sets of line motion tracking tasks. Each task consisted of 31 trials. 31 sinusoidal paths were pre-generated for each task. The first 11 trials were considered practice and used paths 1–11 in a random order. During the task, the Tactsuit was programmed to vibrate and simulate a point moving in the y-axis along the third column of motors on the haptic vest. Each trial had a duration of 10 seconds. The motion followed a sinusoidal path pre-generated using MATLAB. These paths were formed by summing together 5 non-harmonic sinusoids that had been randomly phase shifted. The frequencies of these sinusoids were .06 Hz, .11 Hz, .13Hz, .25Hz, and .33 Hz. This method of generating random paths has been used in similar studies with manual tracking tasks [35]. Each of these pathways were normalized to have a range of ±2 in order to span the full range of the graph, and each trial had the target begin in the center of the graph. Subjects were instructed to use the stylus and tablet to follow the simulated point as it moved up and down.

For the first trial, participants were provided with both haptic sensation and a visual cue with the purpose of familiarizing them with the sensation of the haptic stimulation as it corresponds to the target’s pathway on the Unity interface. Trials 2–11 were training trials, where participants were provided with haptic sensation but no visual cue. At the end of each trial, participants were shown a graphic of their drawn path in comparison with the actual point path in order to visualize any errors that occurred. The following 20 trials were the evaluation trials and consisted of the participant drawing on the Unity interface with no visual cue and no visual feedback following the trial, instead relying on the haptic interface to locate the target.

1). PIE method:

We designed the PIE method according to Weber-Fechner’s law, which states that in order for an increase in stimulus intensity to be perceivable the increase must be proportional to the previous stimulus [36]. To satisfy this we defined an exponential relationship between the stimulus amplitude and the target’s position along the y-axis which was derived from Fechner’s measurement formula [37]. In this way, the perceived amplitude of the stimuli at either the top or bottom of the vest would correspond to the target’s location.

A deadband region, where no haptic sensation was given, was introduced for when the target was near the center of the vest at the coordinates ± 0.25 along the y-axis as seen in Figure 2. The deadband serves as a perceptual anchor, a reference stimulus by which other stimuli can be compared to. This makes it easier for participants to perceive if the target is in the top, bottom, or middle of the vest [38]. Equation 1 describes the mathematical formula used for this encoding method.

Fig. 2.

Haptic encoding methods used in line motion tracking task for PIE (left) and PPE (right) methods. M-2 through M2 represent the 5 motors used in the line motion tracking task positioned along the y-axis.

$$\begin{gathered} V_2=\left\{\begin{array}{cc}{S}_0{e}^{by},& y>0.25\\ 0,& y<0.25\end{array}\right. \\ {V}_{-2}=\left\{\begin{array}{cc}{S}_0{e}^{-by},& y<-0.25\\ 0,& y>-0.25\end{array}\right. \end{gathered}$$ (1)

Where $V_2$ and $V_{-2}$ are the amplitudes of the top and bottom motors respectively, $S_0$ is a minimum perceivable stimulus, which we set to 8% max amplitude of the vest’s motors as it was easily perceivable by all participants, $y$ is the position of the target along the $y$ axis and $b$ is the growth rate, set to make the curve scale from 8–100 as $y$ changes from 0–2. Where $\text{b}={(\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.)}^{\ast }\log (100/\text{So})$.

2). PPE method:

The PPE method was designed to make use of the phantom actuator (PA) illusion so that the target’s location can be communicated through the perceived location of the stimuli. The PA illusion uses 2 principles to create the illusion of a continuously moving stimuli between a discrete set of motors. The first principle is haptic apparent motion which is an illusion of a haptic stimuli “moving” across the skin when in reality it’s a discrete set of adjacent motors turning on and off [39]. The second illusion is phantom tactile sensation which is created when multiple adjacent motors are activated at once but the user only perceives a single haptic stimuli centered at a point in-between the adjacent motors [40].

To generate the PA illusion with our vest we used the equation described by Israr & Poupyrev [41] and added a deadband region in the center of the vest from ± 0.25 on the y-axis to act as a perceptual anchor and mirror the deadband used in the PIE method. The formula used to find the proper amplitudes for the PA method follows a power model and can be seen in equation 2. This shape was chosen over a linear model as other studies have shown that power and logarithmic models better convey the illusion of motion along a line than linear models [42]. An illustration of the activation patterns used in this experiment can be seen in Figure 2.

$$\begin{gathered} V_N=\left\{\begin{array}{cc}0,& -0.25<y<0.25\\ \sqrt{1-\left|y-y_N\right|}\times P,& else\end{array}\right. \\ {V}_{N+1}=\left\{\begin{array}{cc}0,& -0.25<y<0.25\\ \sqrt{\left|y-y_N\right|}\times P,& else\end{array}\right. \end{gathered}$$ (2)

Where $V_N$ and $V_{N+1}$ are the amplitudes of the two actuators adjacent to the phantom actuator, $y$ is the target position, $y_N$ is the position of actuator $N$ where $N$ ranges from −2 to 2, and $P$ is a constant scaling factor to represent the maximum amplitude, in our case set to 100.

D. Direction Tracking Task

Participants also completed two sets of the direction tracking task. These trials only used the top four rows of motors on the back of the haptic vest, creating a 4×4 array to keep things symmetrical between the x and y axes. Participants performed 8 training trials and 12 evaluation trials. Each trial lasted approximately 4 seconds. At the beginning of each trial the target began at the center of the 4×4 array. After a random period of time between 1 and 2 seconds the target moved in a random direction to the outer edge of motors on the vest, this random delay followed a uniform distribution across trials. Participants were instructed to draw from the center of the 4×4 array to the outer edge of the array on the Unity interface, and the participant’s final position on the screen was compared to the center of the array to calculate angle from the center. The trial ended once the participant’s drawn line crossed the border of the outer edge.

For the 8 training trials, participants would perform the task as described and after each trial a visual cue would appear on the screen showing the actual target alongside the path drawn by the participant so that they can see how close they came to the target. When the visual cue was displayed, the haptic sensation indicating the target’s position was also played to reinforce this spatial relation between target location and haptic stimulus to the participant. The 12 evaluation trials were completed with no visual cue and no visual feedback following the trial. Angle order was randomized between tasks and subjects.

1). PIE Method:

For the design of the directional PIE method we used the interpolation method that has been used in existing works on wrist based haptic guidance systems [33]. This design uses 4 motors, 1 for each cardinal direction. As we used a 4×4 array of motors for this task, instead of 4 individual motors we used 4 sets of adjacent motors as seen in Figure 3. The top and bottom motors were part of the vertical sets, with set $V+$ on top and $V-$ on bottom, and the right and left motors were part of the horizontal sets referred to as $H+$ and $H-$ respectively. Using this encoding method, the direction of the target is communicated to the participant through the perceived amplitude of the two sets of motors adjacent to the target.

Fig. 3.

Illustrated example of haptic encoding methods used for the direction tracking task, demonstrating how motors would activate when the target is in the center (left) and once it moves to the outer edge (right).

To indicate the start of the trial when the target was in the center of the array, all 4 sets of motors activated at equal intensity set to 25% of their max amplitude. Once the target moved out to the outer edge of the array, the angular information was encoded into the amplitude of the 2 adjacent motor sets based on the parameters in equation 3.

$$\begin{gathered} V_{sgn(\text{sin}(\theta ))}=\alpha \times \ln (|\arcsin (\sin (\theta ))|+1) \\ {V}_{-sgn(\sin (\theta ))}=0 \\ {H}_{sgn(\cos (\theta ))}=\alpha \times \ln \left(\left|\arcsin \left(\left.\sin \left(\theta -\frac{\pi }{2}\right)\right|+1\right)\right.\right. \\ {H}_{-sgn(\cos (\theta ))}=0 \end{gathered}$$ (3)

Where $V$ and $H$ are the amplitudes of the vertical and horizontally positioned motors respectively, $sgn()$ is the sign (positive or negative) of the sine or cosine of the target angle $\theta$ (radians) and $-sgn()$ is the reverse of that sign. $\alpha$ is a constant defined as $P/\ln \left(\frac{\pi }{2}+1\right)$.

2). PPE Method:

For the directional PPE method we expand the PA formula previously described into 2 dimensions, as seen in equation 4. In this way, the target direction is communicated to the participant through the perceived position of the stimuli.

$$\begin{gathered} V_{M,N}=\sqrt{1-|x-M|}\times P \\ {V}_{M+1,N}=\sqrt{|x-M|}\times P \\ {V}_{M,N+1}=\sqrt{1-|y-N|}\times V_{N,M} \\ {V}_{M+1,N+1}=\sqrt{|y-N|}\times V_{N,M+1} \end{gathered}$$ (4)

Where $x$ and $y$ represent the coordinates of the target, and $V$ is the amplitude of each motor adjacent to that target. $M$ represents the motors along the x-axis and $N$ represents the motors along the y-axis, with $V_{M,N}$ being the amplitude of the motor at coordinates $\left(M,N\right)$.

This can also be seen illustrated in Figure 3, where when the target is in the center the amplitudes of all 4 adjacent motors are equal. However, as the target moves to the edge of array only the two adjacent motors along that edge are activated with the amplitudes of the other two motors going to zero.

E. Data Analysis

For the line motion tracking task, the user input for each trial was put through a 2nd order butterworth filter with cutoff frequency 5 Hz to smooth out the signal. Next, the first second of each trial was cut off as to not include any effects from participants’ initial transient responses to the start of the trial. Since there was a consistent time delay between the haptic stimulus and the user’s response, we first calculated the cross correlation between the target trajectory and the user’s input in order to find the lag between the two. Next, we time shifted the user’s input by the calculated lag so that it lined up with the target trajectory. The maximum correlation coefficient is also a valuable statistic as it can give insight on whether or not participants could accurately trace the general path of the moving target.

As was done in our previous work [43], we report metrics in terms of (units) where 1 (unit) is the distance from one row to the next along the y axis of the interface the participants traced on during the experiment, equivalent to approximately 36 mm or 230 pixels.

The main performance metrics used for the line motion tracking task were the average position and velocity root mean square error (RMSE). These values were calculated for each trial, and then averaged across each participant so that there were 21 data points for both methods to compare.

Another metric examined for the line motion tracking task was the approximate sample entropy, which was calculated for each trial using the algorithm described by Dr. Lake [44]. In their work they describe approximate sample entropy as the negative natural logarithm of the conditional probability that given a dataset made up of $N$ samples repeats itself within a tolerance $r$ for $m$ points, said dataset will repeat itself again for $m+1$ points. For our implementation of this algorithm, $N$ was the total number of samples collected for each line motion trial, 1000 samples. This should be sufficient data for the algorithm to work as intended as it has been shown that N > 75 can be a sufficient sample size [45]. $r$ was 20% of the standard deviation for each trial, and m was set to 2, which are standard values to be used for this equation [46].

The final metric examined for the line motion tracking task was the standard deviation for each trial, this was calculated by taking the average standard deviation of the user’s input for each trial in order to evaluate the inter-subject variability and see if the different participants responded consistently to different encoding methods.

For the directional tracking task the main evaluation metric was the directional error. For each trial we recorded the location of the endpoint of the participant’s input to calculate the angular error between the user’s input and the target trajectory. The directional error was the number of degrees between the vectors made from the center of the array to the target’s endpoint and the center of the array to user input’s endpoint.

Except for the standard deviation during line motion trials, all of these metrics were averaged across every trial for each participant. We used the Shapiro-Wilk (SW) test to check that the data collected was normally distributed. For each parameter the SW test failed to reject the null hypothesis that the data was normal with unspecified mean and variance, with p < 0.05. To determine if any significant differences were present between the PIE and PPE methods, two-tailed t-tests were used to check for significant differences for each parameter between the PPE and PIE conditions. Significance was defined as $\alpha <0.05$.

III. Results

A. Line Motion Tracking Performance

For the line motion tracking task it was found that for both the PPE and PIE methods, the average correlation coefficient between the target trajectories and user input was greater than 0.8, signifying that for both methods the participants were able to perceive and follow the path of the moving target. Using the lag calculated from the peak correlation coefficient we were also able to calculate the average input delay for both encoding methods. There was no significant difference where p = 0.26 with the average input delay for the PIE method being 840 ± 133 milliseconds (ms) and the average delay for the PPE method being 792 ± 133 ms.

Figure 4 shows the performance metrics of the line motion tracking task. For position tracking accuracy the PIE method had an average RMSE of 0.60 ± 0.11 (units) and the PPE an average of 0.64 ± 0.17 (units). These results show that there was no significant difference in performance between the two methods at p = 0.42, For the comparison of how well participants could match the velocity of the moving target, the average RMSE with the PIE method was 1.30 ± 0.26 (units/s) and for the PPE method it was 1.33 ± 0.29 (units/s). There was no significant difference here either, with p = 0.70. There were also no observed learning effects for the velocity or position RMSE for either the PIE or PPE methods over the course of the 20 trials participants performed.

Fig. 4.

Box and whisker plots showing key statistics from line motion tracking task. A.) Average RMSE of position of each participant. B.) Average RMSE of velocity of each participant. C.) Average values of approximate sample entropy. D.) Mean standard deviation of each trial. The central red mark indicates the median, the edges of the box represent the 25th and 75th percentiles, and the whiskers extend to the minimum and maximum data points not including outliers which are denoted by a red ‘+’ mark.

In order to better understand the differences in how participants perceived these two encoding methods, entropy was used as a way of quantifying the uncertainty present in each trial. As shown in Figure 4C, we found that there was a statistically significant decrease in entropy with the PIE method at p = 0.049, with an average entropy of 0.042 ± 0.0043 for the PIE method and 0.046 ± 0.0064 for the PPE method.

As the increased entropy present with the PPE method indicates increased uncertainty, we also investigated the inter-subject variability by measuring the standard deviations for each trial, averaged across each participant. We expect that with increased uncertainty that participants would behave differently from one another, increasing the variability in the PPE method when compared to the PIE method. These results can be seen in Figure 4D where it was found that the PPE method had significantly higher standard deviations (p = 9.72e-08) in each trial than the PIE method where the mean std for PPE was 0.60 ± 0.066 and for PIE it was 0.47 ±0.055.

B. Direction Tracking Performance

For the direction tracking task we found that PPE performed significantly better than PIE where the average error for PPE was 13.06 ± 3.54 degrees and for PIE it was 22.43 ± 6.67 degrees with p = 2.43e-05. These results can be seen in Figure 5. These results do support that the PPE method is a more effective way of communicating directional information than the PIE method. Similar to the line motion tracking task, there was no learning effect observed over the 16 trials performed for the direction tracking task.

Fig. 5.

Average directional in degrees for the directional tracking task for both the PIE and PPE methods represented through a traditional box and whisker plot.

Furthermore, we investigated the average directional error for each target angle that was tested. The results of this can be seen in table 1, where for each direction the average error for the PPE method was less than the error for the PIE method. This data was visualized in Figure 6, where the average error for each direction is depicted as a red region around the unit circle.

TABLE I.

AVERAGE ERRORS BY DIRECTION

Target Angle	Avg PPE Error	Avg PIE Error
0°	21.44°	33.40°
30°	9.85°	26.89°
60°	16.05°	23.44°
90°	9.47°	13.74°
120°	10.24°	17.43°
150°	11.39°	13.63°
180°	17.75°	24.78°
210°	14.80°	28.08°
240°	17.79°	28.83°
270°	6.93°	13.07°
300°	12.69°	25.92°
330°	8.54°	19.28°

Fig. 6.

Average error by direction for the direction tracking task for PIE and PPE methods, the darker red regions denote overlapping regions where misclassification was more likely.

IV. Discussion

In this study, we showed the potential of using a haptic vest with an HD vibrotactile array for augmented sensory feedback of body movement. First, due to the HD layout of the vibrotactile motors, the device is very flexible to program for various sensory feedback paradigms for different task contexts. By selecting different numbers of motors, using different motor layouts, and modulating the motors’ amplitudes, we implemented both PPE and PIE methods for line motion feedback and movement direction feedback. In addition, the vibrotactile sensation rendered by the 4×5 array on the back of the participants was able to provide sufficient spatial resolution to elicit a sense of motion through the PA illusion for the line motion task even though it is known that resolution of haptic perception on the trunk is much lower than the resolution on hand or limbs [47]. That means the space between two motors in the device are large enough to enable two-point discrimination on human torso. Finally, the HD haptic vest is easy to don/doff and is comfortable to wear. All these benefits make the HD haptic vest, designed mainly for gaming industries and virtual reality, a potential augmented feedback system for motor rehabilitation.

A. Encoding Methods

The comparative results in this study showed how designing a vibrotactile encoding method that is easy for users to comprehend depends the intended task. These results offer some key takeaways that can be used to guide the design of vibrotactile encoding methods for different tasks.

The first key takeaway from these results is to not over complicate feedback encoding methods relative to the information being conveyed. We found that when the feedback can be conveyed through simple one-dimensional motion along a line, using 2 vibrotactile motors with the PIE approach was adequate for participants to understand. Introducing additional motors and implementing the PPE method to convey the same information did not improve participant accuracy, and instead the additional motors increased the level of uncertainty participants felt as seen in the increased entropy apparent in Figure 4C.

This finding is consistent with other literature that has investigated human perception of different haptic feedback encoding methods, where it’s been found that use of additional vibrotactile motors to convey redundant information saw no improvement of user accuracy, and in some cases even lowered performance [48]. This may be of particular interest in the realm of sensory substitution for lower limb amputees, where providing CoP feedback along a line in the AP direction is a common strategy [25] [28] [30].

The second key takeaway is that when conveying directional information, using a PPE method will be much more intuitive for users than the PIE method. As seen in Figure 6, our results showed that participants were much more accurate in discerning direction with the PPE approach which used more motors along the perimeter of the vest opposed to the PIE method which only used motors in the cardinal directions.

When conveying directional information through haptic feedback it is vital that the information be accurate and easy to understand, as other studies have shown that inaccurate directional haptic cueing can significantly increase the reaction time of those using haptic interfaces [49]. This would be an important consideration when designing sensory substitution strategies for rehabilitation, such as providing feedback on direction of trunk sway for patients with Parkinson’s disease [21], where a quick reaction time may be vital for preventing falls once posture becomes unstable.

The requirement of task-specific design further highlights the potential benefit of the HD haptic jacket in rehabilitation since it is flexible to program. Researchers can quickly prototype different haptic feedback paradigms and compare their effects on human perceptions to determine the optimal haptic encoding method for specific movement tasks. In addition, researchers can use one haptic device to program many feedback paradigms corresponding to various biomechanical feedback measurements during different rehabilitation tasks.

B. Information Processing

The results observed in the simple line movement task were surprisingly counter-intuitive. The PPE method used more motors along the line in order to directly map the spatial location and was supposed to be more straightforward to comprehend; however, it did not bring better perception accuracy and led to more uncertainty than the PIE method with 2 motors. To understand why, it is first important to know the underlying mechanisms behind how humans process information for movement control.

Whiting’s Model of information processing [50] may help to narrow down where this uncertainty is coming from. The model describes information processing as a process that occurs in three steps. The first step occurs through perceptual mechanisms, where sensory input is collected and filtered into relevant information. In the next step the translatory mechanisms interpret the filtered information. In the final step, the effector mechanisms allow humans to respond and react to the information. Based on our results, much of the uncertainty experienced with the PPE method during the line motion tracking task likely came from misclassification of which motors were activated at any given point in time (i.e., uncertainty in translatory mechanisms).

An example of this is illustrated in Figure 7. It shows jerky movement and sudden corrections in the participant’s reported trajectory, particularly in the period between 4 and 7 seconds, as they second guessed the true location of the target. This jerky motion was not as prevalent with the PIE methods rendered by only two motors because the absolute position can be calibrated with high confidence based on the activated motor. For example, if the top motor is activated, the user was certain that the location was in the top half of the line. Hence, when the feedback information is simple the encoding method should likewise be kept simple and use the minimum number of motors required to convey the information in order to reduce uncertainty in the user.

Fig. 7.

Plot showing example trial performance from participant 18 using both PIE and PPE methods.

C. Future Work

The research results of this study will be used to make an informed design of effective motion feedback paradigms using HD haptic vests in motor rehabilitation. In our future work, we are interested in applying the presented HD haptic interface to enable closed-loop control of a neural-controlled robotic prosthetic ankle to enhance motor functions of individuals with transtibial amputations. Our research group previously developed a direct myoelectric controlled prosthetic ankle, which restored normative postural control strategies in individuals with transtibial amputation after prosthesis use and balance training [51] [52] [53]. However, due to the lack of sensory feedback from the prosthetic foot, such neural-controlled devices have seen limited progress in the realm of gait improvement [54].

In addition, haptic sensation may accelerate motor task-based training progress of amputees in using neurally controlled prosthesis ankle by enhancing the sensorimotor integration in gait and balance [55]. Although haptic sensory feedback has also been applied to the amputees wearing passive devices [56], the study of closed-loop neuroprostheses in lower limb devices has never been demonstrated. Based on this study’s results, our future work will focus on using the PIE approach to provide AP CoP motion feedback under the foot of a neuroprosthesis to allow for closed-loop control. Seperately, we will explore using the PPE approach to train amputee control of CoM movement in the transverse plane for improved postural control.

Finally, Future work will expand on this study to evaluate human perception of other haptic tracking tasks, such as expanding the line motion tracking task to a dynamic 2D motion tracking task in which the target can randomly move across both the x and y axes of the grid.

D. Limitations

There are several limitations present in this study. First, the vibrotactile sensation shows variations among and within humans. For example, the sensitivity of vibrotactile sensation was not homogeneously distributed on the back. We found that bony areas were more sensitive to the vibration than the areas covered by muscles or fat. In addition, the tightness of the jacket worn on the body also caused different haptic sensation. Other studies that have explored different vibrotactile encoding methods have found that the perceived sensation can vary depending on which region of the body is being stimulated [57]. As such, future work can extend this study to see if these results hold true when the vibrotactile sensation is applied to other regions of the body such as the arms, legs, or hands.

Additionally, Figure 6 and Table I reveal a physical constraint with our haptic interface. As there is a greater average error for the directions at 0 and 180 degrees than at 90 and 270 degrees. This is likely due to the spacing between the motors on the vest which are positioned 5.5 cm apart horizontally but only 4 cm apart vertically, increasing the likelihood that vertical misclassifications would occur. How to design or program the vest to minimize the inter- and intra-user variations can be a future study.

Another limitation is that we only compared PIE and PPE paradigm based on commonly used feedback tasks and strategies used in literature. More haptic design paradigm should be implemented and compared. Furthermore, we only studied how well humans perceive movement information. Future study should integrate this sensory feedback device in the closed-loop scenario to understand the impact of haptic technology in motor performance and physical rehabilitation.

V. Conclusion

In this exploratory study, we evaluated the feasibility of using a HD haptic vest as a tool for delivering sensory feedback of movement related information. We examined how users interpreted different haptic encoding methods during different tests designed to imitate sensory substitution paradigms often used in rehabilitation settings. The first task looked to see how well participants could track a continuously moving target along a line. While users could sense the motion and track the target using both the PIE and PPE methods, we observed that the PPE method introduced uncertainty as users had a difficult time translating the perceived location of the stimulus to a physical location on the touch screen interface. The second task looked to examine the most effective way to convey directional information through haptic sensation. For this task we found the HD vibrotactile array offered by the haptic vest was greatly beneficial in conveying direction by using the PPE method.

Overall these results showed that the HD haptic vest can be used as an effective tool for delivering sensory feedback. In addition, it was comfortable to wear and flexible to program. An HD haptic vest can act as a single wearable sensory feedback interface to be used in a variety of training tasks or activities in daily living, making it promising tool in physical rehabilitation or assistive device use.