Electro-tactile stimulation (ETS) enhances cochlear-implant Mandarin tone recognition

Abstract

Objective

Electro-acoustic stimulation (EAS) is an effective method to enhance cochlear-implant performance in individuals who have residual low-frequency acoustic hearing. To help the majority of cochlear implant users who do not have any functional residual acoustic hearing, electro-tactile stimulation (ETS) may be used because tactile sensation has a frequency range and perceptual capabilities similar to that produced by acoustic stimulation in the EAS users.

Methods

Following up the first ETS study showing enhanced English sentence recognition in noise,¹ the present study evaluated the effect of ETS on Mandarin tone recognition in noise in two groups of adult Mandarin-speaking individuals. The first group included 11 normal-hearing individuals who listened to a 4-channel, noise-vocoded, cochlear-implant simulation. The second group included 1 unilateral cochlear-implant user and 2 bilateral users with each of their devices being tested independently. Both groups participated in a 4-alternative, forced-choice task, in which they had to identify a tone that was presented in noise at a 0-dB signal-to-noise ratio via electric stimulation (actual or simulated cochlear implants), tactile stimulation or the combined ETS.

Results

While electric or tactile stimulation alone produced similar tone recognition (∼40% correct), the ETS enhanced the cochlear-implant tone recognition by 17–18 percentage points. The size of the present ETS enhancement effect was similar to that of the previously reported EAS effect on Mandarin tone recognition. Psychophysical analysis on tactile sensation showed an important role of frequency discrimination in the ETS enhancement.

Conclusion

Tactile stimulation can potentially enhance Mandarin tone recognition in cochlear-implant users who do not have usable residual acoustic hearing. To optimize this potential, high fundamental frequencies need to be transposed to a 100–200 Hz range.

Introduction

We showed previously that electro-tactile stimulation (ETS) enhanced English speech recognition in noise by a 2.2 dB improvement in signal-to-noise ratio in actual cochlear implant users.¹ This ETS benefit was similar to the electro-acoustic stimulation (EAS) benefit in actual cochlear implant users who had residual low-frequency acoustic hearing.2, 3, 4, 5 The present study attempted to extend the ETS benefit to Mandarin tone recognition in noise. Our hypothesis was that this extension is likely to work based on acoustic properties of Mandarin tones and perceptual similarities of tactile sensation to low-frequency audition.

First, as a tonal language, Mandarin uses variations in voice pitch to carry lexically different meanings. For example, the same consonant and vowel syllable, /ma/, can mean either “mother” or “horse”, depending on whether the voice pitch of the vowel /a/ has a flat or a falling-rising pattern. Although Mandarin voice pitch has many redundant acoustic cues, the most salient ones are fundamental frequency and its harmonics.⁶ The range of speaking fundamental frequency in Mandarin is between 50 and 600 Hz, which is similar to that in English and also falls within the same frequency region as in a typical EAS user.2, 3, 4, 5, 7

Second, both the ETS and EAS approaches operate in the same low-frequency region (i.e., <500–600 Hz) and, more importantly, have similar perceptual properties. Similar to impaired perceptual capabilities associated with a typical EAS user, tactile stimulation produces intensity discrimination of 1–3 dB,8, 9 gap detection of 10 ms and frequency discrimination of ∼20% baseline.10, 11 Different from traditional tactile aids for deaf people, which attempted to replace the entire auditory modality,12, 13 the goal of the present ETS approach is to argument the cochlear implant stimulation that provides high-frequency acoustic cues sufficient for speech recognition in quiet but inadequate for pitch perception including Mandarin tone recognition.¹⁴

Methods

Subjects

Two groups of adult Mandarin-speaking subjects participated in the present study. The first group was 11 normal-hearing subjects who listened to 4-channel noise-vocoded sounds that simulated cochlear implant stimulation (see the Stimuli section below). As a control when listening to the original, unprocessed Mandarin tones in quiet, these 11 normal-hearing subjects achieved nearly perfect performance of (97 ± 1(SE))%. The second group included 1 unilateral cochlear-implant subject and 2 bilateral subjects with each of their devices being tested independently. The unilateral user was a 55 years old male who had used for 5 years a N24R(CA) Freedom device (Cochlear Ltd., Sydney, Australia). One bilateral user was a 48 years old female who had used for 12 years a N22 device in the left ear and for 2 years a N24R(CA) device in the right ear. The other bilateral user was a 44 years old female who has used for 12 years a Clarion Platinum device in the right ear and for 1 year a Clarion Harmony device in the left ear (Advanced Bionics Corp. Valencia, CA). On average, they scored (68 ± 7)% correct recognition for Mandarin tones in quiet, which was similar to the (50–78)% scores reported by Mandarin-speaking cochlear-implant users who listened to the same test materials in quiet.¹⁵ All subjects signed an informed consent approved by the University of California Irvine Institutional Review Board (IRB) to ensure compliance with federal regulations, state laws, and university policies. All subjects were paid for their participation in the study.

Stimuli

Fig. 1 illustrates delivery of 3 stimulation modes.¹ A personal computer was used to control the stimulus generation, calibration, and delivery through custom Matlab programs and a 24-bit external USB sound card at a 44.1 kHz sampling rate (Creative Labs Inc., Milpitas, CA). Electric stimulation was delivered via an audiometer and speaker (Grason-Stadler Inc. 61, Eden Prairie, MN). The subjects were placed in a soundproof booth at a distance of 1 m away from the speaker. The most comfortable level was presented on an individual basis, ranging from 65 to 75 dB SPL across subjects.

Figure 1.

The experimental setup for electric stimulation, tactile stimulation and electro-tactile stimulation (ETS). Electric stimulation is delivered to a cochlear implant (CI) through an audiometer and speaker. Tactile stimulation is delivered to the index finger through an amplifier and a tactile transducer. Different from electric stimulation which delivers the full-spectrum original waveform, tactile stimulation delivers only fundamental frequency. The ETS is simultaneous delivery of both electric and tactile stimulation.

Tactile stimulation was delivered via a tactile transducer (Tactaid Model VBW32, Audiological Engineering Corp., Somerville, MA). The tactile transducer was powered by an amplifier (Crown Audio, Elkhart, IN), and attached to the index fingertip of the non-dominant hand of the subject. The subjects rested their arms on a desk and were asked to place their hand palm-side up to keep the vibration intensity consistent. A 250-Hz sinusoid was used to calibrate the tactile stimulation, with a maximum up to 2.5 V, or a 0 dB reference. The most comfortable level of tactile stimulation ranged between −20 dB and −10 dB across the subjects. The combined ETS was achieved by simultaneously delivery of the above-described electric stimulation and tactile stimulation.

The test stimuli were 100 Mandarin words, consisting of 25 consonant–vowel combinations with each having 4 lexically-meaningful tonal patterns.¹⁴ To simulate electric stimulation in normal-hearing listeners, the Mandarin tone stimuli were subject to 4-channel, noise-vocoded pre-processing.¹⁵ In actual electric stimulation, the original, unprocessed Mandarin tone stimuli were delivered to the sound processor of the actual cochlear implant users. In tactile stimulation, only the fundamental frequency of the Mandarin tones was extracted and delivered to the tactile transducer.16, 17 A noise was produced by spectrally shaping white noise to have the same long-term spectrum as the average of the 100 Mandarin tones. All tests were performed under the same 0-dB signal-to-noise ratio.

Procedure

A 4-alternative, forced-choice procedure was used to obtain Mandarin tone recognition. A graphic-user-interface presented a stimulus and displayed 4 buttons corresponding to tone 1, 2, 3 and 4, respectively. The subject had to choose one of the 4 buttons based on his or her perceived tonality of the stimulus. Each run contained the complete set of the 100 Mandarin tones, with the presentation order of these 100 tones being randomized. At least 5 runs were used to obtain the score for each stimulation mode, with the presentation order of stimulation mode being also randomized. All subjects received extensive training and were familiar with the procedure when they performed Mandarin tone recognition in quiet.

Results

Fig. 2 shows enhanced performance in Mandarin tone recognition by combined ETS over either tactile stimulation (T) or cochlear implant simulation (CI simu) in normal-hearing listeners. On average, the tactile stimulation produced (37 ± 4(SE))% correct recognition of Mandarin tones, while was not significantly different from (41 ± 3)% correct recognition obtained by the cochlear implant simulation (two-tailed, paired t-test, P = 0.30). However, the ETS produced (58 ± 5)% correct recognition score, which was 21 percentage points better than the tactile stimulation (P < 0.01) and 17 percentage points better than the cochlear implant simulation (P < 0.01).

Figure 2.

Average Mandarin tone recognition in 11 normal-hearing individuals who listened to tactile stimulation (T), cochlear-implant simulation (CI simu) or the combined electro-tactile stimulation (ETS). The error bars represent one standard error of the mean. The asterisk indicates a significant effect of the ETS enhancement.

Fig. 3 shows similarly enhanced performance by ETS in actual cochlear-implant users. On average, the tactile stimulation produced (45 ± 2)% correct recognition of Mandarin tones, while was not significantly different from (40 ± 4)% correct recognition obtained by the cochlear implant users (two-tailed, paired t-test, P = 0.20). Interestingly, the ETS produced the same (58 ± 5)% correct recognition score as in the CI simulation condition; the 58% score was 13 percentage points better than the tactile stimulation (P < 0.05) and 18 percentage points better than the cochlear implant simulation (P < 0.01).

Figure 3.

Average Mandarin tone recognition in 3 actual cochlear-implant users who listened to tactile stimulation (T), cochlear-implant stimulation (CI) or the combined electro-tactile stimulation (ETS). The error bars represent one standard error of the mean. The asterisk indicates a significant effect of the ETS enhancement.

Discussion

Comparison with EAS

One study measured Mandarin tone recognition in noise (+5 dB signal-to-noise ratio) in actual cochlear implant users with residual acoustic hearing.¹⁸ Compared with 31% tone recognition with cochlear implants alone, the EAS produced 45% tone recognition. The 14-percentage-point improvement by EAS in the Li et al¹⁸ study was similar to the 18-point enhancement by ETS observed in the present study.

Tone pattern analysis

Taking all stimulation modes into account, a distinctive pattern of tone recognition emerged. Tone 1 and tone 2 were the most difficult to recognize, with 38% and 39% correct scores, respectively; tone 3 was the easiest to recognize with a 58% correct score; tone 4 produced an intermediate 46% correct score. However, analysis on the ETS enhancement didn't show any tone-specific effects. Compared with the baseline or electric stimulation alone, the additional tactile stimulation improved performance relatively evenly for all 4 tones: 19 percentage points for both tone 1 and tone 2, 18 points for tone 3, and 14 points for tone 4.

Psychophysical mechanisms

Because the present tactile stimulation extracted only fundamental frequency information, the observed ETS enhancement was likely due to improved frequency discrimination with the additional tactile cue. To directly test this hypothesis, we measured both detection thresholds and frequency discrimination of sinusoidal tactile stimulus. The top panel of Fig. 4 shows a previously-reported U-shaped detection threshold curve,⁸ from a maximum of −30 dB at 100 Hz, to a minimum of −55 dB at 300 Hz, and rising again to −35 dB at 500 Hz. The presentation levels from −20 to −10 dB ensured that the present tactile stimuli were above the threshold. The bottom panel of Fig. 4 shows frequency discrimination in just-noticeable-difference (filled circles), which increased relatively linearly from 25 Hz at 100 Hz standard frequency to 200 Hz at 500 Hz. The average Weber's fraction or the ratio between the just-noticeable-difference and the standard frequency was 0.34. These results were similar to previously reported tactile frequency discrimination at 100 and 200 Hz.11, 19 Although tactile frequency discrimination is at least 34 times worse than auditory frequency discrimination (Weber's fraction = 0.01 or better),²⁰ tactile stimulation should allow discrimination of fundamental frequency changes in four Mandarin tonal patterns, which were about 100 Hz between 100 and 200 Hz for the male talker in the present study (see Fig. 1).²¹ However, in cases of talkers, such as some females and children, whose fundamental frequencies are above 300 Hz, tactile stimulation does not have sufficient resolution to discriminate frequency changes in the Mandarin tonal patterns. In these instances, transposition of the fundamental frequency to a lower frequency range (e.g., <200 Hz) would be needed to enhance the observed ETS effect.²²

Figure 4.

Top panel: Tactile detection thresholds as a function of sinusoidal frequency, where 0 dB refers to 2.5 V presented to the tactile transducer. Bottom panel: Tactile frequency discrimination as a function of sinusoidal frequency, where the circles represent data from the present study, the crosses and triangles represent data from previous studies.11, 19

Conclusions

Like EAS, ETS enhances tone recognition in Mandarin-speaking cochlear-implant users. The present result, together with the previous study showing an ETS enhancement on cochlear-implant English sentence recognition in noise, suggest that ETS be a viable option for enhancing cochlear-implant performance in users who do not have any functional residual low-frequency acoustic hearing.

Author contributions

Huang J. and Zeng FG. were responsible for all phases of the research. Chang J. helped collect data and edit manuscript.

Conflict of interest

F.G.Z. is a co-founder of Nurotron Biotechnology and Sound Cure.

Edited by Yi Fang