Assessment and improvement of sound quality in cochlear implant users

Abstract

Objectives

Cochlear implants (CIs) have successfully provided speech perception to individuals with sensorineural hearing loss. Recent research has focused on more challenging acoustic stimuli such as music and voice emotion. The purpose of this review is to evaluate and describe sound quality in CI users with the purposes of summarizing novel findings and crucial information about how CI users experience complex sounds.

Data Sources

Here we review the existing literature on PubMed and Scopus to present what is known about perceptual sound quality in CI users, discuss existing measures of sound quality, explore how sound quality may be effectively studied, and examine potential strategies of improving sound quality in the CI population.

Results

Sound quality, defined here as the perceived richness of an auditory stimulus, is an attribute of implant‐mediated listening that remains poorly studied. Sound quality is distinct from appraisal, which is generally defined as the subjective likability or pleasantness of a sound. Existing studies suggest that sound quality perception in the CI population is limited by a range of factors, most notably pitch distortion and dynamic range compression. Although there are currently very few objective measures of sound quality, the CI‐MUSHRA has been used as a means of evaluating sound quality. There exist a number of promising strategies to improve sound quality perception in the CI population including apical cochlear stimulation, pitch tuning, and noise reduction processing strategies.

Conclusions

In the published literature, sound quality perception is severely limited among CI users. Future research should focus on developing systematic, objective, and quantitative sound quality metrics and designing therapies to mitigate poor sound quality perception in CI users.

Level of Evidence

NA

INTRODUCTION

Cochlear implants (CIs), or surgically implanted auditory prostheses, have successfully provided sound and speech perception to individuals all over the world who suffer from sensorineural hearing loss. Despite advances in CI technology, however, significant perceptual limitations remain, particularly for complex sounds like speech in noise, voice emotion, and music. One major limiting–but not well‐studied–perceptual construct is sound quality. Sound quality is different from sound appraisal, which is a construct not consistently defined in the literature but often referred to as subjective pleasantness or likeability of a sound.1, 2, 3, 4, 5, 6, 7 Sound quality, in contrast, is defined here as the perceived richness of an auditory stimulus. Sound quality and sound appraisal may be related to each other in some cases, but not always; for example, perceived pleasantness of a sound does not necessarily correspond to high sound quality. While a number of studies have highlighted diminished sound appraisal in CI users,1, 2, 3, 8 sound quality perception remains relatively unexplored. The few studies that have focused on sound quality suggest that it is diminished in CI‐mediated listening relative to normal hearing (NH) listeners.9, 10, 11 In this review, we suggest that sound quality is significantly impaired in CI users, as evidenced by a variety of limiting factors and research; identify existing measures of sound quality and discuss how it can be effectively studied; and explore potential strategies of improving sound quality in the CI population.

Limiting Factors

Sound quality in CI users is poor relative to NH people due to degradation of multiple auditory components. Arguably the aspect of sound most profoundly affected in electrical hearing is pitch, defined here as the perceptual correlate of frequency. Frequency perception, though not fundamental to speech intelligibility, plays a crucial role in the perception of more complex forms of sound such as speech prosody and music. In CI users, pitch perception is significantly degraded as evidenced by limited pitch discrimination, pitch change direction identification, harmony perception, recognition of pitch‐driven musical emotion, timbre identification, and song/melody recognition, and enjoyment and engagement in musical activities.2, 12, 13, 14, 15, 16, 17, 18 Limitations in CI‐mediated pitch perception are also manifested in poor perception of pitch‐driven voice emotion and cues.19, 20, 21 This degraded pitch quality in CI users stems from a combination of physical and electrophysical factors. In CI technology, pitch information is transmitted via two mechanisms: place pitch, in which the incoming signal stimulates physical location inside the cochlea corresponding to the transmitted frequency; and rate pitch, or the rate of electrode stimulation.

A healthy human cochlea transmits pitch according to a tonotopic frequency map, meaning that high frequencies are processed toward the cochlear base and lower frequencies towards the apex. This phenomenon is known as Greenwood's Function.22 CIs are designed to imitate this by transmitting pitch information to the electrode at the cochlear location biologically optimized to transmit the assigned frequency. Most often, however, the programmed characteristic frequency and the theoretical characteristic frequency (based on the location along the basilar membrane) do not match, leading to an inaccurate pitch percept.23, 24 This physical mismatch is due to a combination of variation in size of the cochlea, the length of the electrode array, proximity to nerve fibers, and insertion depth, among other factors. In addition to place pitch mismatch, a normal hearing cochlea utilizes 3,500 hair cells to transmit pitch, allowing for a broad and extremely precise frequency perception of 1,400 individual frequency steps between 20–20,000 Hz.12, 25 In contrast, a typical CI array contains at most 22 electrodes and transmits a significantly narrower range of ∼200–8500 Hz.26 Not only does this severely limit the pitch precision, but it also contributes to current spread, in which a single electrode stimulates a relatively broad population of cochlear nerve fibers. Current spread can lead to a range of frequencies being perceived as a single pitch, further degrading sound quality presented through electrical hearing. Current spread is exacerbated by stimulation at increased current levels, which is unfortunately necessary at times depending on individual anatomical and technological needs. Relatedly, pitch discrimination is furthermore worsened by channel interaction, in which both simultaneous and non‐simultaneous stimulation of adjacent electrodes results in a pitch percept somewhere in between the stimulus frequencies.15, 27

CIs also transmit pitch using temporal or rate pitch mechanisms, or the rate of electrode stimulation. While the firing rate of normal hearing auditory neurons can phase‐lock with incoming frequency information up to 5,000 Hz, CI users’ ability to discriminate temporal stimulation rates generally saturates above 300 Hz.28 Sound can be separated into temporal envelope and fine structure components. The temporal envelope consists of amplitude information and plays an important role in speech intelligibility, whereas fine structure corresponds to spectral cues and is more heavily utilized during perception of pitch, music, and sound localization.11, 29, 30, 31 Unfortunately, fine structure processing (FSP) has not been incorporated into the majority of current processing strategies. Continuous interleaved sampling (CIS), currently fundamental to processing strategies offered by all CI manufacturers, interleaves biphasic pulse trains so that no two pulses occur simultaneously.32 While this has helped mitigate channel interaction, CIS transmits only envelope information via fast fixed rate electrode stimulation and discards fine structure entirely. This lack of FSP can be detrimental to music sound quality perception. For example, Roy et al.11 found that while NH listeners are able to distinguish between music containing varying levels of bass information, CI users are not; the CI group provided similar sound quality ratings to music clips containing full bass as to the same clips missing up to 400 Hz of bass information. The authors stipulate that this could be due to a lack of FSP and a reliance on envelope information to process music, as clips of the same song or piece will have the same envelope, regardless of bass information present11 (Fig. 1). Recently, however, CI companies have begun integrating strategies that emphasize FSP. These FSP strategies may be superior over CIS strategies for processing complex stimuli heavily dependent on pitch, such as music,30, 33 though this requires further study.34, 35

Figure 1.

Average sound quality ratings for CI users and NH listeners listening to music clips containing varying levels of bass information (11). Bass was altered using high pass filters. Numbers on the x‐axis indicate cutoff frequency of each stimulus version. Error bars represent one standard deviation from the mean. Asterisks indicate a significant difference between CI and NH ratings (p<.0001).

In additional to pitch distortion, the volume/loudness range–or, the perceptual correlate of amplitude–is highly compressed in CI users. The ratio of the difference limen to the original stimulus intensity (known as Weber's fraction) is significantly higher in CI listeners compared to the NH population, indicating that CI users require a greater amplitude increase in order to discern a noticeable difference in loudness.36 Indeed, while NH listeners are able to discern a dynamic range of 120 dB with 6–100 discrete steps, CI users are only able to perceive a range of 6–30 dB with 20 discrete steps due mostly to high degree of neural synchrony, steep rate‐intensity functions, limited neuron survival and activity, along with other factors.26, 37, 38, 39, 40 Since intensity range is already compressed, CIs compress incoming signals further and transmit dynamic information by varying the current according to the amplitude signal. While this is somewhat effective, increasing current levels can contribute to current spread, interference and further subsequent pitch distortion.38, 41 Amplitude perception limitations can have enormous implications for perceived quality of sound, particularly of music, in which dynamics are fundamental to conveying musical emotion; musical crescendos (increases in volume) and decrescendos (decreases in volume) are tools that allow for emotional expression via dramatic buildup and resolution. Compressed dynamic range inherent to electrical hearing has also been shown to impact timbre perception and perception of speech, particularly vowel sounds.40, 42, 43 Amplitude variations additionally play a crucial role in voice emotion cues,20, 44, 45 suggesting that amplitude range compression may impact speech prosody recognition.

Sound Quality Assessment

There are few existing measures of CI‐mediated sound quality. The majority of sound perception measures that exist rely on subjective patient reporting of sound likeability, and while these provide enormous insight into CI‐mediated sound perception, appraisal (the subjective pleasantness or likeability of a sound) is independent from sound quality and should be studied as such. This separation is evidenced by studies indicating independence of perceptual accuracy and enjoyment.5, 6, 46 Most existing measures of sound quality employ subjective rating systems. Gfeller et al.47 developed a musical questionnaire featuring participant ratings of musical sound quality using a series of bipolar scales, such as “natural‐unnatural” and “empty‐full”. The vast majority of existing CI sound quality studies utilize either this instrument or an adaptation (for example, Lassaletta et al.10). Measures like these are certainly helpful, but do not offer an objective insight into the richness of sounds heard through a CI relative to acoustic hearing. One option utilized by many studies is to present the same auditory stimuli to NH listeners using both normal, acoustic hearing and using CI‐simulated stimuli. This partially removes subjectivity with a within‐subject comparison and eliminates biases of personal preference or familiarity. However, evidence that CI users consistently perform comparably to NH listeners with CI simulations is shaky at best48 and studies like these thus may not provide a reliable sound quality representation. By shedding a more accurate light on the perceptual auditory gaps that exist between NH listeners and CI users, tools that quantitatively and objectively measure sound quality would be monumental to furthering CI technology.

Roy, et al.11 developed one such tool with the MUltiple Stimulus with Hidden Reference and Anchor adapted for CI users (CI‐MUSHRA). The CI‐MUSHRA is adapted from the MUSHRA, a tool commonly used in the audio industry. The CI‐MUSHRA presents participants with a series of sound clips including varying levels of bass information and asks listeners to rate them for sound quality (Fig. 2). Low frequency information was chosen because of its role as an important sound quality parameter and its known impairment in CI‐mediated listening. Roy, et al.11 compared CI‐MUSHRA performance of CI users with that of NH listeners and found that the CI group consistently exhibited greater difficulty differentiating between low‐ and high‐quality sounds compared to the NH group, demonstrating that 1) available low frequency information is an important measure of sound quality, and 2) the CI‐MUSHRA provides a reliable and systematic metric of sound quality perception.

Figure 2.

Screenshot of the CI‐MUSHRA subject interface (11). Participants first complete the Training Phase (A) in which they simply click and listen to the reference, or full‐quality sound clip, along with the 6 versions of the reference carrying a range of sound quality levels. In the Testing Phase (B), subjects are presented with the reference and 6 versions of one sound at a time and use the sliding bars to rate the sound quality of each sound.

The development of additional sound quality measures that are both objective and quantitative would allow for a significantly more comprehensive understanding of what CI users hear, and would thus be invaluable to the improvement (normalization) of CI hearing. The ability to adapt a tool to fit various sound quality parameters would also be ideal. For example, the CI‐MUSHRA has been used evaluate sound quality as it relates to insertion angle/depth,49 reverberation,50 and modified processing strategies.51 Apart from bass frequency information, it could be further adapted to measure sound quality perception in the context of other adjusted parameters such as dynamic range, number of channels, or frequency mapping. Tools like the CI‐MUSHRA that are objective, quantitative, reliable, and adaptable will be monumentally helpful in identifying areas of normalization in CI hearing.

Apical Cochlear Stimulation for Cochlear Implant Sound Quality Improvement

Although there are many areas of research for sound quality improvement, we will focus on the topics of apical cochlear stimulation, place‐pitch maps, and noise reduction processing strategies. As mentioned previously, due to limitations in CI biomedical design and surgical technique, electrode arrays rarely come into contact with the apical regions of the cochlea, where low frequency sounds are encoded by place pitch stimulation. Low frequency information is particularly important in processing complex sounds, such as music. Thus, delivering low frequency information to CI users may be an effective way to improve sound quality. Current methods to enhance low frequency perception in electric hearing include acoustic stimulation of low frequency areas, deeper insertion depths, and bass‐enhancing modified processing strategies. The benefits of electric‐acoustic stimulation (EAS) in music and speech perception have been demonstrated repeatedly.52, 53, 54 In a recent study by Roy et al.,49 standard (31.5 mm) and medium (24 mm) array length Med‐EL CI users completed the CI‐MUSHRA task (described in the previous section), in which participants are asked to provide sound quality ratings to real‐world musical stimuli with increasing amounts of low‐frequency information removed. Imaging was used to confirm that medium arrays and standard arrays users had significantly different insertion depths. The study findings showed that CI users with greater apical stimulation reported sound quality ratings that more closely resembled their NH counterparts, suggesting superior sound quality perception. More recently, a sound processing strategy called partial bipolar stimulation emerged as means to expand low‐frequency range available to CI users.55, 56 Partial bipolar stimulation relies on current steering to create virtual “phantom” channels that extend beyond the physical end of an electrode array. A study enrolling 12 post‐lingually deaf CI users compared Phantom stimulation to the standard Advanced Bionics HiRes Fidelity 120 processing strategy. Although there was no significant difference between Phantom and the control processing strategy for most components of the music questionnaire, Phantom CI users reported a statistically significant difference in improved sound balance and preferred listening to music using this strategy.55 In another recent study, Munjal et al.51 utilized the CI‐MUSHRA to find that creation of a phantom electrode through partial bipolar stimulation allowed for superior (more normalized) sound quality perception relative to Fidelity 120 processing strategy. Such findings suggest that apical cochlear stimulation, whether it be by deeper angular insertions57 or current steering, may improve sound quality and listening experiences for CI users by delivering low‐frequency information.

Place‐Pitch Mapping for Cochlear Implant Sound Quality Improvement

As described previously, most CI electrode arrays are not designed with the length to reach the most apical regions of the cochlea. Furthermore, anatomic variations in cochlea lengths58 and intraoperative events affect individual electrode contact placement after array insertion. Consequentially, electrodes programmed to carry low‐frequency information (apically located on the electrode array) and electrodes programmed to carry high‐frequency information (basally located on the electrode array) commonly stimulate areas of the basilar membrane that contain spiral ganglion cells associated with a lower frequency and higher frequency, respectively.23, 24 This place‐pitch mismatch exists between the frequencies transmitted by the individual channels and the corresponding characteristic frequency given the final electrode position (Fig. 3). Evidence suggests that place‐pitch mismatch reduces CI‐mediated sound quality, and increases the interval of time and rate at which it takes for users to reach asymptotic levels of speech perception.57, 59 Recent advances in imaging, such as flat‐panel CT scans and other high‐resolution 3D techniques, allow for post‐implantation visualization of final electrode placement and thus offers the opportunity for personalized pitch‐place programming for improved sound quality.23, 60, 61 Although much research is needed in evaluating the impact of personalized pitch‐place mapping on speech and music perception, this field of work holds promise for improved sound quality and listening experience for CI users.

Figure 3.

Average predicted frequencies (black) versus average programmed ranges (orange) for 23 Med‐El standardarray cochlear implants (23). The black line indicates where the range median should be based on the predicted characteristic frequency. The orange bar indicates the programmed range of the cochlear implant. If there is a green line, it means that the calculated frequency is within the programmed range. In this graph, the average calculated frequencies for electrodes 4, 5, 6, and 7 are within the average programmed range.

Noise‐Reduction Processing Strategies for Cochlear Implant Sound Quality Improvement

One of the more well‐studied perceptual limitations of CIs is poor spectral resolution delivery. Much of this has been attributed to current spread between neighboring electrode contacts.62 Over the years, many noise‐reduction algorithms have emerged in an attempt to improve CI sound processing by removing interfering noise from the desired signal conveyed in each independent spectral channel. This in theory increases the audibility of speech in noisy environments, a known obstacle for many CI users. While these strategies have provided substantial noise reduction benefit, there is clear room for improvement as CI users continue to experience difficulty in perceiving speech in noise.63 A common problem limiting the extent of these processing strategies is distortion of the target signal. Among single‐channel noise reduction algorithms, strategies such as spectral subtraction,64, 65 binary mask,66 or Wiener‐filter algorithms67, 68 have emerged. To summarize a few of the noise‐reduction processing strategies: spectral subtraction computes the short‐time spectral magnitude of speech by subtracting the estimated noise spectral magnitude from the noisy speech spectral magnitude.69 Binary mask algorithms emphasize time‐frequency points where the target sound stimulus is most prominent and exploits the disconnectedness between the background and target spectra. The Wiener‐filter approach functions by comparing the incoming stimuli to a dynamic extrapolation of background noise levels. The Wiener‐filter approach is particularly effective when used with stationary background noise such as white‐ or car‐noise.70 In the past, studies have tested speech enhancement algorithms with the argument that conclusions drawn from NH listeners can be generalized to severely hearing impaired people. A recent study by Koning et al.48 found that results obtained with speech enhancement algorithms with NH subjects do not translate to CI subjects, suggesting the importance of developing speech processing strategies with the end‐users (CI listeners) themselves.

CONCLUSIONS

Within the past 10 years, CI research has expanded beyond speech intelligibility to include perception of discrete auditory components such as pitch, amplitude and rhythm. This research is important and provides a foundation on which to better understand CI limitations; however, sound quality as a separate construct is just as crucial to understanding optimized auditory performance and yet remains poorly studied. Physical and electrophysical limitations inherent to CI‐mediated listening, in combination with existing studies of CI performance on various auditory tasks, suggest that sound quality perception in the CI population is limited by a range of factors, most notably pitch distortion and dynamic range compression. The study of CI‐mediated sound quality requires the development of more objective, systematic, and quantitative measures, of which there are currently very few. There exist a number of promising strategies to improve sound quality perception in the CI population. Research‐based efforts to study and improve sound quality in CI users should include the development of effective measurement tools along with therapies focused on apical cochlear stimulation, place‐pitch maps, and noise reduction processing strategies.