Abstract
More than 60,000 people worldwide use cochlear implants as a means to restore functional hearing. Although individual performance variability is still high, an average implant user can talk on the phone in a quiet environment. Cochlear-implant research has also matured as a field, as evidenced by the exponential growth in both the patient population and scientific publication. The present report examines current issues related to audiologic, clinical, engineering, anatomic, and physiologic aspects of cochlear implants, focusing on their psychophysical, speech, music, and cognitive performance. This report also forecasts clinical and research trends related to presurgical evaluation, fitting protocols, signal processing, and postsurgical rehabilitation in cochlear implants. Finally, a future landscape in amplification is presented that requires a unique, yet complementary, contribution from hearing aids, middle ear implants, and cochlear implants to achieve a total solution to the entire spectrum of hearing loss treatment and management.
Introduction
Cochlear implants are the only medical intervention that can restore partial hearing to a totally deafened person via electric stimulation of the residual auditory nerve. Twenty years ago, the cochlear implant started as a single-electrode device that was used mainly for enhancing lipreading and providing sound awareness. Today, it is a sophisticated multielectrode device that allows most of its 60,000 users to talk on the phone. The implant candidacy has been expanded to include children as young as 3 months and adults who have significant functional residual hearing, particularly at low frequencies. Commercial and research enterprises have also matured to generate annual revenues of hundreds of millions of dollars and to command attention from multidisciplinary fields including engineering, medicine, and neuroscience.
This review describes what a cochlear implant is and how it works, and discusses its past, present, and future. It also focuses on audiologic and clinical issues, engineering issues, anatomic and physiologic issues, and cochlear-implant performance in basic psychophysics, speech, music, and cognition. Finally, trends in cochlear implants from clinical, research, and system points of view are presented.
History of Cochlear Implants
Development of cochlear implants can be traced back at least 200 years to the Italian scientist Alessandro Volta, who invented the battery (the unit volt was named after him). He used the battery as a research tool to demonstrate that electric stimulation could directly evoke auditory, visual, olfactory, and touch sensations in humans (Volta, 1800). When placing one of the two ends of a 50-volt battery in each of his ears, he observed that
“… at the moment when the circuit was completed, I received a shock in the head, and some moments after I began to hear a sound, or rather noise in the ears, which I cannot well define: it was a kind of crackling with shocks, as if some paste or tenacious matter had been boiling … The disagreeable sensation, which I believe might be dangerous because of the shock in the brain, prevented me from repeating this experiment…”
Safe and systematic studies on the effect of electric stimulation on hearing were not reported for another 150 years, until modern electronic technology emerged. Equipped with vacuum tube based oscillators and amplifiers, Stevens and colleagues identified at least three mechanisms that were responsible for the “electrophonic perception” (Stevens, 1937; Stevens and Jones, 1939; Jones et al., 1940).
The first mechanism was related to the “electromechanical effect,” in which electric stimulation causes the hair cells in the cochlea to vibrate, resulting in a perceived tonal pitch at the signal frequency at which they were acoustically stimulated.
The second mechanism was related to the tympanic membrane's conversion of the electric signal into an acoustic signal, resulting in a tonal pitch perception but at the doubled signal frequency. Stevens and colleagues were able to isolate the second mechanism from the first, because they found that only the original signal pitch was perceived with electric stimulation in patients lacking the tympanic membrane.
The third mechanism was related to direct electric activation of the auditory nerve, because some patients reported a noise-like sensation in response to sinusoidal electrical stimulation, much steeper loudness growth with electric currents, and occasional activation of facial nerves.
The first direct evidence of electric stimulation of the auditory nerve, however, was offered by a group of Russian scientists who reported that electric stimulation caused hearing sensation in a deaf patient whose middle and inner ears were damaged (Andreev et al., 1935).
In France, Djourno and colleagues reported in 1957 successful hearing by using electric stimulation in two totally deafened patients (Djourno and Eyries, 1957; Djourno et al., 1957a; Djourno et al., 1957b). Their success spurred an intensive level of activity in attempts to restore hearing to deaf people on the United States west coast in the 1960s and 1970s (Doyle et al., 1964; Simmons et al., 1965; Michelson, 1971; House and Urban, 1973).
Although the methods were crude compared with today's technology, these early studies identified critical problems and limitations that needed to be considered for successful electric stimulation of the auditory nerve. For example, they observed that compared with acoustic hearing, electric hearing of the auditory nerve produced a much narrower dynamic range, much steeper loudness growth, temporal pitch limited to several hundred Hertz, and much broader or no tuning. Bilger provided a detailed account of these earlier activities (Bilger, 1977a, 1977b; Bilger and Black, 1977; Bilger et al., 1977a, 1977b; Bilger and Hopkinson, 1977).
On the commercial side, in 1984 the House-3M single-electrode implant became the first device approved by the U.S. Food and Drug Administration (FDA) and had several hundred users. The University of Utah developed a six-electrode implant with a percutaneous plug interface and also had several hundred users. The Utah device was called either the Ineraid or Symbion device in the literature and was well suited for research purposes (Eddington et al., 1978; Wilson et al., 1991b; Zeng and Shannon, 1994). The University of Antwerp in Belgium developed the Laura device that could deliver either 8-channel bipolar or 15-channel monopolar stimulation. The MXM laboratories in France also developed a 15-channel monopolar device, the Digisonic MX20. These devices were later phased out and are no longer available commercially. At present, the three major cochlear-implant manufacturers are Advanced Bionics Corporation, U.S.A. (Clarion); MED-EL Corporation, Austria; and Cochlear Corporation, Australia (Nucleus).
Present Status
At present, the number of cochlear-implant users has reached 60,000 worldwide, including 20,000 children, and is still growing exponentially. Functionally, the cochlear implant has evolved from the single-electrode device that was used mostly as an aid for lipreading and sound awareness to a modern, multielectrode device that can allow an average user to talk on the telephone. Figure 1 documents the advances made in cochlear implants in terms of speech performance over the past 20 years.
The early single-electrode device provided essentially no open-set speech recognition except in a few subjects. The steady improvement at a rate of about 20 percentage points in speech recognition per 5 years since 1980 was particularly apparent with the Nucleus device. Despite the differences in speech processing and electrode design, there appears to be no significant difference in performance among the present cochlear-implant recipients who use different devices.
Cochlear-implant research has also improved and matured as a scientific field. Figure 2 shows the annual number of publications on cochlear implants and hearing aids retrieved from the MEDLINE database. As of January 27, 2004, 2,699 publications in the database were related to cochlear implants. For comparison, the entry “hearing AND aid” yielded 2,740 publications, while that for “auditory” yielded 58,551. The annual number of publications clearly shows an exponential growth pattern, mirroring the growth pattern in the number of cochlear-implant users and also, most likely, in the amount of funding.
Research in hearing aids clearly started earlier than it did in cochlear implants, as evidenced by the relatively constant 10 to 20 annual publications between the early 1960s and middle 1970s. Hearing aid publications took off in the middle 1970s and have since increased steadily to about 100 publications at present. For comparison, cochlear-implant publications did not start to appear in the database until 1972, when animal studies were conducted with an inner ear electrode implant (Horwitz et al., 1972). The first human study was reported by Dr. William House in 1974 (House, 1974). The implant literature has grown exponentially since the early 1990s and exceeded the number of hearing aid publications in the middle 1990s, reaching 250 publications in the year 2000.
Audiologic and Clinical Issues
Patient Selection Criteria
As another sign of progress and maturity in cochlear implants, the audiologic criteria for cochlear implantation has continuously relaxed from bilateral total deafness (>110 dB HL) in the early 1980s, to severe hearing loss (>70 dB HL) in the 1990s, and then to the current suprathresh-old speech-based criteria (<50% open-set sentence recognition with properly fitted hearing aids). Cochlear implantation for both adults and children has received FDA approval.
In cases of auditory neuropathy, patients with a nearly normal audiogram have received cochlear implants and achieved better performance, particularly in noise (Peterson et al., 2003). In cases of combined acoustic and electric hearing, cochlear implantation with short electrodes has been proposed for patients with significant residual low-frequency hearing (von Ilberg et al., 1999; Gantz and Turner, 2003).
As the performance of the cochlear implant continues to get better, its cost will eventually drop with large volume and the candidate base for cochlear implants will continue to grow and overlap with the traditional hearing aid market. A future hearing-impaired listener may have a choice between hearing aids and cochlear implants, or even both.
Presurgical and Postsurgical Issues
Despite significant research effort, there is no reliable and accurate presurgical predictor of postsurgical performance in cochlear implants. Several factors such as duration and etiology of deafness, as well as presurgical auditory and speech performance, have been shown to be correlated with postsurgical performance (Blamey et al., 1996; van Dijk et al., 1999; Gomaa et al., 2003). However, no presurgical factors have been able to account for a significant enough amount of postsurgical performance variability that would allow the physician to have a high level of confidence to tell the prospective candidate how well he or she may do with the implant. Innovative means such as brain imaging and cognitive measures might be used in the future to help increase accuracy and reliability of presurgical prediction of postsurgical performance (Giraud et al., 2001; Lee et al., 2001; Pisoni and Cleary, 2003).
The cochlear-implant surgical approach and procedure has also been greatly improved and simplified as physicians continue to gain experience, thus minimizing the surgical trauma, duration, and potential complications. For example, the duration of surgery has been decreased from a typical hospital stay of 1 to 2 days in the early 1980s to the present few hours on an outpatient basis.
An increased risk of bacterial meningitis with cochlear implants has been a recent concern, particularly for those who have received an implant with a positioner that requires a larger opening in the cochlea (Reefhuis et al., 2003). However, the positioner is no longer being used, and presurgical vaccination and postsurgical monitoring are effective to prevent and treat any potential bacterial infections after a patient receives the implant.
Cost-Utility Analysis and Reimbursement
Cost-utility analysis has been performed in both adults and children to show a favorable comparison of cochlear implants with other life-saving medical devices in terms of improving the implant user's quality of life (Cheng and Niparko, 1999; Palmer et al., 1999; Cheng et al., 2000). These studies, coupled with formal FDA approval, have helped convince many insurance companies to pay full or nearly full device, surgery, and hospital costs for cochlear implants, which may range from US $40,000 to $75,000. However, Medicare still reimburses significantly less (about US $20,000), requiring those who depend on Medicare to pay the remaining cost. In developing countries, the surgical cost is low but the device cost is prohibitively high, thus denying the access of cochlear implants to the overwhelming majority of deaf people there.
Educational and Linguistic Issues
A key question in implanting deaf children is whether they can develop normal language in a mainstream educational environment. The evidence is accumulating to support the hypothesis that early implantation promotes the maturation process in the auditory cortices and normal language development.
Two research groups have provided evidence showing delayed cortical maturation without auditory stimulation and additionally, that cochlear implants can restart this maturation process in children who received cochlear implants before about 7 years old (Ponton et al., 1996; Sharma et al., 2002). Correspondingly, language development measures have shown that children with cochlear implants performed significantly better than expected from unimplanted deaf children and approached or reached the rate of language development observed in children with normal hearing (Moog and Geers, 1999; Svirsky et al., 2000).
(Re)habilitation Issues
No formal structured habilitation or rehabilitation protocols have been developed for children or adults who have received cochlear implants. Most patients have received only basic follow-up visits to tune the “map” in the speech processor in their cochlear implant. On average, postlingually deafened cochlear-implant users go through a learning or adaptation process from a few months to as long as a few years, during which their speech performance continues to get better (Tyler et al., 1997). The individual variability is large in terms of both the speed of adaptation and the final plateau performance. The causes of this large individual variability are not clear, possibly reflecting more patient related variables than device related variables (Wilson et al., 1993).
The rapidly growing number of cochlear-implant users has demanded that researchers and educators develop field-tested, effective, structured (re)habilitation programs and protocols, and additionally, train audiologists, speech pathologists, and special education teachers in the skills required to work with adult and child implant users.
Engineering Issues
Taking advantage of explosive growth and innovations in technology, particularly in microelectronics in the last two decades, cochlear implants have evolved from analog to digital, from single electrode to multiple electrodes, from percutaneous plugs to transcutaneous transmission, and from simple modulation to complicated feature extraction processing.
System Design
In normal hearing, sound travels from the outer ear through the middle ear to the cochlea, where it is converted into electric impulses that the brain can understand. Most cases with severe hearing loss involve damage to this conversion of a sound to an electric impulse in the cochlea. A cochlear implant bypasses this natural conversion process by directly stimulating the auditory nerve with electric pulses. Hence, the cochlear implant will have to mimic and replace auditory functions from the external ear to the inner ear.
Figure 3 shows a modern cochlear implant worn on the body. The basic components include a microphone that sends the sound via a wire to the speech processor that is worn externally, where it is converted into a digital signal. The signal travels back to a headpiece, held in place by a magnet attached to the implant on the other side of the skin, that transmits coded radio frequencies across the skin. The implant decodes the signals, converts them into electric currents, and sends them along a wire threaded into the cochlea. The electrodes at the end of the wire stimulate the auditory nerve connected to the central nervous system, where the electrical impulses are interpreted as sound.
Although the specific components and designs may be different among the implant manufacturers, the general working principles are the same. For example, the microphone can be mounted at the ear level on top of the ear hook, or pinned at the chest level. The shape, color, and radio frequency (RF) of the transmission coil may be different, but the magnetic coupling is identical. The following section discusses in detail speech processors, electrodes, telemetry, and fitting systems in the present cochlear implants.
Speech Processor
The speech processor is the brain in a cochlear implant. It extracts specific acoustic features, codes them via RF transmission, and controls the parameters in electric stimulation. In the earliest single-electrode 3M/House implant, an analog sound waveform was compressed in amplitude and then was simply amplitude modulated to a 16-kHz sinusoidal carrier to serve as the effective electric stimulus.
All modern multielectrode implants have been developed to take advantage of the so-called tonotopic organization in the cochlea, namely, the apical part of the cochlea encodes low frequencies while the basal part encodes high frequencies. These implants, therefore, all have implemented a bank of filters to divide speech into different frequency bands, but they differ significantly in their processing strategies to extract, encode, and deliver the right features.
One school of thought was to deliver the band-specific, amplitude-compressed analog waveforms to different electrode locations in the cochlea (Eddington et al., 1978). Figure 4 shows the schematics of a typical compressed-analog strategy. In the Ineraid device, the return or ground electrode is located outside of the cochlea in temporalis muscle (the so-called monopolar stimulation mode). In the Clarion device, the compressed-analog processing is called the Simultaneous-Analog-Strategy (SAS), in which the return electrode can be an adjacent intro-cochlear electrode (the so-called bipolar mode).
Another school of thought was based on speech production and perception, in which spectral peaks or formants, which reflect the resonance properties of the vocal tract, are extracted and delivered to different electrodes according to the presumed tonotopic relationship between the place of the electrode and its evoked pitch. The earliest version was the F0F2 strategy implemented in the Nucleus Wearable Speech Processor (WSP II) (Clark et al., 1984), in which the fundamental frequency (F0) and the second formant frequency (F2) were extracted, with F0 determining the stimulation rate and F2 determining the stimulation electrode location of the electrode. Sequentially, the first formant (F1) was added in the Nucleus WSP III processor and up to six spectral peaks in the Nucleus MPEAK strategy (Skinner et al., 1991).
A third school of thought was based on the implicit encoding of temporal envelope cues and could be traced back to earlier work in speech vocoder (Dudly, 1939). Figure 5 shows the schematic diagram of the Continuous-Interleaved-Sampling (CIS) processor, an implant strategy intended to extract and faithfully deliver the temporal envelope cues (Wilson et al., 1991a).
CIS preprocessing is similar to the compressed-analog processor, but the band-specific signal is subject to envelope extraction via a rectifier and a low-pass filter with its cutoff frequencies typically set at a range of 160 to 320 Hz. The slowly varying temporal envelopes from 3 to 4 spectral bands can deliver high levels of speech intelligibility in quiet (Shannon et al., 1995). The envelope, again, has to be compressed, usually logarithmically, to match the narrow dynamic range in electric stimulation.
To avoid electrode interactions caused by electrical field overlap in simultaneous stimulation, which conceivably can smear the band-specific envelope cues, the envelope from each subband is then amplitude-modulated to a high-rate (>800 Hz) pulsatile carrier that interleaves with pulsatile carriers from other subbands. In other words, only one electrode is being stimulated at any given time.
Variations of the temporal envelope extraction have been developed and implemented. In a typical CIS implementation, the number of bandpass filters is identical to the number of electrodes, ranging from 6 in the earlier implementation in the Ineraid device to 8 in the Clarion CI device, 12 in the Medel device, and 22 in the Nucleus 24 device. Alternatively, the number of filters (m) can be greater than the number of stimulating electrodes (n), in which the envelopes from 6 to 8 subbands with the highest energy are used to stimulate the corresponding electrodes. This implementation has been called the n-of-m, or peak picking strategy, and is currently used in the Nucleus SPEAK processor (McDermott et al., 1992).
To balance the need for high-rate stimulation and for minimizing electrode interaction, pulsatile carriers from two or more distant electrodes may be simultaneously stimulated, and hence are called the Paired Pulsatile Sampler (PPS) and Multiple Pulsatile Stimulation (MPS), respectively (Loizou et al., 2003). To take advantage of better temporal envelope representation with high-rate stimulation (>2000 Hz), all manufacturers have recently introduced high-rate stimulation strategies, including the HiResolution from Advanced Bionics, ACE from Cochlear Corporation, and Tempo from Med El. Figure 6 summarizes these processing strategies on the basis of their different philosophies and implementations. Their corresponding performance in speech can be found in Figure 1.
Electrodes
Early electrodes included copper and gold wires, but the modern electrodes are all made of platinum or platinum-iridium alloy. These electrodes, however, differ significantly both by geometric parameters and how they are stimulated. Figure 7 shows a schematic drawing of an electrode array inserted into the scala tympani (bottom compartment) of the cochlea. In this figure, the electrode array has made two full turns and stayed in the scala tympani inside the cochlea. In reality, the electrode array may not have such deep insertion, rather it may have kinks and reversals, or even penetrate the basilar membrane ending up in the scale vestibuli, the top compartment (Skinner et al., 2002).
The electrodes are similar to the rings presented in Figure 7 in the Nucleus device, but are single balls in the Clarion device and double-balls (like a dumb bell) in the Medel device. The distance between electrode contacts may be fixed at 0.75 mm in the Nucleus device or varied from far to near towards the apical end in the recent Clarion device. The electrode array may be shortened significantly for shallow insertion in combined acoustic and electric stimulation, or split into double arrays to be inserted separately into the first and second turns in case of ossification.
Depending on the location of the return or ground electrode, electrode configurations can vary in the mode of monopolar, bipolar, and tripolar stimulation:
In a monopolar mode, the return electrode is located outside of the cochlea, usually in the temporalis muscle behind the ear but can be attached to the case housing the internal electronics.
In a bipolar mode, the return electrode is an adjacent intro-cochlear electrode to the stimulation electrode.
In a tripolar mode, the return electrodes are two adjacent electrodes with each receiving half of the current delivered to the stimulating electrode.
Both modeling work and physiologic data have shown decreased spatial activation from monopolar, bipolar, to tripolar stimulation (Jolly et al., 1996; Bierer and Middlebrooks, 2004). Psychophysical data have shown an increased threshold from monopolar to bipolar stimulation, but whether bipolar stimulation can produce restricted stimulation and better speech performance has been equivocal (Pfingst et al., 1995; Pfingst et al., 2001).
Telemetry
Most modern cochlear implants have already had or will soon have sophisticated telemetry functions that allow close, accurate monitoring and measurement of electrode impedance, electrical field distribution, and nerve activities. The impedance monitoring can detect both open and shorted electrodes, while electrical field distribution and nerve activities allow objective measures of electrode interaction and nerve survival. These measures are becoming increasingly important for the objective fitting of the cochlear implant, particularly in children who cannot provide reliable subjective responses (Brown, 2003).
Fitting Systems
Each implant user has to be individually fitted or mapped to ensure safe and effective electric stimulation. Both the choice of different speech processing strategies and the number of adjustable parameters have increased significantly with the modern devices, allowing an individual user to store multiple maps in the speech processor for listening to different sounds in different environments such as quiet versus noise and speech versus music.
Typically, audiologists spend the most time in measuring the appropriate current values on each individual electrode that reaches the threshold (T-level) and the maximum or most comfortable loudness level (M-level or C-level). This amplitude map needs to be adjusted, particularly in the first several months following the first turn-on of the speech processor, as both the implant system and the implant user are being adapted to electric stimulation.
The goal of amplitude mapping is to optimally convert acoustic amplitudes in speech sounds to electric currents that evoke sensations that are between just audible and the maximal comfortable. Therefore, in addition to the electric dynamic range defined by the T and C levels, two important parameters need to be measured and adjusted for optimal amplitude mapping.
The first parameter is the input acoustic range, which used to be set at 30 dB but is now usually set at 50 to 60 dB to accommodate the amplitude variations in speech sounds (Zeng et al., 2002). The second parameter is the nature and degree of the compression between acoustic and electric amplitudes. Both power and logarithmic compression functions have been used in the current devices. Although the degree of compression shows a relatively minor effect on speech intelligibility, the compression that restores normal loudness sensation tends to provide better speech intelligibility and quality (Fu and Shannon, 1998; Zeng and Galvin, 1999). Usually, the input dynamic range and the compression function are set at the default values suggested by the manufacturers.
Unfortunately, audiologists typically spend little or no time on the frequency-to-electrode map, an important fitting parameter. As suggested by physiologic evidence, the auditory nerve is tonotopically organized depending on its location in the cochlea: the nerve innervating the apical cochlea produces pitch sensation corresponding to low frequency and that innervating the basal cochlea corresponds to high frequency (Bekesy, 1960; Liberman, 1982). However, there is simply no guarantee that pitch will be determined by the relative electrode position.
Depending on insertion depth, presence of kinks in the electrode array, and the extent and pattern of nerve survival, the same electrode may produce a different pitch sensation on different individuals, whereas different electrodes evoked the identical pitch. Even worse, pitch reversal could occur in which the apical electrode evoked a higher pitch sensation than the more basal electrodes. This mismatched frequency-to-electrode map is one of the major reasons for the cochlear-implant users’ poor performance in tests of noise and music perception (see sections on speech and music performance).
Finally, an important goal in fitting a cochlear implant is to get rid of electrodes that produce undesirable stimulations. The complications include stimulation of other facial nerves and muscles, such as vibration, pain, eye twitching, and vestibular responses (Niparko et al., 1991; Stoddart and Cooper, 1999). Occasionally, these complications can be avoided by adjusting the electric stimulation parameters such as the pulse duration and the electrode mode. Guidelines are lacking, so only a few experienced audiologists have ventured into this part of fitting the systems.
Anatomic and Physiologic Issues
The unique nature of electric stimulation is likely to produce perceptual consequences in electric hearing. Highlighted here are the anatomic and physiologic differences between acoustic and electric stimulation that will pave the way to an examination of their perceptual consequences in later sections.
Cochlea and Auditory Nerves
A normal human cochlea has roughly 3,000 inner hair cells that are tuned to different frequencies from 20 to 20,000 Hz. Each of the inner hair cells has 10 to 20 innervating auditory nerve fibers that send information to the central nervous system. In a deafened ear, the number of auditory nerve fibers is likely to be significantly less. The surviving fibers are likely to have an unhealthy status such as shrinkage, loss of dendrites, and demyelination (Linthicum et al., 1991; Nadol et al., 2001).
The degree and pattern of nerve survival is related to the etiology and duration of deafness as well as surgical trauma. Chronic and patterned electric stimulation has been shown to be a potent means of promoting nerve survival in deafened animals, providing further evidence for early intervention of cochlear implants in deaf children (Leake et al., 1999).
Differences Between Acoustic and Electric Stimulation
Different from acoustic stimulation, an auditory nerve is directly activated by membrane potential changes (depolarization) in electric stimulation. Neither passive nor active mechanical tuning is present in electric stimulation, rather the excitation pattern is determined by the electric field distribution, the cochlear electrical impedance, and the excitability of the nerve tissues (van den Honert and Stypulkowski, 1987; Abbas and Brown, 1988; Suesserman and Spelman, 1993; Frijns et al., 1996).
Two additional differences are also apparent between acoustic and electric stimulation, producing significant consequences in nerve responses. First, the loss of cochlear compression (caused by the outer hair cell function) produces much steeper rate-intensity functions in electric stimulation than in acoustic stimulation (Javel and Shepherd, 2000).
Second, the lack of stochastic synaptic transmission produces highly synchronized firing in electrically stimulated nerves (Dynes and Delgutte, 1992; Litvak et al., 2001). The steep rate-intensity function is likely to contribute to the narrow dynamic range while the highly synchronized response is likely to contribute to the fine temporalmodulation detection in cochlear-implant users (see section on psychophysical performance).
Central Responses to Electric Stimulation
The central auditory system shows a great degree of plasticity in response to deprivation of sensory input and its re-introduction via electric stimulation (Shepherd and Hardie, 2001; Kral et al., 2002). For example, cells in the cochlear nuclei shrink in response to a lack of sensory input and then come back to normal size as a result of electric stimulation. The inferior colliculus has significantly different temporal and spatial properties in response to a different pattern and duration of electric stimulation (Snyder et al., 1990; Vollmer et al., 1999; Moore et al., 2002). The auditory cortex also shows totally different responses to electric stimulation in a cat model of acquired and congenital hearing loss, with the latter having significantly less cortical activity (Klinke et al., 1999).
This finding in animal models parallels the human brain imaging data showing also significantly less cortical activity in prelingually deafened persons than in postlingually deafened persons (Lee et al., 2001). What is more interesting is that brain imaging suggests a strong cross-modality plasticity: cochlear-implant users recruit visual cortex to help perform auditory tasks, particularly among the better implant users (Giraud et al., 2001).
Psychophysical Performance
Because the cochlear implant bypasses the first stage of processing in the cochlea and stimulates the auditory nerve directly, it is important to compare the psychophysical performance between acoustic and electric hearing. The differences in intensive, spectral, and temporal processing are highlighted to provide not only critical information regarding the stimulus coding in electric hearing but also, indirectly, the role of cochlear processing in auditory perception.
Intensity, Loudness and Dynamic Range
A person with normal hearing boasts a 120-dB dynamic range and as many as 200 discriminable steps within this range. In contrast, a cochlear-implant user typically has a 10-db to 20-dB dynamic range and 20 discriminable steps (Nelson et al., 1996; Zeng et al., 1998; Zeng and Shannon, 1999). In acoustic hearing, loudness grows as a power function of intensity, whereas loudness tends to grow as an exponential function of electric currents in electric hearing (Zeng and Shannon, 1992, 1994). These differences are likely due to the loss of cochlear processing and have to be accounted for in both the design and fitting of the speech processor in cochlear implants (see section on the engineering issues).
Frequency, Pitch, and Tonotopic Organization
Frequency is likely to be encoded by both time and place mechanisms in normal-hearing listeners. The time code reflects the auditory nerve's ability to phase lock to an acoustic stimulus for frequencies up to 5,000 Hz, whereas the place code reflects the cochlear filter's property to divide an acoustic stimulus into separate bands. The center frequency of these bands corresponds to different places along the cochlea and is preserved along the entire auditory pathway, including the auditory cortex. The frequency-to-structure map is called tonotopic organization.
In cochlear implants, the time code is mimicked by varying the stimulation rate, whereas the place code is approximated by the electrode position. Psychophysical data have shown that this time code is limited to 300 to 500 Hz and the differences in electrode insertion depth and kinks, and nerve survival patterns comprise the place code (Nelson et al., 1995; Collins et al., 1997; Zeng, 2002).
Temporal Processing
Traditional measures in temporal processing include temporal integration, gap detection, and temporal modulation transfer function. In general, cochlear-implant users perform nearly normally or even slightly better than their normal-hearing counterparts for these tasks. This normalcy, however, is usually achieved after the abnormal intensity processing in electric hearing has been accounted for. For example, a normal-hearing person's temporal integration function has a slope of roughly −3 dB per doubling duration for durations up to 100 to 200 milliseconds (msec). The implant data show a similar time course of integration, but a much shallower temporal integration function, with a slope as flat as −0.06 dB per doubling duration (Donaldson et al., 1997).
This difference in slope between acoustic and electric stimulation is most likely due to the loss of cochlear compression in cochlear implants. Of interest is that the loss of compression has a beneficial consequence in temporal processing, allowing the implant users to detect much smaller amplitude modulations (sometimes 1% or less) than the typical 5% to 10% modulation achieved by the normal-hearing listeners (Shannon, 1992).
Figure 8 shows gap detection data as a final example to illustrate the interaction between temporal and pitch resolution. The data were collected from five Ineraid implant subjects using the most apical electrode in a monopolar mode. The stimuli included two sinusoidal markers with each having a 200-msec duration and 0-msec ramps. The first marker frequency was always 100 Hz, whereas the second marker frequency was systematically varied from 100 to 3,000 Hz (x-axis). The standard had a 0-msec gap between the markers, while the signal always included a gap. The initial value for the gap was large for easy detection, but was adaptively adjusted to converge to a value that corresponded to 79.4% correct level of performance in a two-interval, forced-choice procedure. All stimuli were presented at the most comfortable level. Figure 8 demonstrates a dual-mechanism pattern, in which gap detection was at a minimum of about 4 msec when the second marker frequency was equal to the 100-Hz first marker frequency and then monotonically increased to a plateau value of about 30 msec when the second marker frequency reached 300 Hz and above. The plateau between 300 and 3,000 Hz suggests that listeners with implants cannot discern pitch differences at frequencies above 300 Hz.
Complex Processing
In terms of temporal masking, both normal-hearing and cochlear-implant listeners show a 10- to 20-msec backward masking effect and a 100- to 200-msec forward masking effect (Shannon, 1990). Simultaneous masking in terms of detection of tone in noise and overshoot has not been reported in the implant literature.
Figure 9 shows preliminary overshoot data from a Nucleus-24 implant subject (unpublished data by Han, Chen, and Zeng at the University of California, Irvine). The masker was a 400-msec, 1,000-Hz biphasic pulse train (100 μsec/phase). The probe was a 10-msec, 1,000-Hz biphasic pulse train (100 μsec/phase) that was presented at various temporal positions from a 0- to 390-msec delay from the onset of the masker. The masker was presented at four levels from threshold to maximal comfortable loudness.
There was no sign of any significant overshoot effect, defined by the probe threshold difference between the onset (e.g., 0 msec) and the steady-state (e.g., 200 msec) positions for the probe. Spatial separation between the masker and the probe to two different electrodes did not produce any significant overshoot effect either. The lack of the overshoot effect in cochlear implants may reduce the temporal edge enhancement as observed in normal-hearing listeners (Formby et al., 2000) and an attempt to restore this enhancement has been shown to improve speech performance (Vandali, 2001).
Spatial masking in electric stimulation reflects mostly the overlap in electrical fields between electrodes. Figure 10 shows schematically how electrode interactions can reduce the number of independent neural or functional channels. The top panel shows two totally independent electrodes and the bottom panel shows two totally dependent electrodes. The cause for such electrode interactions may be sparse nerve survival distribution, or long distance between the electrodes and the neurons, or both.
This schematic demonstration should also make a clear distinction between two often-confused concepts in relation to the number of electrodes and the number of channels. If two electrodes are totally dependent, then they are effectively acting as a single channel. Therefore, the number of electrodes is determined physically and is most likely greater than the number of effective channels due to electrode interactions.
Figure 11 shows the consequence of electrode interactions when cochlear implants are fitted (from #4 Quarterly Progress Report, 1999, by Zeng et al. for NIH Contract NO1-DC-92100 on Speech Processors for Auditory Prostheses). We first performed the standard Clarion clinical fitting procedure to establish the T and M levels on each individual electrode (open symbols) for an SAS user. Then we set the T level equal to the M level in the clinical fitting system, and by doing so, we fixed the current-level contour across the electrodes. We changed the overall volume control to move this current-level contour up and down while we used live voice to ask the subject to report at what level the speech was first audible (“speech-adjusted T level”) and was maximally comfortable (“speech-adjusted M level”). Figure 11 shows that the T and M levels were greatly reduced with the new speech-adjusted procedure (solid symbols).
These results suggest that speech-adjusted M-levels are needed to ensure that stimulation not be overly loud when all eight electrodes are simultaneously stimulated with a realistic broad-band signal similar to speech. As a matter of fact, this fitting protocol that takes electrode interactions into account can produce better speech performance and has been adopted by manufacturers in the newer versions of the implant fitting software.
Other complex psychophysical tasks that have been performed in the evaluation of cochlear implants have shown a positive correlation to speech performance. For example, modulation detection and interference in the time domain have been measured and were highly correlated to speech performance (Fu, 2002; Chatterjee, 2003). On the other hand, the resolution of complex spectral patterns (i.e., rippled noise detection in the spectral domain) has also been shown to be correlated to speech performance (Henry and Turner, 2003).
Another important area of psychophysical performance is in quantifying the bilateral cochlear-implant user's binaural processing capability. Preliminary data suggested that people with a bilateral implant can use the interaural level difference but less effectively than the interaural time difference, requiring matched pitch between the contralaterally and ipsilaterally stimulated electrodes (van Hoesel and Clark, 1997; Lawson et al., 1998; van Hoesel et al., 2002; Long et al., 2003).
Speech Performance
As the most important sound for human communication, speech has been systematically studied at all levels, from motor control and production to acoustical and perceptual analysis. As a matter of fact, the “brain” in the cochlear implant has been called a speech processor rather than a sound processor.
As the earlier section on speech processor design has indicated, speech perception can be performed using either primarily spectral or temporal cues. Earlier effort focused on extraction of spectral cues, but was less successful because of the significantly degraded spectral processing capability in the current cochlear-implant users (see the section on psychophysical performance). Recent effort has taken advantage of the superior temporal processing capability in electric stimulation and focused on temporal feature extraction and encoding, particularly the encoding of the temporal envelope and fine structure.
Hilbert Envelope and Fine Structure
Let us begin with a formal mathematical definition of the temporal envelope and fine structure and then proceed with examples to illustrate their meanings. The formal definition is based on the Hilbert transform that (different from the Fourier transform that separates a complex signal into low- and high-frequency components) divides a signal into a slowly varying and a rapidly varying component. For a real signal sr(t), an analytic signal s(t) can be generated by the following equation:
where i is the imaginary number (i.e., square root of −1) and si(t) is the Hilbert transform of sr(t).
The Hilbert envelope is the magnitude of the analytic signal:
The Hilbert fine structure is cosφ;(t), where φ(t) is the phase of the analytic signal:
The instantaneous frequency of the Hilbert fine structure is:
The original real signal sr(t) can be recovered by simply performing:
To provide some intuition, let us examine what the Hilbert envelope and fine structure are for a sinusoidal signal [sr(t) = Acos(2πft)]. Its Hilbert transform is orthogonal to the original signal, [si(t) = A sin(2πft)]. Performing simple trigonometry, we can show that a sinusoid's Hilbert envelope is simply the amplitude (A) and its fine structure is simply the cosine waveform [cos(2πft)] with a unit amplitude and an instantaneous frequency (f).
Figure 12 shows an illustrative example of decomposition of amplitude and frequency modulations. The top panel is a complex signal containing both amplitude frequency and modulations. The middle panel shows the amplitude modulation, and the bottom panel shows the frequency modulation. This example shows that the Hilbert envelope varies relatively slower than the Hilbert fine structure.
Temporal and Spectral Processing
Unfortunately, temporal envelope and fine structure are usually loosely defined and often cause confusion in the literature. One confusion is between acoustics and perception, stemming from a functional definition of the temporal cues and its apparent perception via mixed temporal and spectral mechanisms in normal-hearing listeners (Rosen, 1992). Rosen defined three types of temporal cues in speech based on the rate of amplitude fluctuations: (1) envelope (2–50 Hz), (2) periodicity (50–500 Hz), and fine structure (500–10,000 Hz). The envelope cue is likely perceived via a temporal mechanism and the fine structure cue is most likely perceived via a spectral mechanism, whereas the periodicity cue is possibly perceived via both.
Another source of confusion is related to engineering implementation in extracting and encoding temporal cues. Typically in a cochlear implant, the temporal envelope is extracted by passing a stimulus through a nonlinear device such as a half- or full-wave rectifier and then a low-pass filter. Depending on the cutoff frequency of the low-pass filter, the envelope, the periodicity, or even the fine structure cue can be present at the output of the envelope extractor. The confusion arises with the misperception that all we need to do is to increase the cutoff frequency to encode the periodicity and fine structure information. In addition to the perceptual limit of amplitude fluctuations via the purely temporal mechanisms, the higher cutoff frequencies violate a basic rule in demodulation, as distortion products from the nonlinear process will be leaked through the low-pass filter to produce a confounded waveform.
A third confusion stems from the ignorance of the ear's ability to recover the narrow-band envelope from the broad-band fine structure. A famous example is the high intelligibility of the amplitude-infinitely-clipped speech, which has removed the broad-band amplitude cue but preserved the fine-structure cue via zero crossings (Licklider and Pollack, 1948). Another example is the auditory chimera study, again showing high intelligibility of a chimerized sound with speech fine structure and a flat noise envelope (Smith et al., 2002). Recent studies have shown that the original broad-band fine structure with a flat envelope actually contained the original speech narrow-band envelopes that can be recovered by the auditory filters and contribute to the observed high intelligibility (Zeng et al., 2004a).
Speech Recognition
Speech recognition has been systematically measured as a function of a number of spectral bands in normal-hearing subjects and a function of the number of electrodes in cochlear-implant subjects (Shannon et al., 1995; Dorman et al., 1997; Fishman et al., 1997; Friesen et al., 1999). With high-context speech materials, temporal envelope cues from as few as 3 spectral bands are sufficient to support nearly perfect speech intelligibility. Most strikingly, even with as many as 22 intracochlear electrodes, the cochlear-implant users performed similarly to 4 to 10 electrodes.
Figure 13 presents recent results from our laboratory (Nie et al., presented at the 2003 ARO meeting), showing percent scores of low-context IEEE sentences as a function of the number of spectral bands (x-axis) in normal-hearing subjects in quiet and at a 10-dB signal-to-noise ratio. Different from previous studies, the masker was a competing sentence from another talker. The cochlear implant performance from 9 subjects is included as the horizontal lines with their corresponding vertical lines indicating the equivalent number of spectral bands.
Several important points are noteworthy:
First, for these low-context sentences, as many as 16 bands are needed to reach a performance plateau of about 90% correct in the quiet condition. In noise, the performance was decreased by 15 to 20 percentage points, except for the 1- and 2-band conditions, where the floor effect was present.
Second, in quiet, the cochlear-implant subjects performed relatively well at 70% correct, corresponding to an equivalent level of performance with about 10 spectral bands in normal-hearing subjects. However, in noise, the cochlear-implant performance dropped to less than 10% correct, corresponding to an equivalent level of performance with 4 spectral bands in normal-hearing subjects.
Third, we note that the normal-hearing subjects could achieve nearly perfect performance (about 98% correct) with the original unprocessed sentences under the same 10-dB S/N condition. The performance was only at about 75% correct even with 32 spectral bands, suggesting that the fine structure within each band needs to be encoded for improving the performance in noise.
Bilateral Cochlear Implants and Combined Acoustic and Electric Stimulation
To improve cochlear implant performance in noise, researchers have also recently taken advantage of bilateral cochlear implants, combined acoustic and electric stimulation, and advanced hearing aid technologies. Earlier studies on bilateral cochlear implants often confused the better-ear effect with the true binaural summation and squelch effects.
The better-ear effect is due to the physically improved signal-to-noise ratio as a result of head shadow, while the true binaural effects are physiologically based as a result of complicated neural processing (Hawley et al., 1999). Recent studies on bilateral cochlear implants have largely controlled this better-ear effect and have shown accordingly rather small and nonuniform binaural effects in terms of improvement in sound location and speech intelligibility in noise (Muller et al., 2002; Tyler et al., 2003; van Hoesel and Tyler, 2003). Although commercial motivation for bilateral cochlear implants is strong, the current scientific evidence for binaural advantage that may be derived from the current bilateral implants is not compelling considering the doubling of the already high cost for a single cochlear implant (Summerfield et al., 2002).
Another increasingly active area of research to improve cochlear implant performance is to combine acoustic and electric stimulation by taking advantage of the best of both worlds. The combined stimulation can be achieved by using a conventional cochlear implant on one side and a hearing aid on the other side (Tyler et al., 2002; Ching et al., 2004; Kong et al., 2004b).
Alternatively, the combined stimulation uses a specially designed short-electrode cochlear implant while preserving the low-frequency acoustic hearing in the apex of the same cochlea (von Ilberg et al., 1999; Gantz and Turner, 2003).
Combined acoustic and electric hearing, while providing minimal improvement in speech recognition in quiet, is effective in improving speech recognition in noise, particularly when the noise is a competing voice. The most striking demonstration of this effect is that the low-frequency acoustic stimulation alone (40 dB HL or better <500 Hz) provided essentially zero intelligibility, but when combined with the high-frequency electric stimulation, it could significantly improve speech intelligibility with the electric stimulation alone.
Figure 14 proposes a possible sound segregation and grouping mechanism that explains how the additional low-frequency acoustic cue might improve cochlear implant speech recognition in noise (Kong et al., 2004b). Panel A shows speech temporal envelopes extracted from signal (S) and noise (N), respectively. Panel B shows the combined acoustic stream when both are present. In the absence of any low-frequency cues, a subject has to base on the weak pitch cue in the temporal envelope to differentiate between the signal and the noise, producing two closely spaced perceptual streams, making it difficult for the subject to identify which is signal and which is noise (Panel C). On the other hand, the presence of the salient low-frequency acoustic cue is likely to help the subject segregate and then group appropriately the temporal envelopes between the signal and the noise, producing two distinct perceptual streams (Panel D). Research on the combined acoustic and electric stimulation is in its initial stage, but has promise to overcome the greatest weakness in the current cochlear implants, i.e., speech recognition in noise.
A third area of emerging research is to apply advanced hearing aid technologies, such as directional microphones, noise reduction algorithms, and automatic switches, to serve as preprocessors to cochlear implant speech processors. Chung et al. (2004) measured cochlear-implant users’ ability to listen to speech in noise using materials pre-processed by hearing aid omnidirectional microphones, or directional microphones, or directional microphones plus noise reduction algorithms. They found that directional microphones significantly improved speech understanding in cochlear-implant users, whereas the addition of noise reduction algorithms significantly improved listening comfort. Further research is needed to better integrate hearing aid and cochlear implant technologies to enhance speech understanding and improve listening comfort in cochlear-implant users.
Speaker and Tone Recognition
This section provides two additional sets of data showing the limitation of the current speech processing strategies and additionally, the importance of encoding the temporal fine structure in cochlear implants. The first experiment measured vowel and speaker recognition using the same stimuli in a group of eight cochlear-implant users (Vongphoe and Zeng, 2004). A clear dichotomy was observed in which the implant subjects were able to recognize most vowels (about 70% correct), but not the speakers who produced these vowels (about 20% correct).
The second experiment compared the performance of Mandarin tone recognition as a function of the number of spectral bands (Kong et al., presented at the 2003 ARO meeting) and a function of the number of electrodes in 4 Nucleus native-Mandarin speakers (Wei et al., 1999; Wei et al., 2004). Figure 15 shows that the normal-hearing subjects were able to use the envelope cue effectively, even with 1 band, producing 70% to 80% correct performance (Fu and Zeng, 2000; Xu et al., 2002), whereas the cochlear-implant subjects performed at chance with 1 electrode.
The implant performance peaked at about 70% correct with 10 electrodes and was not improved with an additional number of electrodes. We recently found that the implant performance with tones dropped nearly to the chance level with noise presented at a 5-dB signal-to-noise ratio (Wei, Cao, and Zeng, unpublished data).
Music Performance
As stated before, the cochlear implant has been designed to emphasize speech transmission with little or no attention being paid to other environmental sounds, including music. The consequence of this emphasis is that cochlear-implant users often complain about the poor quality of music perception. Compared with the vast literature in speech perception, research in music perception is relatively rare and mostly descriptive.
Tempo and Rhythm
Cochlear-implant users generally have no or little problem in performing tempo and rhythmic tasks (Gfeller and Lansing, 1991; Kong et al., 2004a). As a matter of fact, they strongly rely on the rhythmic cue to help them perform pitch-related music tasks such as the identification of familiar melodies (Gfeller et al., 1997; Kong et al., 2004a). However, when the rhythmic pattern becomes more complicated, the cochlear-implant users tend to score slightly worse than the normal-hearing subjects (Collins et al., 1994; Gfeller et al., 1997; Kong et al., 2004a), possibly reflecting the reduced capacity in working memory in the cochlear-implant subjects (see section on cognitive performance below).
Pitch, Interval, and Melody
Once the tempo and rhythmic cues are eliminated, the cochlear- implant subjects have tremendous difficulty performing pitch-related tasks. This difficulty occurs for both temporal and spectral pitches, particularly for the latter. Typically, the cochlear-implant subjects can perceive changes in pitch via changes in stimulation rate on a single electrode pair for frequencies up to 300 to 500 Hz (Simmons et al., 1965; Eddington, 1978; Shannon, 1983; Zeng, 2002).
Research has shown that the implant users can use this low-frequency temporal cue to reliably identify pitch interval and melody (Pijl and Schwarz, 1995; McDermott and McKay, 1997; Pijl, 1997). Unfortunately, except for those with the SAS strategy and the obsolete Nucleus WSP processor, current cochlear implants do not encode this low-frequency rate change. The current implant subjects have to extract the pitch information from either the temporal envelope or the spectral pitch associated with electrode position.
Figure 16 highlights the inadequacy of current pitch encoding schemes from the temporal envelope and electrode position cues by contrasting the difference in performance between speech and music perception (Kong et al., 2004a). The six cochlear-implant subjects were considered average-to-star performers with vowel performance from 54% to 80% correct and IEEE sentence performance in quiet from 16% to 92% correct. In contrast, their melody recognition score (in the absence of rhythmic cues) was essentially at the chance level (8%), except for the Clarion SAS user (subject C9).
To improve the current cochlear implant performance in pitch perception, pitch encoding in both temporal and spectral domains needs to be improved. In the temporal domain, the low-frequency pitch may be encoded by dynamically adjusting the stimulating carrier rate, while the boundary of the temporal pitch may be pushed to a higher level by introducing the high-rate or noise conditioner (Zeng et al., 2000; Rubinstein and Hong, 2003).
In the spectral domain, the fitting system has to include better frequency-to-electrode mapping and possibly virtual channels to increase the spectral resolution. The virtual channels can be achieved by stimulating two or more electrodes simultaneously or in fast sequence via pulsatile stimulation. The pitch of the virtual channel is often between that of the two adjacent electrodes when stimulated separately.
Timber and Instruments
Cochlear-implant subjects also have a great deal of difficulty in identifying timbers associated with different musical instruments (Galvin and Zeng, 1998; Gfeller et al., 1998). Figure 17 shows musical instrument identification for both normal hearing and cochlear implant subjects (Galvin and Zeng, 1998). Normal-hearing subjects performed the same task under the original unprocessed stimuli and additionally under conditions to simulate 1-, 2-, 4-, and 8-band cochlear implants.
It was not surprising that the subjects with normal hearing performed significantly better than the subjects with cochlear implants, but what was surprising was that performance was not improved, or slightly decreased if anything at all, as the number of bands was increased from 1 to 8. This pattern of results was very different from the speech result, providing yet another piece of evidence for the perceptual dichotomy between speech and music sounds (Smith et al., 2002).
Cognitive Performance
In addition to peripheral factors such as electrode interaction, nerve survival, and basic psychophysical performance, cognitive factors such as learning, memory, and information processing are likely to contribute to the large individual differences observed in current cochlear-implant users. As it will be discussed, both “bottom-up” and “top-down” approaches are needed to fully account for the overall cochlear implant performance.
Individual Variability
The average cochlear implant performance measured by open-set speech recognition has improved steadily over the last two decades from essentially 0% correct with the early single-electrode devices to about 80% correct with the modern multi-electrode devices. Yet, for all cochlear implants, there is a wide range of performance in cochlear-implant users ranging from those deriving minimal benefit to “star” users whose performance in quiet approaches a normal-hearing listener's level of performance. How to account for this large individual variability remains one of the most challenging questions in current cochlear implant research.
Traditionally, researchers have employed a bottom-up approach trying to identify psychophysical factors that may account for this large individual variability. This attempt has achieved a moderate level of success in terms of correlating psychophysical measures to speech performance. For example, temporal processing measures in detecting amplitude modulation and discriminating different temporal patterns were shown to correlate with speech performance in implant users (Cazals et al., 1994; Collins et al., 1994; Fu, 2002). Similarly in Mandarin-speaking cochlear-implant users, rate discrimination and tone recognition were significantly correlated (Wei et al., 1999).
In the spectral domain, electrode discriminability and spectral resolvability were also found to be correlated with speech performance (Nelson et al., 1995; Henry et al., 2000; Henry and Turner, 2003). However, these correlation studies typically did not address the underlying processing mechanisms.
Bottom-Up Versus Top-Down Approaches
To overcome this shortcoming in traditional correlation studies, Svirsky and colleagues have used an innovative approach to predict the individual speech performance from psychophysical measures at a mechanism level (Svirsky, 2000; Teoh et al., 2003). Based on signal detection theory models, Svirsky et al. measured temporal and spectral discriminability in cochlear implants and constructed a perceptual space to predict multidimensional phoneme identification (MPI). The MPI model clearly takes a bottom-up approach. Because the psychophysical measures were related to speech features in voicing, manner, and place of articulation, Svirsky et al. could predict not only the overall speech performance but also the error patterns the individual users might make.
Alternatively, a top-down approach could be taken to predict speech performance on the basis of frequency of occurrence and acoustic similarity for individual words (Luce and Pisoni, 1998). This neighborhood activation model (NAM) has been used to predict speech performance in both postlingually deafened (Meyer et al., 2003) and pediatric cochlear-implant users (Kirk et al., 1995).
As with the traditional correlation studies, both the bottom-up (MPI) and top-down (NAM) predictions were highly correlated with speech performance. The additional ability to predict quantitatively the performance showed an extremely interesting bias in the result.
Figure 18 summarizes the work of Svirsky et al. and shows schematically the different biases in the predicted performance between the top-down NAM (labeled as lexical model) and the bottom-up MPI (labeled as psychophysical model). The interesting result is that while both measures are highly correlated with speech performance, the psychophysical measure overpredicted speech performance, whereas the lexical measure underpredicted speech performance. In other words, we have probably overestimated the role of basic psychophysical measures but underestimated the brain's power in speech performance. These interesting results demonstrate that we need to take both top-down and bottom-up approaches to fully account for speech performance in cochlear-implant users.
Short-Term Working Memory
To process spoken language, a listener has to effectively use short-term working memory as an interface between the auditory input and the stored linguistic information in long-term memory. Recently, short-term memory has been used to account for the large individual variability in speech perception by children with cochlear implants (Pisoni, 2000; Geers, 2003; Pisoni and Cleary, 2003). One such measure is forward and backward digit span tests, in which a subject repeats a list of digits either in the same order as presented or in a reversed order. The outcome is simply the length of the digits that can be correctly recalled.
On average, children with cochlear implants had a digit span of 5.5 in the forward digit span test and 3.5 in the backward digit span test. The age-matched children with normal hearing had a significantly longer digit span of about 8.0 and 4.5, respectively. Most important, they found that the length of digit span, particularly the forward digit span, was significantly correlated with both communication mode and word recognition.
In addition to the short-term working memory measure, other cognitive measures such as attention, categorization, learning, and memory have been suggested for assessing the central cognitive processing efficiency of the initial sensory input provided by a cochlear implant (Pisoni and Cleary, 2004). This line of research will likely provide new information and possibly new diagnostic tools that were not possible with the traditional psychophysical and electrophysiologic measures. One example will be to conduct visual attention and memory test for temporal sequences that could be used to predict presurgically the postsurgical performance, a challenging task that has been attempted many times but remains unresolved.
Cognitive Rehabilitation
At present, cochlear-implant users do not have access to structured and guided rehabilitation strategies. Instead, they are simply left alone to figure out how to deal with the distorted sensory input provided by the cochlear implant. Clearly, a cochlear-implant user does not hear what a normal-hearing person would hear. The current electrodes are too few and far between while most likely providing upward spectrally shifted auditory input. Some users are able to adapt to the new mode of stimulation more rapidly and efficiently than are others.
Unlike the somewhat controversial acclimatization effect in hearing aids (Arlinger et al., 1996), there is strong evidence for the existence of such an effect in cochlear implants. As a matter of fact, recent research has started asking pointed questions regarding how much can be learned and what is learned via cochlear implants.
An early study showed that normal-hearing subjects listening to cochlear implant simulations were the most susceptible to spectral warping and shift (Shannon et al., 1998). However, normal-hearing listeners could significantly improve their performance with the spectrally distorted speech stimuli with merely nine 20-minute sessions of training, suggesting that the brain is capable of adapting to the distorted input (Rosen et al., 1999).
Another study was conducted in actual cochlear-implant users, who showed continuous improvement in speech performance with frequency-shifted speech over a 3-month period (Fu et al., 2002). More interestingly, Fu et al. found that the cochlear- implant subjects could immediately restore to the same level of performance with the original frequency-to-electrode map, even though they had not listened to the original map for 3 months, suggesting that these subjects had learned two sets of maps in their brain.
A third study found that improved longitudinal performance in cochlear-implant users was mainly due to improved labeling with the distorted stimuli rather than improved psychophysical performance, such as electrode discrimination, suggesting that structured cognitive rehabilitation be introduced to help particularly the less successful cochlear-implant users adapt to and learn the distorted sensory input (Svirsky et al., 2001). How to unlock the brain's power has become an important area of research in cochlear implants.
Trends
That the cochlear implant has matured as a field is evidenced by its exponential growth in the patient population and scientific literature, as well as the breadth and depth—and most importantly—the quality of its research. Long gone is the early argument on whether cochlear implants would work at all: “the elephant is flying.”1 More than 60,000 cochlear-implant users are functioning effectively with their devices.
Different questions have emerged, presenting new challenges and opportunities. Owing to the limited scope of this review, I did not discuss cultural, educational, ethical, social, and economic issues that restrict the widespread application of the cochlear implant, rather I focused on the likely clinical and scientific trends in cochlear implants as an amplification tool to improve hearing in hearing-impaired people. For an expanded discussion on these issues, readers are referred to a recent volume on cochlear implants in the Springer Handbook of Auditory Research (Zeng et al., 2004b).
Clinical Issues
The first clinical issue is presurgical evaluation and prediction of postsurgical performance. This issue has become increasingly important as cochlear implants are being applied to a wider range of candidates, including children and adults as well as special populations such as people with residual hearing and auditory neuropathy. For example, how can we advise a hearing-impaired person with 50% or more intelligibility to either get or not get a cochlear implant if we cannot reliably and accurately predict postsurgical performance? Electrophysiologic, brain imaging, and cognitive measures are likely to provide innovative solutions to address this issue.
The second clinical issue is to improve the current fitting procedure to achieve better efficiency and effectiveness. At present, adjusting the threshold and maximal comfortable loudness on a single electrode stimulation basis is labor intensive and time consuming and also requires high levels of attention and cooperation that are not possible in the pediatric population.
Most of all, the current fitting procedure, although important to maintain safety and to avoid unnecessary side effects such as facial nerve stimulation, has relatively little consequences on speech performance. Electrical and physiologic measures via the built-in telemetry system coupled with expert systems are likely to produce objective and automated fitting procedures in the near future. However, to improve speech performance in noise and music appreciation, extensive and innovative research is needed to produce efficient and accurate frequency-to-electrode mapping in individual cochlear-implant listeners. Recent studies have suggested that pitch of complex stimuli seems to depend on hard-wired neural circuits, further underscoring the importance of research in this area (Zeng, 2002; Oxenham et al., 2004).
The third clinical issue is related to the lack of postsurgical rehabilitation that possibly contributes to the large individual variability in current cochlear-implant users. In addition to learning platforms that are either general purpose or developed for special populations such as Fast Forward (http://www.scientificlearning.com/), tailored, formal, structured, and systematic learning protocols are being developed to help cochlear-implant users adapt to the new modality with electric hearing. Perceptual learning, cross-modality training, and other cognitive rehabilitation should also contribute to the overall cochlear implant performance, including language development and music appreciation.
Next-Generation Cochlear Implants
Although they are not getting any cheaper, cochlear implants have been and will continue to get better and smaller. Rapid advances in technology are poised to make significant changes in the next-generation cochlear implant that will improve the overall functional and physical performance. New signal processing strategies, coupled with hearing aid technology, bilateral cochlear implants, and combined acoustic and electric stimulation, will be able to extract, encode, and deliver important acoustic features, particularly spectral and temporal fine structure, in speech recognition in noise, music appreciation, sound localization, sound segregation, and auditory object identification (Wilson et al., 2003; Chung et al., 2004; Nie et al., 2004).
Micromachining, and nano- and biomimetic technology will produce innovative interfaces, electrodes, microphones, power sources, and packaging for totally biocompatible and totally implantable cochlear implants as well as other auditory prostheses and drug delivery devices that stimulate other auditory structures and promote nerve survival and even regeneration (McCreery et al., 1998; Rousche and Normann, 1999; Shinohara et al., 2002).
Future Amplification Landscape
Cochlear implants will likely play an increasingly greater role in contributing to a total solution to hearing loss treatment and management. Figure 19 schematically depicts a likely scenario for the future landscape in amplification, in which hearing aids, middle ear implants, and cochlear implants will each have a unique yet complementary role:
Hearing aids will be used to address mostly mild-to-moderate hearing loss, as they are the most effective, although acoustic feedback and cosmetic appearance are of serious concerns (Ching et al., 1998; Hogan and Turner, 1998).
Middle ear implants are likely to occupy a niche market, serving people with moderate-to-severe hearing loss with minimal acoustic feedback problems and moderate cost (Chasin, 2002).
Cochlear implants will serve not only people with severe-to-profound hearing loss but also special populations such as those with auditory neuropathy (Starr et al., 1996) and dead cochlear regions (Moore et al., 2000) whose pure-tone average thresholds may be normal or nearly normal.
Acknowledgments
The preparation of this manuscript was partially supported by a grant from the National Institutes of Health (NIH RO1-DC02267). The author thanks Ackland Jones and Kaibao Nie for producing some of the figures used in the manuscript. The author also thanks King Chung, Ying-Yee Kong, Molly Pitassi, Ayaskanta Rout, and three anonymous reviewers for their helpful comments on an earlier version of the manuscript.
Footnotes
The early cochlear implant was questioned and compared to a flying elephant by some auditory researchers in the field, personal communication with Dr. William House).
References
- Abbas PJ, Brown CJ. Electrically evoked brainstem potentials in cochlear implant patients with multi-electrode stimulation. Hear Res 36: 153–162, 1988 [DOI] [PubMed] [Google Scholar]
- Andreev AM, Gersuni GV, Volokhov AA. On the electrical excitability of the human ear: On the effect of alternating currents on the affected auditory apparatus. J Physiol USSR 18: 250–265, 1935 [Google Scholar]
- Arlinger S, Gatehouse S, Bentler RA, et al. Report of the Eriksholm Workshop on auditory deprivation and acclimatization. Ear Hear 17: 87S–98S, 1996 [DOI] [PubMed] [Google Scholar]
- Bekesy GV. Experiments in Hearing. New York: McGraw-Hill, 1960 [Google Scholar]
- Bierer JA, Middlebrooks JC. Cortical responses to cochlear implant stimulation: Channel interactions. J Assoc Res Otolaryngol 5: 32–48, 2004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bilger RC. Electrical stimulation of the auditory nerve and auditory prostheses: A review of the literature. Ann Otol Rhinol Laryngol Suppl 86: 11–20, 1977a [DOI] [PubMed] [Google Scholar]
- Bilger RC. Psychoacoustic evaluation of present prostheses. Ann Otol Rhinol Laryngol Suppl 86: 92–104, 1977b [PubMed] [Google Scholar]
- Bilger RC, Black FO. Auditory prostheses in perspective. Ann Otol Rhinol Laryngol Suppl 86: 3–10, 1977 [DOI] [PubMed] [Google Scholar]
- Bilger RC, Black FO, Hopkinson NT. Research plan for evaluating subjects presently fitted with implanted auditory prostheses. Ann Otol Rhinol Laryngol Suppl 86: 21–24, 1977a [DOI] [PubMed] [Google Scholar]
- Bilger RC, Hopkinson NT. Hearing performance with the auditory prosthesis. Ann Otol Rhinol Laryngol Suppl 86: 76–91, 1977 [DOI] [PubMed] [Google Scholar]
- Bilger RC, Stenson NR, Payne JL. Subject acceptance of implanted auditory prosthesis. Ann Otol Rhinol Laryngol Suppl 86: 165–176, 1977b [PubMed] [Google Scholar]
- Blamey P, Arndt P, Bergeron F, et al. Factors affecting auditory performance of postlinguistically deaf adults using cochlear implants. Audiol Neurootol 1: 293–306, 1996 [DOI] [PubMed] [Google Scholar]
- Brown CJ. Clinical uses of electrically evoked auditory nerve and brainstem responses. Curr Opin Otolaryngol Head Neck Surg 11: 383–387, 2003 [DOI] [PubMed] [Google Scholar]
- Cazals Y, Pelizzone M, Saudan O, et al. Low-pass filtering in amplitude modulation detection associated with vowel and consonant identification in subjects with cochlear implants. J Acoust Soc Am 96: 2048–2054, 1994 [DOI] [PubMed] [Google Scholar]
- Chasin M. Bone anchored and middle ear implant hearing aids. Trends in Amplification 6 (2): 33–38, 2002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chatterjee M. Modulation masking in cochlear implant listeners: Envelope versus tonotopic components. J Acoust Soc Am 113: 2042–2053, 2003 [DOI] [PubMed] [Google Scholar]
- Cheng AK, Niparko JK. Cost-utility of the cochlear implant in adults: A meta-analysis. Arch Otolaryngol Head Neck Surg 125: 1214–1218, 1999 [DOI] [PubMed] [Google Scholar]
- Cheng AK, Rubin HR, Powe NR, et al. Cost-utility analysis of the cochlear implant in children. Jama 284: 850–856, 2000 [DOI] [PubMed] [Google Scholar]
- Ching TY, Dillon H, Byrne D. Speech recognition of hearing-impaired listeners: Predictions from audibility and the limited role of high-frequency amplification. J Acoust Soc Am 103: 1128–1140, 1998 [DOI] [PubMed] [Google Scholar]
- Ching TY, Incerti P, Hill M. Binaural benefits for adults who use hearing aids and cochlear implants in opposite ears. Ear Hear 25: 9–21, 2004 [DOI] [PubMed] [Google Scholar]
- Chung K, Waltzman S, Zeng FG. Using hearing aid directional microphones and noise reduction algorithms to enhance cochlear implant performance. Acoust Res Lett Online (in press), 2004
- Clark GM, Tong YC, Dowell RC. Comparison of two cochlear implant speech-processing strategies. Ann Otol Rhinol Laryngol 93: 127–131, 1984 [DOI] [PubMed] [Google Scholar]
- Collins LM, Wakefield GH, Feinman GR. Temporal pattern discrimination and speech recognition under electrical stimulation. J Acoust Soc Am 96: 2731–2737, 1994 [DOI] [PubMed] [Google Scholar]
- Collins LM, Zwolan TA, Wakefield GH. Comparison of electrode discrimination, pitch ranking, and pitch scaling data in postlingually deafened adult cochlear implant subjects. J Acoust Soc Am 101: 440–455, 1997 [DOI] [PubMed] [Google Scholar]
- Djourno A, Eyries C. Auditory prosthesis by means of a distant electrical stimulation of the sensory nerve with the use of an indwelt coiling. Presse Med 65: 1417, 1957. [PubMed] [Google Scholar]
- Djourno A, Eyries C, Vallancien B. Electric excitation of the cochlear nerve in man by induction at a distance with the aid of micro-coil included in the fixture. C R Seances Soc Biol Fil 151: 423–425, 1957a [PubMed] [Google Scholar]
- Djourno A, Eyries C, Vallancien P. Preliminary attempts of electrical excitation of the auditory nerve in man, by permanently inserted micro-apparatus. Bull Acad Natl Med 141: 481–483, 1957b [PubMed] [Google Scholar]
- Donaldson GS, Viemeister NF, Nelson DA. Psychometric functions and temporal integration in electric hearing. J Acoust Soc Am 101: 3706–3721, 1997 [DOI] [PubMed] [Google Scholar]
- Dorman MF, Loizou PC, Rainey D. Speech intelligibility as a function of the number of channels of stimulation for signal processors using sine-wave and noise-band outputs. J Acoust Soc Am 102: 2403–2411, 1997 [DOI] [PubMed] [Google Scholar]
- Doyle JH, Doyal JB, Turnbull FM. Electrical stimulation of the eighth cranial nerve. Arch Otolaryngol 80: 388–391, 1964 [DOI] [PubMed] [Google Scholar]
- Dudly H. The vocoder. In: Bell Labs Record, pp 122–126, 1939
- Dynes SB, Delgutte B. Phase-locking of auditory-nerve discharges to sinusoidal electric stimulation of the cochlea. Hear Res 58: 79–90, 1992 [DOI] [PubMed] [Google Scholar]
- Eddington DK, Dobelle WH, Brackmann DE, et al. Auditory prosthesis research with multiple channel intracochlear stimulation in man. Ann Otol Rhinol Laryngol 87: 1–39, 1978 [PubMed] [Google Scholar]
- Fishman KE, Shannon RV, Slattery WH. Speech recognition as a function of the number of electrodes used in the SPEAK cochlear implant speech processor. J Speech Lang Hear Res 40: 1201–1215, 1997 [DOI] [PubMed] [Google Scholar]
- Formby C, Sherlock LP, Ferguson SH. Enhancements of the edges of temporal masking functions by complex patterns of overshoot and undershoot. J Acoust Soc Am 107: 2169–2187, 2000 [DOI] [PubMed] [Google Scholar]
- Friesen LM, Shannon RV, Slattery WH. The effect of frequency allocation on phoneme recognition with the nucleus 22 cochlear implant. Am J Otol 20: 729–734, 1999 [PubMed] [Google Scholar]
- Frijns JH, de Snoo SL, ten Kate JH. Spatial selectivity in a rotationally symmetric model of the electrically stimulated cochlea. Hear Res 95: 33–48, 1996 [DOI] [PubMed] [Google Scholar]
- Fu QJ. Temporal processing and speech recognition in cochlear-implant users. Neuroreport 13: 1635–1639, 2002 [DOI] [PubMed] [Google Scholar]
- Fu QJ, Shannon RV. Effects of amplitude nonlinearity on phoneme recognition by cochlear-implant users and normal-hearing listeners. J Acoust Soc Am 104: 2570–2577, 1998 [DOI] [PubMed] [Google Scholar]
- Fu QJ, Shannon RV, Galvin JJ. Perceptual learning following changes in the frequency-to-electrode assignment with the Nucleus-22 cochlear implant. J Acoust Soc Am 112: 1664–1674, 2002 [DOI] [PubMed] [Google Scholar]
- Fu QJ, Zeng FG. Identification of temporal envelope cues in Chinese tone recognition. Asia Pacific J Speech Lang Hear 5: 45–57, 2000 [Google Scholar]
- Galvin J, Zeng FG. Musical instrument perception in cochlear implant listeners. Proc of 16th Inter Cong Acoust and 135th Meet Acoust Soc Am 3: 2219–2220, 1998 [Google Scholar]
- Gantz BJ, Turner CW. Combining acoustic and electrical hearing. Laryngoscope 113: 1726–1730, 2003 [DOI] [PubMed] [Google Scholar]
- Geers AE. Predictors of reading skill development in children with early cochlear implantation. Ear Hear 24: 59S–68S, 2003 [DOI] [PubMed] [Google Scholar]
- Gfeller K, Knutson JF, Woodworth G, et al. Timbral recognition and appraisal by adult cochlear-implant users and normal-hearing adults. J Am Acad Audiol 9: 1–19, 1998 [PubMed] [Google Scholar]
- Gfeller K, Lansing CR. Melodic, rhythmic, and timbral perception of adult cochlear-implant users. J Speech Hear Res 34: 916–920, 1991 [DOI] [PubMed] [Google Scholar]
- Gfeller K, Woodworth G, Robin DA, et al. Perception of rhythmic and sequential pitch patterns by normally hearing adults and adult cochlear-implant users. Ear Hear 18: 252–260, 1997 [DOI] [PubMed] [Google Scholar]
- Giraud AL, Price CJ, Graham JM, et al. Cross-modal plasticity underpins language recovery after cochlear implantation. Neuron 30: 657–663, 2001 [DOI] [PubMed] [Google Scholar]
- Gomaa NA, Rubinstein JT, Lowder MW, et al. Residual speech perception and cochlear implant performance in postlingually deafened adults. Ear Hear 24: 539–544, 2003 [DOI] [PubMed] [Google Scholar]
- Hawley ML, Litovsky RY, Colburn HS. Speech intelligibility and localization in a multi-source environment. J Acoust Soc Am 105: 3436–3448, 1999 [DOI] [PubMed] [Google Scholar]
- Henry BA, McKay CM, McDermott HJ, et al. The relationship between speech perception and electrode discrimination in cochlear implantees. J Acoust Soc Am 108: 1269–1280, 2000 [DOI] [PubMed] [Google Scholar]
- Henry BA, Turner CW. The resolution of complex spectral patterns by cochlear implant and normal-hearing listeners. J Acoust Soc Am 113: 2861–2873, 2003 [DOI] [PubMed] [Google Scholar]
- Hogan CA, Turner CW. High-frequency audibility: Benefits for hearing-impaired listeners. J Acoust Soc Am 104: 432–441, 1998 [DOI] [PubMed] [Google Scholar]
- Horwitz MR, Pugh JE, Jr., Anderson DJ, et al. Long-term recording of primate cochlear potentials through an inner ear electrode implant. Surg Forum 23: 479–480, 1972 [PubMed] [Google Scholar]
- House WF. Goals of the cochlear implant. Laryngoscope 84: 1883–1887, 1974 [PubMed] [Google Scholar]
- House WF, Urban J. Long term results of electrode implantation and electronic stimulation of the cochlea in man. Ann Otol Rhinol Laryngol 82: 504–517, 1973 [DOI] [PubMed] [Google Scholar]
- Javel E, Shepherd RK. Electrical stimulation of the auditory nerve. III. Response initiation sites and temporal fine structure Hear Res 140: 45–76, 2000 [DOI] [PubMed] [Google Scholar]
- Jolly CN, Spelman FA, Clopton BM. Quadrupolar stimulation for cochlear prostheses: Modeling and experimental data. IEEE Trans Biomed Eng 43: 857–865, 1996 [DOI] [PubMed] [Google Scholar]
- Jones R, Stevens S, Lurie M. Three mechanisms of hearing by electrical stimulation. J Acoust Soc Am 12: 281–290, 1940 [Google Scholar]
- Kirk KI, Pisoni DB, Osberger MJ. Lexical effects on spoken word recognition by pediatric cochlear-implant users. Ear Hear 16: 470–481, 1995 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Klinke R, Kral A, Heid S, et al. Recruitment of the auditory cortex in congenitally deaf cats by long-term cochlear electrostimulation. Science 285: 1729–1733, 1999 [DOI] [PubMed] [Google Scholar]
- Kong YY, Cruz R, Jones JA, et al. Music perception with temporal cues in acoustic and electric hearing. Ear Hear 25: 173–185, 2004a [DOI] [PubMed] [Google Scholar]
- Kong YY, Stickney GS, Zeng FG. Speech and melody recognition in binaurally combined acoustic and electric hearing. J Acoust Soc Am (in revision), 2004b [DOI] [PubMed]
- Kral A, Hartmann R, Tillein J, et al. Hearing after congenital deafness: Central auditory plasticity and sensory deprivation. Cereb Cortex 12: 797–807, 2002 [DOI] [PubMed] [Google Scholar]
- Lawson DT, Wilson BS, Zerbi M, et al. Bilateral cochlear implants controlled by a single speech processor. Am J Otol 19: 758–761, 1998 [PubMed] [Google Scholar]
- Leake PA, Hradek GT, Snyder RL. Chronic electrical stimulation by a cochlear implant promotes survival of spiral ganglion neurons after neonatal deafness. J Comp Neurol 412: 543–562, 1999 [DOI] [PubMed] [Google Scholar]
- Lee DS, Lee JS, Oh SH, et al. Cross-modal plasticity and cochlear implants. Nature 409: 149–150, 2001 [DOI] [PubMed] [Google Scholar]
- Liberman MC. The cochlear frequency map for the cat: Labeling auditory-nerve fibers of known characteristic frequency. J Acoust Soc Am 72: 1441–1449, 1982 [DOI] [PubMed] [Google Scholar]
- Licklider J, Pollack I. Effects of differentiation, integration, and infinite peak clipping on the intelligibility of speech. J Acoust Soc Am 20: 42–51, 1948 [Google Scholar]
- Linthicum FH, Jr., Fayad J, Otto SR, et al. Cochlear implant histopathology. Am J Otol 12: 245–311, 1991 [PubMed] [Google Scholar]
- Litvak L, Delgutte B, Eddington D. Auditory nerve fiber responses to electric stimulation: Modulated and unmodulated pulse trains. J Acoust Soc Am 110: 368–379, 2001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Loizou PC, Stickney G, Mishra L, et al. Comparison of speech processing strategies used in the Clarion implant processor. Ear Hear 24: 12–19, 2003 [DOI] [PubMed] [Google Scholar]
- Long CJ, Eddington DK, Colburn HS, et al. Binaural sensitivity as a function of interaural electrode position with a bilateral cochlear-implant user. J Acoust Soc Am 114: 1565–1574, 2003 [DOI] [PubMed] [Google Scholar]
- Luce PA, Pisoni DB. Recognizing spoken words: The neighborhood activation model. Ear Hear 19: 1–36, 1998 [DOI] [PMC free article] [PubMed] [Google Scholar]
- McCreery DB, Shannon RV, Moore JK, Chatterjee M. Accessing the tonotopic organization of the ventral cochlear nucleus by intranuclear microstimulation. IEEE Trans Rehabil Eng 6: 391–399, 1998 [DOI] [PubMed] [Google Scholar]
- McDermott HJ, McKay CM. Musical pitch perception with electrical stimulation of the cochlea. J Acoust Soc Am 101: 1622–1631, 1997 [DOI] [PubMed] [Google Scholar]
- McDermott HJ, McKay CM, Vandali AE. A new portable sound processor for the University of Melbourne/Nucleus Limited multielectrode cochlear implant. J Acoust Soc Am 91: 3367–3371, 1992 [DOI] [PubMed] [Google Scholar]
- Meyer TA, Frisch SA, Pisoni DB, et al. Modeling open-set spoken word recognition in postlingually deafened adults after cochlear implantation: Some preliminary results with the neighborhood activation model. Otol Neurotol 24: 612–620, 2003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Michelson RP. The results of electrical stimulation of the cochlea in human sensory deafness. Ann Otol Rhinol Laryngol 80: 914–919, 1971 [DOI] [PubMed] [Google Scholar]
- Moog JS, Geers AE. Speech and language acquisition in young children after cochlear implantation. Otolaryngol Clin North Am 32: 1127–1141, 1999 [DOI] [PubMed] [Google Scholar]
- Moore BC, Huss M, Vickers DA, et al. A test for the diagnosis of dead regions in the cochlea. Br J Audiol 34: 205–224, 2000 [DOI] [PubMed] [Google Scholar]
- Moore CM, Vollmer M, Leake PA, et al. The effects of chronic intracochlear electrical stimulation on inferior colliculus spatial representation in adult deafened cats. Hear Res 164: 82–96, 2002 [DOI] [PubMed] [Google Scholar]
- Muller J, Schon F, Helms J. Speech understanding in quiet and noise in bilateral users of the MED-EL COMBI 40/40+ cochlear implant system. Ear Hear 23: 198–206, 2002 [DOI] [PubMed] [Google Scholar]
- Nadol JB, Jr., Shiao JY, Burgess BJ, et al. Histopathology of cochlear implants in humans. Ann Otol Rhinol Laryngol 110: 883–891, 2001 [DOI] [PubMed] [Google Scholar]
- Nelson DA, Schmitz JL, Donaldson GS, et al. Intensity discrimination as a function of stimulus level with electric stimulation. J Acoust Soc Am 100: 2393–2414, 1996 [DOI] [PubMed] [Google Scholar]
- Nelson DA, Van Tasell DJ, Schroder AC, et al. Electrode ranking of “place pitch” and speech recognition in electrical hearing. J Acoust Soc Am 98: 1987–1999, 1995 [DOI] [PubMed] [Google Scholar]
- Nie K, Stickney G, Zeng FG. Encoding fine structure to improve cochlear implant performance in noise. IEEE Trans Biomed Eng (in press), 2004 [DOI] [PubMed]
- Niparko JK, Oviatt DL, Coker NJ, et al. Facial nerve stimulation with cochlear implantation. VA Cooperative Study Group on Cochlear Implantation. Otolaryngol Head Neck Surg 104: 826–830, 1991 [DOI] [PubMed] [Google Scholar]
- Oxenham AJ, Bernstein JG, Penagos H. Correct tonotopic representation is necessary for complex pitch perception. Proc Natl Acad Sci U S A 101: 1421–1425, 2004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Palmer CS, Niparko JK, Wyatt JR, et al. A prospective study of the cost-utility of the multichannel cochlear implant. Arch Otolaryngol Head Neck Surg 125: 1221–1228, 1999 [DOI] [PubMed] [Google Scholar]
- Peterson A, Shallop J, Driscoll C, et al. Outcomes of cochlear implantation in children with auditory neuropathy. J Am Acad Audiol 14: 188–201, 2003 [PubMed] [Google Scholar]
- Pfingst BE, Franck KH, Xu L, et al. Effects of electrode configuration and place of stimulation on speech perception with cochlear prostheses. J Assoc Res Otolaryngol 2: 87–103, 2001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pfingst BE, Morris DJ, Miller AL. Effects of electrode configuration on threshold functions for electrical stimulation of the cochlea. Hear Res 85: 76–84, 1995 [DOI] [PubMed] [Google Scholar]
- Pijl S. Labeling of musical interval size by cochlear implant patients and normally hearing subjects. Ear Hear 18: 364–372, 1997 [DOI] [PubMed] [Google Scholar]
- Pijl S, Schwarz DW. Melody recognition and musical interval perception by deaf subjects stimulated with electrical pulse trains through single cochlear implant electrodes. J Acoust Soc Am 98: 886–895, 1995 [DOI] [PubMed] [Google Scholar]
- Pisoni DB. Cognitive factors and cochlear implants: Some thoughts on perception, learning, and memory in speech perception. Ear Hear 21: 70–78, 2000 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pisoni DB, Cleary M. Measures of working memory span and verbal rehearsal speed in deaf children after cochlear implantation. Ear Hear 24: 106S–120S, 2003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pisoni DB, Cleary M. Learning, memory, and cogntive processes in deaf children following cochlear implantation. In: Zeng FG, Popper AN, Fay RR, ed. Springer Handbook of Auditory Research. New York. Springer, 377–426, 2004
- Ponton CW, Don M, Eggermont JJ, et al. Auditory system plasticity in children after long periods of complete deafness. Neuroreport 8: 61–65, 1996 [DOI] [PubMed] [Google Scholar]
- Reefhuis J, Honein MA, Whitney CG, et al. Risk of bacterial meningitis in children with cochlear implants. N Engl J Med 349: 435–445, 2003 [DOI] [PubMed] [Google Scholar]
- Rosen S. Temporal information in speech: Acoustic, auditory and linguistic aspects. Philos Trans R Soc Lond B Biol Sci 336: 367–373, 1992 [DOI] [PubMed] [Google Scholar]
- Rosen S, Faulkner A, Wilkinson L. Adaptation by normal listeners to upward spectral shifts of speech: Implications for cochlear implants. J Acoust Soc Am 106: 3629–3636, 1999 [DOI] [PubMed] [Google Scholar]
- Rousche PJ, Normann RA. Chronic intracortical microstimulation (ICMS) of cat sensory cortex using the Utah Intracortical Electrode Array. IEEE Trans Rehabil Eng 7: 56–68, 1999 [DOI] [PubMed] [Google Scholar]
- Rubinstein JT, Hong R. Signal coding in cochlear implants: Exploiting stochastic effects of electrical stimulation. Ann Otol Rhinol Laryngol Suppl 191: 14–19, 2003 [DOI] [PubMed] [Google Scholar]
- Shannon RV. Multichannel electrical stimulation of the auditory nerve in man. I. Basic psychophysics. Hear Res 11: 157–189, 1983 [DOI] [PubMed] [Google Scholar]
- Shannon RV. Forward masking in patients with cochlear implants. J Acoust Soc Am 88: 741–744, 1990 [DOI] [PubMed] [Google Scholar]
- Shannon RV. Temporal modulation transfer functions in patients with cochlear implants. J Acoust Soc Am 91: 2156–2164, 1992 [DOI] [PubMed] [Google Scholar]
- Shannon RV, Zeng FG, Kamath V, et al. Speech recognition with primarily temporal cues. Science 270: 303–304, 1995 [DOI] [PubMed] [Google Scholar]
- Shannon RV, Zeng FG, Wygonski J. Speech recognition with altered spectral distribution of envelope cues. J Acoust Soc Am 104: 2467–2476, 1998 [DOI] [PubMed] [Google Scholar]
- Sharma A, Dorman MF, Spahr AJ. A sensitive period for the development of the central auditory system in children with cochlear implants: Implications for age of implantation. Ear Hear 23: 532–539, 2002 [DOI] [PubMed] [Google Scholar]
- Shepherd RK, Hardie NA. Deafness-induced changes in the auditory pathway: implications for cochlear implants. Audiol Neurootol 6: 305–318, 2001 [DOI] [PubMed] [Google Scholar]
- Shinohara T, Bredberg G, Ulfendahl M, et al. Neurotrophic factor intervention restores auditory function in deafened animals. Proc Natl Acad Sci U S A 99: 1657–1660, 2002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Simmons FB, Epley JM, Lummis RC, et al. Auditory nerve: Electrical stimulation in man. Science 148: 104–106, 1965 [DOI] [PubMed] [Google Scholar]
- Skinner MW, Holden LK, Holden TA, et al. Performance of postlinguistically deaf adults with the Wearable Speech Processor (WSP III) and Mini Speech Processor (MSP) of the Nucleus Multi-Electrode Cochlear Implant. Ear Hear 12: 3–22, 1991 [DOI] [PubMed] [Google Scholar]
- Skinner MW, Ketten DR, Holden LK, et al. CT-derived estimation of cochlear morphology and electrode array position in relation to word recognition in Nucleus-22 recipients. J Assoc Res Otolaryngol 3: 332–350, 2002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith ZM, Delgutte B, Oxenham AJ. Chimaeric sounds reveal dichotomies in auditory perception. Nature 416: 87–90, 2002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Snyder RL, Rebscher SJ, Cao KL, et al. Chronic intra-cochlear electrical stimulation in the neonatally deafened cat. I: Expansion of central representation. Hear Res 50: 7–33, 1990 [DOI] [PubMed] [Google Scholar]
- Starr A, Picton TW, Sininger Y, et al. Auditory neuropathy. Brain 119 (Pt 3):741–753, 1996 [DOI] [PubMed]
- Stevens S. On hearing by electrical stimulation. J Acoust Soc Am 8: 191–195, 1937 [Google Scholar]
- Stevens S, Jones R. The mechanism of hearing by electrical stimulation. J Acoust Soc Am 10: 261–269, 1939 [Google Scholar]
- Stoddart RL, Cooper HR. Electrode complications in 100 adults with multichannel cochlear implants. J Laryngol Otol Suppl 24: 18–20, 1999 [PubMed] [Google Scholar]
- Suesserman MF, Spelman FA. Lumped-parameter model for in vivo cochlear stimulation. IEEE Trans Biomed Eng 40: 237–245, 1993 [DOI] [PubMed] [Google Scholar]
- Summerfield AQ, Marshall DH, Barton GR, et al. A costutility scenario analysis of bilateral cochlear implantation. Arch Otolaryngol Head Neck Surg 128: 1255–1262, 2002 [DOI] [PubMed] [Google Scholar]
- Svirsky MA. Mathematical modeling of vowel perception by users of analog multichannel cochlear implants: Temporal and channel-amplitude cues. J Acoust Soc Am 107: 1521–1529, 2000 [DOI] [PubMed] [Google Scholar]
- Svirsky MA, Robbins AM, Kirk KI, et al. Language development in profoundly deaf children with cochlear implants. Psychol Sci 11: 153–158, 2000 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Svirsky MA, Silveira A, Suarez H, et al. Auditory learning and adaptation after cochlear implantation: A preliminary study of discrimination and labeling of vowel sounds by cochlear-implant users. Acta Otolaryngol 121: 262–265, 2001 [DOI] [PubMed] [Google Scholar]
- Teoh SW, Neuburger HS, Svirsky MA. Acoustic and electrical pattern analysis of consonant perceptual cues used by cochlear-implant users. Audiol Neurootol 8: 269–285, 2003 [DOI] [PubMed] [Google Scholar]
- Tyler R, Parkinson AJ, Fryauf-Bertchy H, et al. Speech perception by prelingually deaf children and postlingually deaf adults with cochlear implant. Scand Audiol Suppl 46: 65–71, 1997 [PubMed] [Google Scholar]
- Tyler RS, Dunn CC, Witt SA, et al. Update on bilateral cochlear implantation. Curr Opin Otolaryngol Head Neck Surg 11: 388–393, 2003 [DOI] [PubMed] [Google Scholar]
- Tyler RS, Parkinson AJ, Wilson BS, et al. Patients utilizing a hearing aid and a cochlear implant: Speech perception and localization. Ear Hear 23: 98–105, 2002 [DOI] [PubMed] [Google Scholar]
- van den Honert C, Stypulkowski PH. Single fiber mapping of spatial excitation patterns in the electrically stimulated auditory nerve. Hear Res 29: 195–206, 1987 [DOI] [PubMed] [Google Scholar]
- van Dijk JE, van Olphen AF, Langereis MC, et al. Predictors of cochlear implant performance. Audiology 38: 109–116, 1999 [DOI] [PubMed] [Google Scholar]
- van Hoesel R, Ramsden R, Odriscoll M. Sound-direction identification, interaural time delay discrimination, and speech intelligibility advantages in noise for a bilateral cochlear-implant user. Ear Hear 23: 137–149, 2002 [DOI] [PubMed] [Google Scholar]
- van Hoesel RJ, Clark GM. Psychophysical studies with two binaural cochlear implant subjects. J Acoust Soc Am 102: 495–507, 1997 [DOI] [PubMed] [Google Scholar]
- van Hoesel RJ, Tyler RS. Speech perception, localization, and lateralization with bilateral cochlear implants. J Acoust Soc Am 113: 1617–1630, 2003 [DOI] [PubMed] [Google Scholar]
- Vandali AE. Emphasis of short-duration acoustic speech cues for cochlear-implant users. J Acoust Soc Am 109: 2049–2061, 2001 [DOI] [PubMed] [Google Scholar]
- Vollmer M, Snyder RL, Leake PA, et al. Temporal properties of chronic cochlear electrical stimulation determine temporal resolution of neurons in cat inferior colliculus. J Neurophysiol 82: 2883–2902, 1999 [DOI] [PubMed] [Google Scholar]
- Volta A. On the electricity excited by mere contact of conducting substances of different kinds. Royal Soc Philos Trans 90: 403–431, 1800 [Google Scholar]
- von Ilberg C, Kiefer J, Tillein J, et al. Electric-acoustic stimulation of the auditory system. New technology for severe hearing loss ORL J Otorhinolaryngol Relat Spec 61: 334–340, 1999 [DOI] [PubMed] [Google Scholar]
- Vongphoe M, Zeng FG. Speaker recognition with temporal cues in acoustic and electric hearing. J Acoust Soc Am (Submitted), 2004 [DOI] [PubMed]
- Wei C, Cao K, Wang Z, et al. Rate discrimination and tone recognition in mandarin-speaking cochlear-implant listeners. Chinese J. Otorhinolaryngol. 34: 84–88, 1999 [PubMed] [Google Scholar]
- Wei C, Cao K, Zeng FG. Mandarin tone recognition in cochlear-implant subjects. Hear Res (submitted), 2004 [DOI] [PubMed]
- Wilson BS, Finley CC, Lawson DT, et al. Better speech recognition with cochlear implants. Nature 352: 236–238, 1991a [DOI] [PubMed] [Google Scholar]
- Wilson BS, Lawson DT, Finley CC, et al. Coding strategies for multichannel cochlear prostheses. Am J Otol 12 Suppl:56–61, 1991b [PubMed]
- Wilson BS, Lawson DT, Finley CC, et al. Importance of patient and processor variables in determining outcomes with cochlear implants. J Speech Hear Res 36: 373–379, 1993 [DOI] [PubMed] [Google Scholar]
- Wilson BS, Lawson DT, Muller JM, et al. Cochlear implants: Some likely next steps. Annu Rev Biomed Eng, 2003 [DOI] [PubMed]
- Xu L, Tsai Y, Pfingst BE. Features of stimulation affecting tonal-speech perception: Implications for cochlear prostheses. J Acoust Soc Am 112: 247–258, 2002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zeng FG. Temporal pitch in electric hearing. Hear Res 174: 101–106, 2002 [DOI] [PubMed] [Google Scholar]
- Zeng FG, Fu QJ, Morse R. Human hearing enhanced by noise. Brain Res 869: 251–255, 2000 [DOI] [PubMed] [Google Scholar]
- Zeng FG, Galvin JJ. Amplitude mapping and phoneme recognition in cochlear implant listeners. Ear Hear 20: 60–74, 1999 [DOI] [PubMed] [Google Scholar]
- Zeng FG, Galvin JJ, 3rd, Zhang C. Encoding loudness by electric stimulation of the auditory nerve. Neuroreport 9: 1845–1848, 1998 [DOI] [PubMed] [Google Scholar]
- Zeng FG, Grant G, Niparko J, et al. Speech dynamic range and its effect on cochlear implant performance. J Acoust Soc Am 111: 377–386, 2002 [DOI] [PubMed] [Google Scholar]
- Zeng FG, Nie K, Liu S, et al. On the dichotomy in auditory perception between temporal envelope and fine structure cues. J Acoust Soc Am (in press), 2004a [DOI] [PubMed]
- Zeng FG, Popper AN, Fay RR. Cochlear implants: Auditory prostheses and electric hearing. New York: Springer-Verlag, 2004b [Google Scholar]
- Zeng FG, Shannon RV. Loudness balance between electric and acoustic stimulation. Hear Res 60: 231–235, 1992 [DOI] [PubMed] [Google Scholar]
- Zeng FG, Shannon RV. Loudness-coding mechanisms inferred from electric stimulation of the human auditory system. Science 264: 564–566, 1994 [DOI] [PubMed] [Google Scholar]
- Zeng FG, Shannon RV. Psychophysical laws revealed by electric hearing. Neuroreport 10: 1931–1935, 1999 [DOI] [PubMed] [Google Scholar]