Plasticity in Human Pitch Perception Induced by Tonotopically Mismatched Electro-Acoustic Stimulation

Abstract

Under normal conditions, the acoustic pitch percept of a pure tone is determined mainly by the tonotopic place of the stimulation along the cochlea. Unlike acoustic stimulation, electric stimulation of a cochlear implant (CI) allows for the direct manipulation of the place of stimulation in human subjects. CI sound processors analyze the range of frequencies needed for speech perception and allocate portions of this range to the small number of electrodes distributed in the cochlea. Because the allocation is assigned independently of the original resonant frequency of the basilar membrane associated with the location of each electrode, CI users who have access to residual hearing in either or both ears often have tonotopic mismatches between the acoustic and electric stimulation. Here we demonstrate plasticity of place pitch representations of up to 3 octaves in Hybrid CI users after experience with combined electro-acoustic stimulation. The pitch percept evoked by single CI electrodes, measured relative to acoustic tones presented to the non-implanted ear, changed over time in directions that reduced the electro-acoustic pitch mismatch introduced by the CI programming. This trend was particularly apparent when the allocations of stimulus frequencies to electrodes were changed over time, with pitch changes even reversing direction in some subjects. These findings show that pitch plasticity can occur more rapidly and on a greater scale in the mature auditory system than previously thought possible. Overall, the results suggest that the adult auditory system can impose perceptual order on disordered arrays of inputs.

Introduction

In the normal auditory system, the basilar membrane of the cochlea vibrates differentially to sound frequency along its length, such that maximal amplitudes of vibration occur for low frequency sounds apically and for high frequency sounds basally (von Bekesy, 1960). Hair cells on the basilar membrane transduce these vibrations into electrical impulses in the auditory nerve, creating a spatial map of sound frequencies, or tonotopic map, that is preserved at each processing stage in the auditory system up to the cortex. In such a system, the pitch of a pure acoustic tone is determined mainly by the location of maximal vibration on the basilar membrane. In addition, the phase-locking of the auditory nerve to low-frequency tones or amplitude modulated noise permits a temporal analysis of signal periodicity; however, this cue is not typically transmitted by cochlear implants and will not be further discussed..

In sensorineural hearing loss, the hair cells responsible for transduction are damaged. A cochlear implant (CI), an array of electrodes surgically implanted into the cochlea, bypasses the damaged transduction mechanism in individuals with sensorineural hearing loss by directly stimulating the auditory nerve via an electrical current. By positioning electrodes along the length of the cochlea, the CI provides an approximation of frequency-specific information by utilizing the tonotopic organization of the neural output, the auditory nerve. In this case, electrical stimulation directly controls the cochlear place of stimulation. Further, CI sound processors analyze the sound frequencies needed for speech perception, and divide and allocate these frequencies to the electrodes. For example, a frequency range of 200–8000 Hz might be allocated to 22 CI electrodes. This frequency range will be filtered into the same number of bands as electrodes, and the envelopes extracted from each band will be used to modulate the amplitude of electrical pulse trains delivered to the corresponding electrodes. However, due to anatomical and design limitations, the electrode array is typically implanted to depths ranging from 8–21 mm (Lee et al., 2010), corresponding to cochlear place frequencies of no lower than 500–1500 Hz (Greenwood, 1990). This leads to a tonotopic mismatch between the sound frequencies analyzed by the processor versus the characteristic frequencies of the auditory nerve fibers actually stimulated electrically in the cochlea.

In addition, in the past 10–15 years, CI candidacy criteria have expanded to include those with usable residual hearing in one or both ears. Recently, a new type of CI, the Hybrid short-electrode cochlear implant, was devised for patients with partial hearing loss at the high frequencies only (Gantz and Turner, 2003). For the Hybrid CI, “soft” surgery techniques that minimize cochlear trauma are combined with the use of a shorter electrode array to implant only the base of the cochlea in order to minimize damage to the residual low-frequency hearing arising from the apex of the cochlea. The Hybrid CI is programmed to complement the residual hearing range and provide the missing speech frequencies, again resulting in a tonotopic mismatch between the sound frequencies analyzed by the processor and the electrically stimulated auditory nerve characteristic frequencies (Fig. 1A). In addition, because of the presence of residual acoustic hearing in both ears, a mismatch is also introduced between the hair cells and nerve fibers stimulated acoustically and the nerve fibers stimulated electrically. For example, a 1063 Hz tone will acoustically stimulate surviving hair cells and evoke auditory nerve fiber responses in both ears at the cochlear location corresponding tonotopically to 1063 Hz. At the same time, this tone will be allocated by the cochlear implant processor to electrode 10 in the S10 Hybrid array, and evoke auditory nerve fiber responses in the implanted ear at the cochlear location corresponding to 3688 Hz. This corresponds to an approximately 8–9 mm difference in cochlear location of stimulation for this frequency (Greenwood, 1990). Thus, because of the simultaneous electro-acoustic stimulation, substantial mismatches also arise between the pitches evoked acoustically and electrically from a single sound source.

Figure 1.

Illustration of spectral discrepancies introduced by typical frequency-to-electrode allocations in the S12 10-mm, 10-electrode cochlear implant (A) and the L24 16-mm, 22-electrode cochlear implant (B). The lower axis shows sound frequencies before processing; the upper axis shows the frequencies that are stimulated in the cochlea, with a schematic of the electrode array superimposed. The solid red arrows show the mapping from processed sound frequency to electrically-stimulated cochlear place-frequency; most sound frequencies are delivered to electrodes at a higher cochlear place frequency. The blue dashed arrows show that in contrast, the low-frequency acoustic stimulation in both the implanted and the contralateral ear go to the approximately correct cochlear place, though the amplitude of the input that can be provided is gradually reduced at higher frequencies.

While attempts have been made to match the analyzed frequencies to the electrically stimulated characteristic frequencies, Hybrid CI users tend to prefer and perform as well or better with the broader, mismatched frequency ranges rather than a tonotopically matched frequency range, due to the greater low-frequency information provided (Turner et al., 2008; Simpson et al., 2009). As shown by Shannon and colleagues, CI users do better at speech recognition with a mismatched frequency map that they have become accustomed to, compared to a new frequency map that more closely attempts to match stimulation frequencies to the normal place of stimulation (Fu and Shannon, 1999), suggesting that plasticity and previous experience can strongly influence optimal program parameters for speech recognition with cochlear implants. In this study, we expand the importance of plasticity to the basic sensation of pitch perception.

How does the brain resolve this perceived electro-acoustic spectral mismatch? Recent findings suggest a hypothesis that pitch perception is plastic and can adapt by as much as 3 octaves to reduce the electric-acoustic pitch mismatch after several months of experience with a Hybrid CI (Reiss et al., 2007). However, these pitch changes were measured at long intervals of up to years, so that it was not clear whether the changes occurred over days or months. In addition, the majority of the patients exhibited downward pitch shifts; while consistent with the hypothesis that the pitch changes occurred to reduce mismatch with acoustic analysis frequencies that were also 2–3 octaves below the stimulation frequencies, the findings did not completely rule out alternative possibilities that the changes were instead due to loss of high-frequency nerve fibers or even apical electrode movements into the cochlea. The goal of this study was to determine whether pitch changes were directly linked to electric-acoustic pitch mismatches.

In the current study, electric pitch changes were tracked in Hybrid CI patients over shorter time intervals from immediately after the CI was first activated, to as long as 2 years of experience with the CI. In addition, with this group of patients, because the clinicians made frequent changes to the CI programming, we directly compared the direction of pitch plasticity with changes in the direction of electric-acoustic pitch mismatch over time.

Experimental Procedures

Subjects

These studies were conducted according to the guidelines for the protection of human subjects as set forth by the Institutional Review Board of the University of Iowa, and the methods employed were approved by that board. Eleven adult Hybrid cochlear implant subjects, with ages ranging from 25 to 67 years at the time of implantation, participated in this study. All subjects had at least 1 year of experience with the CI at the conclusion of testing.

Hybrid subjects in this study had one of two different types of Hybrid electrode arrays, varying in electrode number, distance, and total array length, as shown in Figure 1. The shorter 10-electrode S12 Hybrid array is 10 mm long, with electrodes spaced 0.6 mm apart, and the short length allows insertion into just the cochlear base. When potential variability due to angle or cochlear length is accounted for, the Greenwood frequency–place function for the basilar membrane predicts a pitch sensation between 2,800 and 4,700 Hz for the most apical electrode (Greenwood 1990; Stakhovskaya et al. 2007). The remaining electrodes would evoke correspondingly higher pitches. The longer 22-electrode L24 Hybrid array is 16 mm long, with electrodes spaced 0.75 mm apart. According to the Greenwood function at the most apical electrode, this longer array is inserted to approximately the 1,500–1,900 Hz cochlear place. Of the eleven Hybrid subjects, eight had the 10-electrode array (identification numbers starting with S) and three had the 22-electrode array (identification numbers starting with L).

All subjects used an advanced combinatorial encoder (ACE) strategy. The Hybrid subjects’ ages, gender, etiology of hearing loss (HL), duration of high-frequency (HF) severe-profound (S/P) deafness, and ipsilateral and contralateral hearing aid (HA) use, and lower frequency cutoff of the speech processor program are shown in Table 1. The detailed long-term audiograms of the implanted and non-implanted ears obtained closest to the time of the latest test are shown for each subject in Figure 2. With the exception of SE41 who had symmetric hearing loss in both ears, subjects generally had better audiometric thresholds in the contralateral, non-implanted ear (blue x-symbols) than in the implanted ear (red circles). This trend is due to the clinical guideline of implanting the worse ear as well as the occasional loss of residual hearing in the implanted ear as a result of implantation.

Table 1.

Demographic information for Hybrid S12 and L24 subjects. Subject characteristics described include age in years, gender, etiology of hearing loss (HL), duration of high-frequency (HF) severe-profound (S/P) hearing loss in years, ipsilateral and contralateral frequency of hearing aid (HA) use, and the lower frequency allocation cutoff frequency, where “>” symbols indicate changes in allocation over time. The specific electrodes tested and the electrodes that could not be pitch matched (out of range) are also given in the last two columns.

Subject	Age (yrs)	Gender	Etiology of HL	Dur. HF S/P HL (yrs)	Ipsi. HA use	Contra. HA use	MAP low cutoff (Hz)	Elec. w/ 12+ mo. of data	Out of range
SE32	63	F	Unknown	4	y	y	1688>688	10,8	10,8
SE33	63	M	Noise Exposure	9	0–2 wks	y	1200>813>188>813	10,6,2	2
SE34	67	M	Familial	25	y except 0.5–7 mo and 12–24 mo	y	1688>688	10, 8, 6	6
SE36	55	M	Familial	9	y	y except 0.5–3 mo	813	10,8,6
SE37	64	F	Unknown	22	y except 0–2 wks	y	1688>1063	10,8,6
SE38	59	F	Unknown	2	y except 0–3 mo	y except 0–3 mo	1688>563>1688	10,8	10,8
SE39	64	M	Unknown	10	y except 3–12 mo	y except 3 mo	1313>938	10
SE41	42	F	Familial	16	y except 9–12 mo	y	1188	10,8
LH1	25	M	Familial	17	0–2 mo	y	563>188	22,20,18
LH2	55	F	Auto-immune	3	y	y	813>938	22	22
LH3	42	F	Unknown	8	y	y	688>813	22,18

Figure 2.

Audiograms for the implanted ear (red circles) and contralateral, non-implanted ear (blue x-symbols) are shown for each subject. NR indicates that a threshold could not be measured.

Pitch Perception Measurement

Pitch data were collected longitudinally at various time points during implant use, typically at approximately 0, 1, 3, 6, and 12 months post-activation. Some subjects were tested up to as late as 26 months post-activation.

Electric-to-acoustic pitch matches were conducted using a computer to control both electric and acoustic stimulus presentations. Electric stimuli were delivered to the CI using NIC2 cochlear implant research software (Cochlear) via the implant programming interface. Stimulation of each electrode consisted of a pulse train of 25 μsec biphasic pulses presented at 1200 pps with a total duration of 500 msec. The pulse rate of 1200 pps per electrode was selected to reduce the effects of any temporal cues on pitch. The electrode ground was set to monopolar stimulation referencing both the ball and plate electrodes (ball electrode near the electrode array and plate on internal receiving coil, respectively). The level of the electric stimulation for each electrode was set to a “medium loud and comfortable” current level.

Acoustic stimuli were delivered using a Creative Labs EMU 1818M sound card, TDT PA4 digital attenuator, and Sennheiser HD-25 headphones. Acoustic tones were presented to the contralateral ear and set to “medium loud and comfortable” levels. Loudness was balanced across all tone frequencies; only tone frequencies that could be comfortably loudness balanced at medium loud levels were included. Then, each CI electrode was loudness balanced with the acoustic tones to reduce loudness effects on electric-to-acoustic pitch comparisons.

Generally, a two-interval, forced-choice constant-stimulus procedure was used. One interval contained the electric pulse train delivered to a particular electrode in the implant ear, and the other interval contained the acoustic tone delivered to the non-implanted ear, with the order of presentation varied. The electric and acoustic stimuli were each 500 ms in duration and separated by a 500 ms inter-stimulus interval. The patient was instructed to indicate on a touch screen the interval that contained the higher pitched stimulus. Trials were repeated by holding the stimulated electrode constant and varying the acoustic tone frequency in ¼ octave steps in pseudorandom sequence to reduce possible order effects (Reiss et al., 2007). Specifically, due to the difficulty of the electric to acoustic comparison, it has been observed in all subjects that the previous comparison tone influences the response to the subsequent comparison tone, such that the pitch evoked by the electrode is judged to be higher in pitch with a descending sequence than with an ascending sequence (Reiss et al., 2011, 2012). Thus, the sequence is counterbalanced to “average” out these effects, with the first half of the pseudorandom sequence mirrored by the second half. The sequence itself is selected from a subset of a Latin square set of sequences, such that each tone frequency was repeated 6 times. The exact same sequence was used in each run and session. Due to time constraints, pitch matches were conducted for between one to three electrodes and limited to those electrodes with frequency-to-electrode allocations within the loudness-balanced portion of the residual hearing frequency range of the non-implanted ear. The electrodes tested for each subject are listed in Table 1.

Data analysis

The averaged pitch matched responses were used to construct psychometric functions for each cochlear implant electrode that was tested. The range of pitch-matched frequencies was computed as those falling between the 25% and 75% on the psychometric function (refer to Figure 3). For a pitch match result to be considered valid, the psychometric function had to reach 100%, i.e., at least one acoustic tone had to be judged as higher in pitch than the electrode-evoked pitch 100% of the time. In some cases, the electric stimulation produced a pitch sensation too high-pitched for the subject to consistently rank any acoustic tones as always higher in pitch, due to the upper limit of the low-frequency hearing range available for comparison in the non-implanted ear. If this occurred, the psychometric function never reached unity (e.g. green curve in Fig. 3B), and the pitch matches were recorded as “out of range”; these electrodes are noted in Table 1. Statistical significances of differences between pitch matches at different times were evaluated using non-parametric bootstrap estimation of 95% confidence intervals (Efron and Tibshirani, 1993; Wichmann and Hill, 2001).

Figure 3.

Examples of smoothed pitch match psychometric function changes over time for two electrodes in a single subject, SE37. For each psychometric function, the percent of time that the electrode pitch was higher in pitch that the comparison acoustic pitch is plotted versus comparison acoustic pitch frequency. Symbols show the 25, 50, and 75 percent points on the functions, which are a measure of the pitch match range. Different colors and symbols indicate psychometric functions for different time points ranging from hookup (0 months) to 12 months of experience with the CI. A. For electrode 10, the function shifts down in pitch over time for electrode 10 from hookup (red squares) to 12 months (purple circles). B. For electrode 8, the function shifts first upward between hookup (red squares) and 3 months (out of range in green triangles, when raw function does not reach 100%), then changes direction to shift downward at 6 and 12 months (teal diamonds and purple circles). The 50% points of the data shown here are plotted in Figure 5 as circles versus frequency allocations over time, with the 25%–75% points indicated by vertical lines.

Results

Figure 3 shows examples of how electric-to-acoustic pitch match psychometric functions changed over time in a single subject. Clearly, the pitch match center (P50 or 50% point) and range (P25–P75 or 25%–75% points) of the functions shifted in frequency at each time point, and moved progressively over a 400 Hz range for electrode 10 and over a 600 Hz range for electrode 8. The overall pitch changes in relation to the frequency allocation changes are seen more clearly for the same subject in Figures 4A–B, where the pitch match centers are shown as circles and the ranges shown as solid vertical lines.

Figure 4.

Example pitch changes over time in three Hybrid subjects. Pitch match centers are shown as blue circles, the pitch match 25–75% ranges are shown as vertical lines, and the frequency allocations are shown as shaded gray bars. Horizontal red dashed lines indicate the upper frequency limit of the residual acoustic hearing in the contralateral ear, which were generally steady but occasionally lowered over time. A–C. Pitch changes that gradually followed frequency allocation changes for all electrodes in one subject, SE37. In particular, electrode 8 shows pitch changes that change direction with the direction of discrepancy with the frequency allocation (B). D–F. Large pitch changes were observed for the most apical electrode only in another subject, SE33, following a large frequency allocation change of more than 2 octaves (D). Again, reversals of pitch changes were observed for this electrode. G–I. Large pitch changes and changes in bandwidth of pitch percept were observed for another subject, SE36. In particular, electrode 8 showed a shift in pitch match center and the pitch match range has also narrowed to match the width of the frequency allocation.

For some subjects, pitch changes gradually aligned with the stimulus frequencies allocated to each electrode, as shown in Fig. 4A–C for the same subject shown in Fig. 3. For this subject, the pitch evoked by electrode 10 showed gradual, consistent downward changes that followed and aligned with the frequency allocations, plotted in different ways in Figs. 3A and 4A. The changes were statistically significant between 1 month and 3 months, as well as between 3 months and 6 months.

Interestingly, the pitch evoked by electrode 8 showed changes that also changed direction with the direction of discrepancy with the frequency allocation, plotted in Figs. 3B and 4B. Initially, the frequency allocation was higher than the perceived pitch at hookup, and the pitch slowly shifted upward to align with the frequency allocation within the first 3 months, even shifting out of the frequency range of measurement of the non-implanted ear (dashed line in Fig. 4B); initial changes between 0–1 month were not significant, and the out of range data point could not be evaluated for significance. After the frequency allocation was changed to be lower in frequency at 3 months, the pitch evoked by electrode 8 shifted down to follow the frequency allocation changes, and reverted to a lower pitch at 6 and 12 months, which were both significantly different from the results at 1 month, and the changes at 12 months were also significantly different from 6 months.

The pitch evoked by electrode 6, plotted in Fig. 4C, was initially measured to be below the frequency allocation, but at 1 month quickly moved above the frequency range of measurement of the non-implanted ear (indicated by the x-symbols), which was expected because the mapped frequency allocation was above this frequency range of measurement. Note that the x-symbols indicate instances when pitch match tests were conducted but failed to yield pitch matches within the loudness-balanced frequency range of measurement of the non-implanted ear. Even after the frequency allocation was lowered at 3 months, no pitch adaptation was observed; this may have been because the frequency allocation was still near the upper limit of the frequency range of measurement of the non-implanted ear, the frequency allocation remained above the more restricted range of residual hearing in the implanted ear (SE37 in Fig. 2), or more time was needed for the adaptation to occur.

For other subjects, pitch changes were observed for the most apical electrode only. Data from one subject in Fig. 4D shows a large and rapid downward shift in pitch for electrode 10 following a large downward change in frequency allocation of more than 2 octaves to around 250 Hz at 3 months, which was maintained to 6 months. The pitch measured at 7 months, but not 3 months, was significantly different from the initial pitch measured at 2 weeks. After just three months with the new frequency allocation, the frequency allocation was returned to the original allocation, and a slower reversal of this large pitch shift was observed for this electrode; note that the pitch remained at approximately 300 Hz even 1 month after the frequency allocation was restored to around 1000 Hz, but gradually increased at 12 months and more closely approaching the frequency allocation by 24 months. The change became significant after 24 months, compared to 7 months. Similarly for two subjects shown in Figs. 4E and 4F, adaptation was observed to shift from an initially high frequency allocation to a new lower frequency allocation. However, for these three subjects, no adaptation was observed for more basal electrodes. In these cases, as for the example shown in Fig. 4C, the frequency allocations for the more basal electrodes were above or just below the upper limit of the frequency range of measurement in the non-implanted ear (dashed lines in Fig. 4D–F), as well as above the limit of measurable residual hearing in the implanted ear (SE33, SE34, and LH1 in Fig. 2), and thus out of the frequency range of interaction between electric hearing and acoustic hearing in either ear.

Changes in pitch perception were not limited to just the pitch center. Large changes in the pitch match range or bandwidth of pitch percept were also observed in addition to pitch center shifts, especially when bandwidths were large to start with. Data from one subject with especially large pitch match bandwidths are shown in Fig. 4G–I. For this subject, as shown in Fig. 4G, the pitch evoked by electrode 10 had a gradual shift of the pitch center to align with the frequency allocation for that electrode, but the pitch match range remained broad at over 2 octaves wide. In contrast, the pitch evoked by electrode 8 in Fig. 4H exhibited both shifts in pitch center and a narrowing of the pitch match range to approximately the width of the frequency allocation, which was significantly narrower at 12 months compared to 4 months. For electrode 6 in Fig. 4I, the pitch center shifted to be out of range at 12 months, as expected due to the mapped frequency allocations being above the frequency range of measurement in the non-implanted ear and most likely also above the aidable frequency range of the implanted ear (Fig. 2; threshold shifts for SE36 were greater than 100 dB HL above 750 Hz). It should be noted that this same subject reported stimulation through each electrode to be noise-like and buzzy when the CI was first activated, but later at 12 months reported the pitch sensations to be generally more tone-like, consistent with the narrowing of the bandwidth over time for electrode 8, but not electrode 10..

Figure 5 shows population summary data pooled across subjects and electrodes. In Figure 5A, each pitch mismatch was plotted over time as the difference between the 50% point of the electrode pitch match function and the geometric center of the corresponding frequency-to-electrode allocation, in octaves. Specifically, to show how the pitch mismatches changed after each new frequency-to-electrode allocation, the data were plotted at times relative to the time of the latest allocation change, i.e. every time the allocation was changed, the time was reset to 0 for plotting. To focus on changes in adaptation to new frequency-to-electrode allocations, only allocations with more than one time point were included.

Figure 5.

Summary of population of pitch change data for all subjects and electrodes. A. The pitch-allocation mismatch plotted over time re: the latest allocation change, i.e. time was reset to 0 and a new plot added each time the allocation changed. Data points without circles indicate points that were out of range and plotted at the upper limit of the residual hearing. B. Net pitch match changes plotted versus frequency allocation changes in subjects with allocation changes over time. Different colors indicate different subjects, with multiple electrodes shown per subject. The net pitch match changes were significantly correlated with the frequency allocation changes (p<0.001).

Figure 5A shows that the pitch-frequency allocation mismatch converged to or remained at less than 1 octave in size in 16/22 instances after each frequency-to-electrode allocation change (or a new CI). Three of the subjects (SE32, SE38, and LH2) were not shown, because their pitch matches always remained above the upper limit of the measurement range in the non-implanted ear for all electrodes. For two of three of these subjects (SE32 and SE38), the lack of pitch adaptation is consistent with the fact that the frequency-to-electrode allocations were also above the upper limit, preventing any interaction between electric and acoustic hearing. However, for the third subject (LH2), the frequency-to-electrode allocations were below the upper limit and do not explain the lack of pitch adaptation seen for this subject.

In subjects who had multiple frequency-to-electrode allocations, the pitch changes were proportional to and occurred in the same direction as the frequency-to-allocation changes, shown in Figure 5B. The correlation was highly significant (R=0.796, p=0.0007, Pearson two-tailed correlation test). Thus, frequency-to-electrode allocation changes account for 64% of the variance in pitch judgments.

Discussion

The majority of Hybrid short-electrode CI users experienced changes in pitch perception relative to an acoustic reference after several months of experience using the CI. The changes often, but not always, caused the electric pitch to align with the frequency range allocated to that electrode by the processor. Sometimes, the pitch changes reversed direction when the direction of mismatch reversed direction. For example, if the electrode pitch was initially lower than the frequencies allocated to that electrode, the pitch changes would initially be in an upward direction in the direction of reducing the mismatch; however, if the frequency allocation was later changed to be lower than the electrode pitch, the pitch changes would reverse direction to be in a downward direction instead. The strong linkage of pitch changes to allocation changes, and the presence of reversals in pitch changes both lend additional support to the hypothesis that the changes occur to reduce perceived pitch mismatch between acoustic and electric inputs between ears and also possibly within ears; since the pitch changes are correlated with allocation changes over time, it is unlikely that the pitch changes are due to random movements of the electrode in the cochlea. Further, if the pitch changes can be reversed, loss of nerve fibers can also be ruled out as possible explanations for pitch changes. The pitch reversal patterns also rule out changes in electrical stimulation levels or dynamic range, which were stable or increased over time for all subjects, as a cause for pitch perception changes.

The reason for the variability in how much individual subjects adapt to reduce mismatch is not yet clear. Possible explanations for lack of adaptation (or adaptation on a slower time scale) in some subjects include differences in age, frequency of hearing aid use, or adequacy of amplification; older brains may be less plastic and able to adapt, and lack of frequent or sufficient acoustic input may prevent perception of any acoustic-electric mismatch that can drive pitch changes. Within subjects that show adaptation for some electrodes but not others (typically more basal electrodes associated with higher frequency-to-electrode allocations), one likely factor is that the allocations were above or just below the upper limit of the frequency range of measurement in the non-implanted ear, and almost always above the limit of the residual hearing in the implanted ear. Thus, these subjects would have had no acoustic-electric mismatch cue to drive pitch plasticity for those electrodes. Note that the range of frequencies that could be comfortably loudness balanced at a medium loud sensation level in the non-implanted ear is more restricted than the frequency range of measurable audiometric thresholds, and is more likely to reflect the effective frequency range of amplification. High frequencies that are measurable in an audiogram, or even high frequencies that could be tested below the loudness-balanced limit, are not necessarily sufficiently amplified by the subject’s hearing aid, especially if gain is turned down to reduce common issues such as feedback or intolerance of high-frequency amplification; amplification is often ineffective for hearing losses exceeding 80–90 dB HL (Summers, 2004; Turner, 2006). Real-ear measurements that indicate how close hearing aids are to target sensation levels may more accurately indicate the effective frequency range of acoustic input for future evaluation of the role of the acoustic input in pitch plasticity.

Another potential factor for inter-electrode variability is the variation in the pitch mismatch; if the electrode pitch mismatch was very large, more time may be needed for the pitch to adapt, or the mismatch may be too large for plasticity to occur; this may have been the case for LH2 who always had a very low frequency-to-electrode allocation centered at 813 Hz and an initial pitch match that was too high to be matched within the frequency range of measurement, compared to the other subjects who generally started with higher allocations that shifted gradually to lower allocations over time (Table I), or had pitch matches that were already low at the time of implant activation (SE36; Fig. 4G–I). Another possible explanation for inter- and intra-subject variability is that abnormal fusion may occur in some cases, such that sounds of even very different pitches between ears are fused and heard as one sound, and no pitch mismatch is perceived. Some preliminary findings indicate that hearing-impaired individuals do have broader fusion ranges than normal-hearing individuals of up to 1–2 octaves (Hong and Turner, ARO 2009; Reiss, et al., ARO 2013). In such cases, no driving force would exist to reduce mismatches because mismatches are not perceived.

The significant correlation of the pitch changes with the frequency allocation changes support the hypothesis that the pitch changes occur as a result of perceived discrepancies between simultaneously heard electric inputs in the implanted ear and acoustic inputs in either or both ears. The findings suggest a mechanism of perceptual adaptation to tonotopic mismatch based on a temporal correlation of electric and acoustic inputs, rather than a model based on matching a preexisting template of pitch perception. Such a perceptual adaptation model could utilize the neural mechanism of spike-timing dependent plasticity (STDP), the strengthening of weak synaptic inputs that fire simultaneously with other inputs. In other words, correlated temporal fluctuations in the amplitude envelopes of spike activity between inputs, at the time scale of neural integration, can induce plasticity. This model may be especially relevant in cases of strong excitation, such as that induced by electrical stimulation from a cochlear implant or other neural prosthesis.

If the pitch changes seen in these patients reflect changes to tonotopic mapping somewhere in the central auditory system, then tonotopic plasticity occurs on a greater scale in the mature auditory system than previously thought possible. Previous studies have shown tonotopic remapping of auditory cortex, as well as thalamus and inferior colliculus, to occur in adults in response to deafferentation or persistent stimulation of limited frequency regions, with cortical takeover of adjacent regions missing input or receiving less input (Review: Pienkowski and Eggermont, 2011). However, these changes are limited to edge effects of less than 1 octave which are less than the effects of 1–3 octaves observed in this study. However, large scale receptive field shifts have been observed in other sensory modalities; in humans, visual field adaptation to inverting lenses occurs after several days of experience wearing the lenses (Stratton, 1896; Taylor, 1962). Alternatively, because the pitch changes are in one ear (electric) relative to the other (acoustic), perhaps a more analogous study is the realignment of auditory-visual space maps in the adult barn owl; after 10 weeks of active hunting experience with visual prisms, the owls were able to adapt partially and shift their visual space map by as much as 25 μs or nearly 10° relative to the auditory space map (Bergan et al., 2005). However, the owls were not able to adapt completely to the total shift of 17° induced by the prisms.

However, unlike the abovementioned studies of topographic remapping, in this study changes were observed on time scales as short as hours rather than weeks. The time scale of these perceptual changes more closely resemble the rapid changes in auditory-visual alignment observed with a phenomenon called the ventriquolism aftereffect, in which exposure to an auditory-visual mismatch for 20–30 minutes is sufficient to shift the representation of auditory space relative to visual space (Recanzone, 1998). It was proposed that higher cortical centers may be involved in this rapid auditory-visual plasticity, and similar mechanisms may apply to changes in interaural auditory frequency alignment, i.e. frequency map representations of inputs from one ear relative to the other ear, or of inputs from electric relative to acoustic inputs. On the other hand, rapid frequency tuning changes have also been observed in primary auditory cortex in studies with behaviorally trained ferrets (Fritz et al., 2003).

Previously, no relationship of speech perception changes to pitch changes has been observed in CI users; generally, speech perception increased and asymptoted over time independent of pitch changes (Reiss et al 2007). However, what this study shows is that CI users are able to adapt to the tonotopic mismatches introduced by their CI processors, which may explain how CI users are able to adapt over time to tonotopically shifted allocations for speech perception (Rosen et al., 1999; Fu et al., 2005). More importantly, the results of this study have implications for how environment can be controlled to purposely drive plasticity in a desired direction. This has particular application to neural prostheses, in which timing and other parameters can be and perhaps should be chosen carefully to maximize plasticity in beneficial rather than harmful directions.

Highlights.

• Tonotopic mismatches introduced via a cochlear implant lead to pitch plasticity.

• The pitch changes for a cochlear implant electrode are as large as 3–4 octaves.

• Pitch changes occur over time scales ranging from hours to months.

• Pitch changes occur in the direction of reducing perceived pitch mismatch.

Abbreviations

ACE

advanced combinatorial encoder

CI

cochlear implant

HA

hearing aid

HF

high-frequency

HL

hearing loss

S/P

severe/profound

STDP

spike-timing dependent plasticity