Comparison of bimodal and bilateral cochlear implant users on speech recognition with competing talker, music perception, affective prosody discrimination and talker identification

Abstract

Objectives

Despite excellent performance in speech recognition in quiet, most cochlear implant users have great difficulty with speech recognition in noise, music perception, identifying tone of voice, and discriminating different talkers. This may be partly due to the pitch coding in cochlear implant speech processing. Most current speech processing strategies use only the envelope information; the temporal fine structure is discarded. One way to improve electric pitch perception is to utilize residual acoustic hearing via a hearing aid on the non-implanted ear (bimodal hearing). This study aimed to test the hypothesis that bimodal users would perform better than bilateral cochlear implant users on tasks requiring good pitch perception.

Design

Four pitch-related tasks were used:

1. Hearing in Noise Test (HINT) sentences spoken by a male talker with a competing female, male, or child talker.

2. Montreal Battery of Evaluation of Amusia. This is a music test with six subtests examining pitch, rhythm and timing perception, and musical memory.

3. Aprosodia Battery. This has five subtests evaluating aspects of affective prosody and recognition of sarcasm.

4. Talker identification using vowels spoken by ten different talkers (three male, three female, two boys, and two girls).

Bilateral cochlear implant users were chosen as the comparison group. Thirteen bimodal and thirteen bilateral adult cochlear implant users were recruited; all had good speech perception in quiet.

Results

There were no significant differences between the mean scores of the bimodal and bilateral groups on any of the tests, although the bimodal group did perform better than the bilateral group on almost all tests. Performance on the different pitch-related tasks was not correlated, meaning that if a subject performed one task well they would not necessarily perform well on another. The correlation between the bimodal users' hearing threshold levels in the aided ear and their performance on these tasks was weak.

Conclusions

Although the bimodal cochlear implant group performed better than the bilateral group on most parts of the four pitch-related tests, the differences were not statistically significant. The lack of correlation between test results shows that the tasks used are not simply providing a measure of pitch ability. Even if the bimodal users have better pitch perception, the real-world tasks used are reflecting more diverse skills than pitch. This research adds to the existing speech perception, language, and localization studies that show no significant difference between bimodal and bilateral cochlear implant users.

Introduction

Speech perception in quiet with current cochlear implant devices can be very impressive, with many cochlear implant users able to use the telephone. However, there are still situations that are very difficult for cochlear implant users, for example listening with other talkers in the background, music appreciation, understanding tonal languages like Mandarin Chinese, perceiving tone of voice, and identifying different talkers. Although cochlear implant recipients were initially happy to receive even modest gains in speech recognition, expectations are now higher. This is partly because cochlear implant candidacy has expanded, and many people with significant residual hearing are now considering cochlear implantation.

Most current speech processing strategies use fixed-rate pulse trains modulated by low-pass filtered amplitude envelopes extracted from a limited number of frequency bands. The temporal fine structure is discarded. In quiet situations, the envelope is sufficient for speech recognition (Shannon et al. 1995), but the fine structure is required for more challenging tasks that require good pitch perception (Smith et al. 2002; Zeng et al. 2005). A cochlear implant attempts to use place coding for pitch, by filtering the incoming signal into several frequency bands, and mapping the signals onto the appropriate electrodes. However, this coding is crude because the implant typically has less than eight effective channels (Fishman et al. 1997), and there is almost always a mismatch in the allocation of frequency bands to electrodes. Some pitch information can also be derived from temporal pitch cues from modulation of the amplitude envelope, although this cue is only usable up to around 300 Hz (Shannon 1992; Zeng 2002). The voice fundamental frequency (F0) may be perceived in this way. However, several factors may prevent this happening: F0 may not be passed by the envelope-smoothing filter, and the pulse rate may not be high enough to represent modulations at F0. McKay et al (1994) studied cochlear implant users' perception of pitch from amplitude modulation and found that the carrier pulse rate needed to be at least four times the frequency of the modulation for subjects to be able to perform pitch ranking. Even if the processing provides modulation at F0, it may still be difficult for a cochlear implant user to perceive.

Much research is focused on improving the representation of temporal fine structure to cochlear implant users in an attempt to increase performance on tasks that require good pitch perception (some work is summarized in Zeng (2004)). Modified speech processing strategies have had promising results (Arnoldner et al. 2007; Filipo et al. 2008; Nie et al. 2005). However, one easier method is the addition of acoustic hearing. Acoustic hearing can be provided to the ear contralateral to the cochlear implant using a conventional hearing aid; this is termed bimodal hearing. Alternatively, in some patients with preserved low-frequency hearing thresholds, great success has been achieved by implanting a modified electrode a reduced distance into the cochlea and using a hearing aid on this same ear, thus providing electrical stimulation for the high frequencies, but preserving the low frequencies for acoustic stimulation. This is usually described as hybrid stimulation. The use of hybrid devices has shown improved speech recognition in quiet (Gantz & Turner 2004) and noise (Gantz et al. 2005; Kiefer et al. 2005; Turner et al. 2004) and music perception (Gantz et al. 2005; Gfeller et al. 2006).

As recently as five to ten years ago, patients receiving cochlear implants were advised to discontinue use of the contralateral hearing aid because of fears that the two modes of stimulation may conflict. Many studies have now shown bimodal benefit in speech perception in quiet (Dettman et al. 2004; Dorman et al. 2008; Mok et al. 2006; Morera et al. 2005; Potts et al. 2007; Ullauri et al. 2007), noise (Ching et al. 2005; Dorman et al. 2008; Dunn et al. 2005; Holt et al. 2005; Kong et al. 2005; Luntz et al. 2005; Luntz et al. 2003; Luntz et al. 2007; Mok et al. 2007; Mok et al. 2006; Morera et al. 2005; Tyler et al. 2002; Ullauri et al. 2007), music perception (El Fata et al. 2009; Kong et al. 2005), and localization (Ching et al. 2005; Potts et al. 2007; Seeber et al. 2004; Tyler et al. 2002). Most studies have evaluated speech perception or localization; Nittrouer and Chapman (2009) showed that children who have had some period of bimodal stimulation have superior generative language abilities, presumably because the low-frequency speech signals provide prosody and help the perceptual organization of the signal from the cochlear implant. The current recommendation is that all cochlear implant patients should use a contralateral hearing aid unless there is clear evidence to suggest that this will have a negative effect on their performance (Ching et al. 2004a; Firszt et al. 2008). Few subjects have been identified whereby the use of a contralateral hearing aid degraded performance (Dunn et al. 2005; Litovsky et al. 2006; Mok et al. 2006).

In recent years, many cochlear implant users have chosen to receive a second (bilateral) cochlear implant, either at the same surgery as the first implant (simultaneous implantation) or at a later date (sequential implantation). Many studies with bilateral implants have shown improved localization and improved hearing in background noise over unilateral implant use, especially when the sources are spatially separated (for a review of 37 studies see Murphy & O'Donoghue 2007). Speech in noise benefits are predominantly related to the head shadow effect, which is not a true binaural effect; it is a purely physical monaural effect. However, a few studies have shown limited benefit in some subjects from binaural summation (Laszig et al. 2004; Muller et al. 2002; Ramsden et al. 2005; Schleich et al. 2004; Schon et al. 2002) and binaural squelch (Eapen et al. 2009; Laszig et al. 2004; Muller et al. 2002; Schleich et al. 2004; Schon et al. 2002; van Hoesel & Tyler 2003); these are true binaural processing effects.

It is well-documented that bilateral cochlear implants provide benefit over a unilateral device. It is more appropriate to compare bilateral implants to a bimodal situation. Litovsky et al (2006) and Mok et al (2007) compared bimodal and bilateral children in speech perception in noise, and additionally localization in Litovsky et al (2006). Litovsky and colleagues found that the mean results for the tests were not significantly different between the two groups, but there was significantly greater binaural advantage (the improvement gained from wearing two devices compared to one) in the bilateral cochlear implant group. Mok et al (2007) assessed spatial unmasking (the improvement in speech perception when the target speech and background noise are spatially separated compared to when they come from the same direction) and binaural advantage. The results showed that most of the bimodal and bilateral children demonstrated spatial unmasking and binaural advantage. The bilateral group showed a greater head shadow effect, which could be related to the cochlear implant having a directional microphone and to better high-frequency hearing with the cochlear implant than the hearing aid. When Nittrouer and Chapman (2009) compared children's language abilities according to whether they were bimodal, bilateral, or single cochlear implant users, they found no significant difference in any measure. A self-rating questionnaire study in adults, comparing a unilateral implant group with a bimodal and a bilateral group, revealed that the bimodal group rated themselves lowest of the three groups in terms of benefit (Noble et al. 2008). The authors suggested that those subjects who were not doing so well with their cochlear implant continued to use a hearing aid, while those who felt that their cochlear implant performance was effective would discontinue use of the hearing aid.

Schafer et al (2007) used a meta-analytic approach to examine results from sixteen peer-reviewed publications related to speech-recognition in noise at fixed signal-to-noise ratios (SNR) for bimodal or bilateral stimulation. They found that binaural stimulation (either bimodal or bilateral) provided clear benefit over monaural stimulation. There were no significant differences between the bimodal and bilateral groups for any of the binaural phenomena. Because performance in noise was equivalent between the two groups, they recommended that a bimodal arrangement should be attempted first, only proceeding to bilateral implantation if there is no benefit from the contralateral hearing aid use. Ching et al (2007) provided another review of bimodal and bilateral literature. They commented that benefits existed with both conditions, and unequivocally, subjects should wear some device on the second ear. However, the authors concluded that there was currently insufficient evidence to guide decisions on which option was best for an individual.

Comparisons of bimodal and bilateral cochlear implant users have focused on speech perception, localization, and language. The results have been inconclusive with no configuration proving better overall. The current study selected four pitch-related tasks that may favor the bimodal cochlear implant users. The following sections review relevant literature in these four tasks.

Speech recognition with competing talker

Although most clinical tests of speech perception in noise use a steady-state masker, in real life, speech is more often masked by time-varying stimuli, usually other speech. Previous research has shown that additional acoustic information provided a significant benefit over unilateral cochlear implant use when listening with competing talkers, especially when the competing talker was a different gender from the target (Cullington & Zeng 2010; Kong et al. 2005).

Music perception

In a comprehensive review of music perception in cochlear implant users (McDermott 2004), the author suggested that the perception of music is likely to be much better when acoustic and electric stimulation are combined. This has been shown in users of the hybrid cochlear implant (Gantz et al. 2005; Gfeller et al. 2006; Gfeller et al. 2007) and in bimodal cochlear implant users (Buechler et al. 2004; Dorman et al. 2008; El Fata et al. 2009; Kong et al. 2005). Two studies have evaluated music perception for hearing aid users who are audiometrically candidates for cochlear implantation (Looi et al. 2008a, 2008b). Both showed that rhythm perception and musical instrument recognition were equal in hearing aid and cochlear implant users and that pitch ranking was better in the hearing aid users. Contradictory findings were obtained in terms of the subjects' quality ratings of music; the cochlear implant users rated music as sounding more pleasant than did the hearing aid users (Looi et al. 2007). Of course there may be a bias involved in these results because the cochlear implant users may perceive the implant as being a superior device. These results suggest that the use of a hearing aid in conjunction with a cochlear implant would offer the best solution. Very few studies have evaluated music perception in bilateral users. Veekmans et al (2009) used a questionnaire to compare unilateral and bilateral cochlear implant users; they found bilateral users generally reported better music perception than unilateral users. Gfeller et al (2010) found that bilateral implants had a positive impact on the recognition of music with lyrics, while bimodal use was associated with better perception and enjoyment of instrumental music. There are also two anecdotal reports of improvement in music for bilateral users (Falcon et al. 2009; Kuhn-Inacker et al. 2004).

Affective prosody discrimination

Intonation refers to the variation in pitch when speaking. This is one element of prosody (rhythm and stress are others). Affective prosody is a nonlinguistic aspect of language that conveys emotions and attitudes during discourse, such as anger, sadness, surprise, and happiness. Both F0 and duration cues are believed to play an important role in affective prosody, with F0 traditionally considered to be the most important cue.

Green et al (2005) tested nine cochlear implant users on 30 sentences, where subjects were required to choose whether the sentence was spoken as a question or statement. The mean score was approximately 69% correct, where chance is 50%. Performance was significantly improved in both simulation listeners and cochlear implant users with a modified processing strategy that enhanced temporal cues to voice pitch. Peng et al (2008) also assessed perception of question versus statement in 26 children and young people using a cochlear implant and 17 age-matched normal-hearing controls. An average score of 97% correct indicated that the normal-hearing participants completed the task with ease, while the subjects with cochlear implants scored 70%.

Luo et al (2007) assessed vocal emotion recognition in eight normal-hearing listeners and eight cochlear implant recipients, with sentences spoken by a male and a female talker using five emotions: angry, anxious, happy, sad, and neutral. The mean result were 89.8% for normal-hearing listeners and 44.9% for cochlear implant users.

Talker identification

The ability to discern if an adult talker is male or female is mostly dependent on voice pitch (Fu et al. 2004), with males having a lower fundamental frequency due to having larger anatomical structures, such as vocal folds and vocal tract. A typical male F0 is 120 Hz; F0 of female talkers is usually around one octave higher. Other vocal properties are important in children. Perry et al (2001) found that up to age 12 years, F0 was very similar for young boys and girls. Listeners could still determine gender reliably using the first, second, and third formant frequencies, the average of which was consistently lower for boys than girls.

Fu et al (2005) reported that cochlear implant users typically can perform voice gender discrimination well (mean score of 94% correct) when the F0 of male and female voices was very different. This suggests that temporal cues can be used for widely disparate F0. When there was overlap in fundamental frequency between the males and females, the gender discrimination deteriorated to 68% correct, presumably due to reduced spectral resolution.

In addition to the gross male or female categorization, it is important for listeners to be able to discriminate different voices within the genders. Cleary and Pisoni (2002) examined 44 children with cochlear implants on a task where they had to identify two sentences spoken by a female as same or different talker. The children were essentially unable to do the task (mean score of 57% correct, where chance is 50%) if the linguistic content of the two sentences was different. Normal-hearing controls scored 89%. Preliminary adult data also suggested that talker discrimination was more difficult when the linguistic content of the stimuli varied (Kirk et al. 2002; McDonald et al. 2003). Vongphoe and Zeng (2005) evaluated ten cochlear implant users on vowel tokens spoken by three men, three women, two boys, and two girls. They found that mean talker identification was 23% (chance was 10%). Because a contralateral hearing aid is assumed to provide low-frequency fine structure, it is expected that bimodal users will have better access to F0. However, Dorman et al (2008) were surprised to find that talker identification was no different in bimodal cochlear implant users tested in the three conditions: acoustic alone, cochlear implant alone, and bimodal.

Current study

The present study tested the hypothesis that bimodal cochlear implant users would perform better on tasks requiring good pitch perception due to the better spectral resolution at low frequencies provided by the residual hearing. The null hypothesis was that there would be no difference between the bimodal and bilateral groups on the four tasks.

Materials and Methods

Participants

Twenty-six post-lingually deafened adult cochlear implant users (aged 18 years and above) took part in the study. Thirteen of these wore a contralateral hearing aid (bimodal); thirteen had bilateral cochlear implants. Prior to commencing the study, sample size was estimated using standard deviation values from previous studies. Effect sizes that would be clinically significant were estimated for each of the four measures. The probability of a Type I error was set at 0.05 (with Bonferroni correction where appropriate); the probability of a Type II error was set at 0.2. These calculations suggested sample sizes ranging from 4 to 19 for the many subtests. Thirteen in each group was chosen as it would provide 80% power for the majority of measures, and slightly lower power for the others. This was considered a manageable number of subjects to obtain within the duration of the study: there were relatively few bilateral adult cochlear implant users at this time.

All subjects except one were native English speakers. One bilateral subject (bilat6) spoke Chinese in childhood, but the subject stated that English is now their primary language. Twelve bimodal and eight bilateral subjects used American English and were tested in the United States at various centers. One bimodal and five bilateral subjects used British English and were tested at the University of Southampton in the United Kingdom. One subject was blind. Subjects were required to sign an informed consent to participate in the study; University of California Irvine Institutional Review Board and National Health Service Research Ethics Committee approval were obtained. Subjects were compensated for their travel expenses; those tested in the US additionally received $10 per hour for participation.

All testing occurred in the subject's usual listening mode (i.e., bimodal or bilateral). No individual ear testing was performed due to concerns that scores may be depressed when subjects listen in modes to which they are unaccustomed. Subjects using any commercially available cochlear implant device were eligible to participate. Bimodal subjects were questioned on the use of their hearing aid, and only those who used it at least 70% of the time were included. Both sequential and simultaneous bilateral recipients were included. None of the subjects used an electroacoustic (hybrid) device; they were all users of standard cochlear implant devices.

All subjects had Hearing in Noise Test (HINT) (Nilsson et al. 1994) word scores in quiet greater than or equal to 65%. However, adaptive HINT in noise was only performed on subjects scoring 75% or more in quiet because adaptive testing in noise is not appropriate unless the subject is scoring highly for testing in quiet. Adaptive HINT results were only available on 11 bimodal and 12 bilateral implant users.

The mean age of the bimodal group was 63 years (range 42 to 87 years); the mean age of the bilateral group was 56 years (38 to 75 years). This difference was not statistically significant (F(1,24) = 2.0, p = 0.170). The subjects were questioned on their duration of profound deafness before receiving their cochlear implant (their first implant if they were implanted bilaterally). In cases of progressive hearing loss this was often difficult to ascertain. The mean duration of profound deafness was 16.5 years for the bimodal group and 7.5 years for the bilateral group. The difference was not statistically significant (F(1,24) = 2.7, p = 0.112). One significant difference between the groups was the length of cochlear implant use. In the case of those using bilateral implants, this was measured from the initial stimulation of their first implant to the time of testing. The mean device use was 2.6 and 7.2 years for the bimodal and bilateral groups respectively (F(1,24) = 15.5, p = 0.001). All subjects had been using their current configuration (bimodal or bilateral) for at least three months. The mean bilateral use was 3.5 years; this was not significantly different from the mean bimodal use of 2.6 years (F(1,24) = 0.889, p = 0.355). Demographic data for the subjects is shown in Table 1. On average, the speech perception performance of the two groups was well matched in quiet and in noise. Mean values for HINT word score in quiet were 90% for the bimodal group and 91% for the bilateral group. Mean values for the speech reception threshold (SRT) with a steady-state speech-shaped noise masker were 2.9 dB for the bimodal group and 2.1 dB for the bilateral group. The SRT is the SNR at which the subject scores 50% correct.

Table 1.

Demographic characteristics of subjects. Strategy information was not available for subject bimod2. In the contralateral implant column, the notation ‘simult’ indicates that the bilateral cochlear implantation occurred simultaneously. All other bilateral implantations were sequential.

Subject	Age	Sex	Etiology	Implant	Processor	Strategy	Contralateral implant	Contralateral hearing aid or processor	Contralateral ear strategy	Duration profound deafness (years)	Experience with mode (bimod/bilat) (years)
bimod1	68	f	unknown progressive	Nucleus 24	Esprit 3G	ACE		Widex B32		1.0	1.7
bimod2	70	m	unknown progressive	Freedom	Freedom	.		Oticon Syncro		7.0	1.0
bimod3	53	f	unknown	Clarion 2	Auria	HiResP		Widex Senso C18+		14.0	1.5
bimod4	73	f	autoimmune disease	Freedom	Freedom	ACE		Resound		6.5	1.3
bimod5	66	f	noise exposure, progressive	Nucleus 24	Freedom	ACE		Unitron US80 PPL		6.0	2.3
bimod6	87	f	unknown progressive	Freedom	Freedom	ACE		Widex Senso C19		1.0	1.0
bimod7	81	m	unknown	Nucleus 24	Freedom	CIS		Widex Senso C19		12.0	4.1
bimod8	73	m	noise exposure	Freedom	Freedom	ACE		Siemens Acuris		3.0	2.9
bimod9	45	m	meningitis/streptomycin	HiRes 120	Harmony	HiRes 120		Widex Senso P38		42.0	1.3
bimod11	50	m	unknown congenital progressive	Freedom	Freedom	ACE		Unitron US80-PP		50.0	0.8
bimod12	60	f	hereditary	Nucleus 24	Sprint	ACE		Siemens		47.0	5.0
bimod13	42	f	? childhood mumps progressive	HiRes 90k	Harmony	HiRes 120		Siemens Prisma 2Pro DSP		9.5	0.4
bimod14	54	f	unknown progressive	Clarion 1	S series	MPS		Siemens Artis 2SP		15.0	10.1
bilat1	45	m	unknown progressive	Clarion 1	PSP	CIS	Clarion 2	Auria	HiRes S	17.0	2.3
bilat2	57	m	unknown progressive	Nucleus 24	Esprit 3G	ACE	Freedom	Freedom	ACE	0.8	0.8
bilat3	68	f	otosclerosis	Nucleus 22	Esprit 3G	SPEAK	N24C	Freedom	ACE	21.0	5.7
bilat4	52	f	unknown progressive	Med-El	Tempo+	CIS	Med-El	Tempo+	CIS	1.0	3.5
bilat5	75	m	hereditary progressive	Nucleus 24	Freedom	SPEAK	Nucleus 24	Freedom	SPEAK	7.0	5.6
bilat6	48	f	unknown progressive	Nucleus 22	Esprit 3G	SPEAK	Freedom	Freedom	SPEAK	13.0	1.7
bilat7	38	f	congenital rubella, progressive L ear	HiRes 90k	Harmony	HiRes 120	HiRes90k	Harmony	HiRes 120	1.6	0.5
bilat8	75	f	unknown progressive	Nucleus 24	Freedom	SPEAK	Nucleus 24	Freedom	SPEAK	1.0	6.6
bilat9	56	f	meningitis	Nucleus 24	Esprit 3G	ACE	Nucleus 24 (simult)	Esprit 3G	ACE	0.3	3.6
bilat10	60	m	Menieres	Med-El C40+	Tempo+	CIS	Med-El C40+	Tempo+	CIS	5.0	4.0
bilat11	41	m	unknown progressive	Med-El	Tempo+	CIS	Med-El (simult)	Tempo+	CIS	2.0	4.9
bilat12	68	m	unknown progressive	Nucleus 24	Esprit 3G	ACE	Nucleus 24 (simult)	Esprit 3G	ACE	4.0	5.7
bilat13	43	f	R: ototox, L: viral infection	Clarion 1	Platinum BTE	MPS	HiRes90k	Harmony	MPS	24.0	0.3

Test set-up

During initial test explanation, the subject was allowed to adjust the sensitivity or volume of their hearing aid or speech processor if required. Thereafter, no further adjustments were permitted. Testing took place in a sound-treated audiometric booth, with the subject seated approximately 1m from a loudspeaker placed at 0° azimuth. All tests were presented in the sound field; calibration ensured all stimuli were presented at a root mean square (rms) level of 60 dB(A). The test computer was either a Mac Power Book G4 or a Dell Dimension 8100, running Windows 2000. No repetition of any test stimuli was permitted. The operator was also inside the booth, sitting at a computer terminal, scoring the subject's responses and running the test. Testing took three to four hours (including training phases), with adequate breaks. Some subjects chose to complete the testing over two separate sessions. In order to avoid possible validity threats due to order effects, test order was balanced across subjects using digram balanced Latin squares.

Speech recognition with competing talker

The target material consisted of sentences drawn from the HINT database, spoken by a male talker with an average F0 of 109 Hz. A loose keyword scoring method was adopted, as described in Cullington and Zeng (2008). Test material was digitized with a sampling rate of 44.1 kHz and comprised mono 16-bit resolution wav files.

The female and male maskers were obtained from the Institute of Electrical and Electronics Engineers (IEEE) sentence material (IEEE 1969) (used with permission from the Sensory Communication Group of the Research Laboratory of Electronics at MIT). Each spoke 40 different sentences. The IEEE sentences are typically longer and use more complex language than the HINT sentences. The female and male maskers had an average F0 of 214 and 101 Hz respectively. The child masker was obtained from the Carnegie Mellon University (CMU) Kids Corpus (Eskenazi 1996; Eskenazi & Mostow 1997); this is a large database of sentences read aloud by children. The child used was a nine-year-old female; she spoke thirteen different sentences. The average F0 of the child masker was 246 Hz. The masker sentences had greater duration than the longest HINT sentence, ensuring that no part of the target sentence would be presented in quiet. All sentence material (including target HINT sentences) was edited digitally so that there were minimal silent periods at the start and end of each sentence. A steady-state noise masker was used in order to facilitate comparison with clinical testing; the steady-state noise was a three-second sample spectrally-matched to the average long-term spectrum of the HINT sentences (Nilsson et al. 1994), ensuring that, on average, the SNR was approximately equal at all frequencies. This stimulus was also used for calibration.

A MATLAB^® (The MathWorks, Inc., Natick, Massachusetts) program, developed by the first author, was used to present and score the sentences. Testing was done in quiet or in noise, with a choice of masker. The target and masker were added digitally. Testing began with two practice lists (lists 12 and 13) presented in quiet at a rms level of 60 dB(A). Two additional lists were then scored in quiet (lists 14 and 15) and in steady-state noise (lists 16 and 17). Testing then occurred with the female, male, and child maskers (using lists 18 and 19, 20 and 21, and 22 and 23 according to a Latin square design). Higher numbered lists were chosen in the hope that subjects who had previously been evaluated on HINT may be less familiar with the later lists. For those subjects who used British English, the list allocation was changed slightly. The talker maskers were evaluated using lists 7 and 8, 11 and 3, and 22 and 19. These lists were chosen as they contained no words that would be unfamiliar to British people, for example ‘faucet’. The British English listeners were also given lists 1 and 2 as extra practice in quiet to allow them time to acclimatize to the American accent. The rms level of the target remained at 60 dB(A) throughout the testing; the masker intensity was adjusted to create the appropriate SNR. Using a fixed target level avoids presenting target stimuli at intensities where compression occurs; the cochlear implant device has a limited input dynamic range (Stickney et al. 2004; Zeng et al. 2002).

A one-up, one-down adaptive procedure was used to estimate the subject's SRT. The initial SNR was +5 dB. This procedure, first described by Levitt and Rabiner (1967), is commonly used to ensure observations are concentrated in the region of interest. Initially, the same target sentence was presented repeatedly and the SNR was increased by 8 dB until the subject correctly repeated the sentence; this allowed the program to quickly find the approximate SRT. Once this occurred, the step size was reduced to 4 dB, and the adaptive procedure began, with the SNR decreasing by 4 dB when the subject answered correctly and increasing by 4 dB when the response was erroneous. The SRT (in dB) was calculated as the mean of the last six reversals. Although the usual HINT step size is 2 dB, it was found that with only 20 sentences presented, cochlear implant users would produce insufficient reversals with this step size. The masker segment was demonstrated to the subject at the beginning of each condition; they were told to ignore this voice and listen only to the target male talker. The masker sentence began approximately 0.45 s before the target. An onset difference between masker and target has been used by other authors (Drullman & Bronkhorst 2004; Festen & Plomp 1990; Freyman et al. 2004; Wagener & Brand 2005); it provides a basis for attending to the target, although in this experiment the subjects were not instructed as such. Both target and masker were presented from the same speaker. This study did not aim to exploit the binaural effects squelch and head shadow, which require the target and masker to be spatially separate. The test took about 20 minutes to complete. The dependent variables were SRT with a female, male, and child masker.

Music perception

The Montreal Battery of Evaluation of Amusia (MBEA) is a standardized test of music abilities; it is sensitive, normally-distributed, and reliable on test-retest (Peretz et al. 2003). It has previously been validated on 12 cochlear implant users (Cooper et al. 2008). The MBEA comprises six subtests designed to explore musical perception related to pitch, rhythm, and memory. The six subtests are entitled Scale, Contour, Interval, Rhythm, Meter, and Memory. The advantages of using the MBEA over a simple melody identification task are that it is multidimensional, theoretically driven, extensive normative data are available, and it does not rely on cultural or linguistic knowledge. The tests use a pool of 30 novel musical phrases with durations of 3.8 to 6.4 seconds. The frequency range of the melodies was 247 Hz (B3) to 988 Hz (B5). The melodies are played on the piano with one finger (single notes at a time), except the musical phrases for the Meter subtest which last twice as long and are played with two hands (containing chords); they were composed according to Western music rules (Peretz et al. 2003). In the melodic organization subtests (1, 2, and 3), three types of manipulation are applied to the same note of the melodies: Scale, Contour, and Interval.

Scale

An alternate melody is created by modifying one note to be out of scale, though the melodic contour is unchanged.

Contour

One note is altered to change the pitch direction of surrounding intervals while maintaining the original key.

Interval

One note is altered by the same number of semitones as in the Contour test, but the contour is preserved, and the altered note is within the same key. (A semitone is the smallest musical interval commonly used in Western tonal music).

The position of the note that is altered is close to the beginning of the melody for half of the phrases, and close to the end for the other half, although never the first or last note. The temporal organization subtests (4 and 5) comprise the Rhythm and Meter tests. For the Rhythm test, the stimuli are the same as in subtests 1 through 3, but the manipulation involves changing the time interval between two consecutive notes. The position of the change varies. The listener is again required to make a same or different judgment. The Meter test uses harmonized melodies containing chords: half of these are in two-time (march) and half are in three-time (waltz). The subject is required to answer either waltz or march, or two- or three-time. The final subtest assesses musical memory over the course of the last 50 minutes of testing. The subject hears a simple (single-note) melody and is required to judge whether they have already heard it during the test session or if it is a new melody. The subjects were not informed in advance that their memorization of the tunes would later be assessed. There is a progression of difficulty in the first three subtests of MBEA, so the tests were always performed in the same order: Scale, Contour, Interval, Rhythm, Meter, and Melody. In addition, subtest Memory requires previously having completed subtests Scale, Contour, Interval, and Rhythm.

In subtests Scale, Contour, Interval, and Rhythm, a trial consisted of a target melody and comparison melody separated by two seconds of silence; all trials were preceded by a warning tone. The listener was required to judge on each trial if the target and comparison sequence were the same or different. Two practice trials were presented for all subtests except subtest Meter, where there were four. In the first four subtests, one ‘catch’ trial was included to check the listener's attention. In this trial, all pitches were set at random. Each subtest comprised 30 test trials. The tester noted the subject's response on a score sheet. The full test was obtained as 16 bit mono wav files with a sampling rate of 44.1 kHz from Isabelle Peretz at the University of Montreal. Normative data were obtained from the same source for 18 subjects aged 60 to 69 years. A speech-shaped noise sample was made with the same rms level as the mean rms level of the music stimuli; this was used for calibration. The test was implemented in the music player iTunes (Apple Inc, Cupertino, CA). The entire test took around one hour. The dependent variables were the scores (out of 30) on the six subtests of the MBEA.

Affective prosody discrimination

The discrimination of affective prosody was assessed using the comprehension part of the Aprosodia Battery (Ross et al. 1997). The Aprosodia Battery, which was devised to measure functioning in brain-damaged patients, is a standardized test for prosody identification and has been used in several such studies (Monnot et al. 2002; Orbelo et al. 2005; Orbelo et al. 2003; Ross et al. 2001; Testa et al. 2001). The Aprosodia Battery is based on a male talker saying the same sentence ‘I am going to the other movies’ with six different emotional affects: neutral, happy, angry, sad, disinterested, and surprised. The sentences are said with two different stress patterns: one with emphasis on ‘am’ and one with emphasis on ‘other’. This attempts to remove the confounding effect of linguistic prosody cues. Subjects were asked to identify the emotional intonation of each sentence by choosing from six drawings of faces expressing the affects. The written labels were also included on the paper. One bilateral subject was blind; he memorized the choice of affects. There are four subtests: Word, Monosyllabic, Asyllabic, and Discrimination. The subtest Word uses the fully articulated sentence ‘I am going to the other movies’. Subtest Monosyllabic uses the voice saying ‘ba ba ba ba ba’; subtest Asyllabic has the utterance ‘aaahhh’ as the stimulus. Each stimulus is presented 24 times; the talker uses one of the six different emotions. The Word subtest stimuli demonstrate a full range of prosodic cues, including variations in pitch, rhythm, loudness, and voice quality. The Monosyllabic stimuli include all of the above, although the syllabic information is no longer linguistically meaningful. The Asyllabic stimuli mainly demonstrate suprasegmental variations in pitch, loudness, and voice quality, with limited rhythmic cues and no segmental cues. The fourth subtest, Discrimination, uses 24 pairs of low-pass filtered (70 to 300 Hz) stimuli from the Word subtest. Phonetic information was therefore much reduced, although prosodic-acoustic information involving intonation and intensity were preserved. For this subtest, the subject was just asked to indicate if the intonation used was the same or different in the two stimuli.

One additional subtest previously used in a study of 62 elderly adults was included: Attitudinal prosody (Orbelo et al. 2005). Ten sentences were recorded by a female talker (e.g. ‘The children behave so well’), spoken once in a sincere tone of voice and once with a sarcastic tone. The resulting 20 sentences were randomized and presented to the listener twice. Subjects were asked to choose if the statements were true (sincere tone of voice) or false (sarcastic tone).

Due to the decreasing levels of articulatory information presented in the subtests, the subtests were always presented in the order: Word, Monosyllabic, Asyllabic, Discrimination, and Attitudinal. Two practice stimuli for each emotion were played to the subjects before each subtest. For the Attitudinal subtest, two sentences spoken in true and false tones were demonstrated as examples. The tester noted the subject's responses on a sheet.

The entire Aprosodia Battery was obtained from Elliott Ross at the University of Oklahoma Health Sciences Centre, including normative data from twenty-seven 60 to 69 year olds. The sound stimuli (aiff format) were converted to mono 16 bit wav files with a sampling rate of 44.1 kHz. Ten stimuli were taken from each subtest, and the rms intensity of the speech was calculated. A speech-shaped noise stimulus was then produced for each subtest with the same rms intensity as the average of the ten stimuli. This was used as a calibration signal. Because the Discrimination subtest contains filtered tokens with energy below 300 Hz, a low-pass filtered speech-shaped noise calibration stimulus was used. No attempt was made to make every stimulus the same intensity, thus intensity cues still remained. In addition, there were clearly differences in sentence duration, with emotions like ‘disinterest’ or ‘sad’ producing sentences that were longer than ‘happy’ or ‘angry’. This was left unchanged. This was done in order to more closely mimic real life, where cochlear implant users are able to use intensity and duration cues.

All stimuli were presented through the audio player iTunes. Figures 1a-1f show the F0 contours for each emotion, using stimuli from the Asyllabic subtest. These were calculated using SFS (University College London Speech Filing System, London, UK). The test took approximately 30 minutes to complete. There were five dependent variables: the number of correct responses on Word, Monosyllabic, Asyllable, and Discrimination subtests (out of 24), and on Attitudinal subtest (out of 40). These were converted to percentages so they could be plotted together.

Figure 1.

F0 contours for speech stimuli from Asyllabic subtest of the Aprosodia Battery. The talker was a man speaking the sound ‘Aaah’ with the emotional affects a neutral, b happy, c angry, d sad, e disinterested, f surprised.

Talker identification

The Hillenbrand vowel stimuli were used to assess talker identification; these were downloaded from James Hillenbrand's website with his permission as mono 16 bit wav files with a sampling rate of 16 kHz. Vowel stimuli avoid linguistic content and speaker rate cues. The original data collected by Hillenbrand used 45 men, 48 women, and 46 children aged 10-12 years (27 boys, 19 girls) reading lists of the consonant-vowel-consonant clusters: ‘heed’, ‘hid’, ‘hayed’, ‘head’, ‘had’, ‘hod’, ‘hawed’, ‘hoed’, ‘hood’, ‘who’d', ‘hud’, ‘heard’, ‘hoyed’, ‘hide’, ‘hewed’, ‘how’d' (Hillenbrand et al. 1995). The current study used three male, three female, two boy, and two girl speakers. These particular talkers were chosen to have different F0s and sound quite different to a normal-hearing listener. Mean F0, F1, F2, and F3 values for each talker are shown in Figures 2a-2d; these values were also downloaded from Hillenbrand's website. Mimicking a previous study, only two sets of three vowels were used (Vongphoe & Zeng 2005). One set was used for practice and training: ‘had’, ‘heed’ and ‘hayed’. The test set was: ‘heard’, ‘hid’ and ‘hoed’.

Figure 2.

Mean fundamental and formant frequency values for the ten talkers used in the Talker identification task. The averages were taken from the actual stimuli used in the test: ‘heard’, ‘hid’, and ‘hoed’. The black bars represent the three male talkers (m1, m2, m3) and the dark gray bars represent the three woman talkers (w1, w2, w3); the two boys (b1, b2) and two girls (g1, g2) are represented by the light gray and white bars respectively. Figure 2a shows fundamental frequency (F0) values, Figure 2b shows first formant (F1), Figure 2c shows second formant (F2), and Figure 2d shows third formant (F3) values.

Ten photos of faces were chosen from a website that provided royalty-free images. Photos of three men, three women, two boys, and two girls were selected and assigned to man1, man2, woman1, etc. The face was always shown to the subject at the same time as the particular talker spoke. This represents a modification of the procedure described in Vongphoe and Zeng's paper, where the talkers were simply identified by number (man1, man2, etc). It was hoped that the association of a face gave the test more validity. A graphical user interface was written in MATLAB^® by the first author to control and run the test. In most cases, the subject operated the program themselves using the mouse, although a few subjects were not accustomed to using a mouse so the tester operated the program for them. In the case of the one blind subject, the tester ran the program, and the subject referred to the talkers by number. A speech-shaped noise calibration signal was used.

The test procedure had four phases: ‘Initial training’, ‘Test yourself’, ‘Practice’, and ‘Main test’. During Initial training, the subject was introduced to each talker in turn and was encouraged to click on their faces to hear them saying the three stimuli ‘had’, ‘heed’, and ‘hayed’ several times. The second phase was a self-test. The subject heard a voice saying a word twice and had to click on the face of the talker they thought it was. There were 70 trials, with feedback provided after each trial. The third phase provided five minutes additional practice time. The subject could choose which talkers they felt that they needed extra practice on or could listen to them all. The total training and practice phases took about 30 minutes. The final test phase used three different stimuli: ‘heard’, ‘hid’, and ‘hoed’. The subject heard a word spoken twice and was required to click on the talker they thought it was. There were 50 trials, and feedback was not provided. Normative data were obtained on five normal-hearing (≤ 20 dBHL re ANSI-1996 for octave frequencies between 0.25 and 8 kHz) female subjects aged from 20 to 24 years (mean = 22 years).

The dependent variable was the number of correctly identified talkers (out of 50). This was termed the ‘exact score’. The number of talkers who were identified correctly as being either male, female, or child was also recorded; this was termed the ‘category score’ (out of 50). The entire test (including training) took about 40 minutes to complete.

Results

Main analysis

Figures 3 to 6 show mean scores for the two groups (bimodal and bilateral) on HINT with female, male, and child maskers (Figure 3), MBEA (Figure 4), the Aprosodia Battery (Figure 5), and Talker identification (Figure 6). Results are also shown for normal-hearing listeners, although these were not obtained in this study. This is displayed simply to allow the reader to gauge the difficulty of the tests. An ANOVA test was used to examine the main effect of group (bimodal or bilateral) on the dependent variables for HINT (SRT with female, male, and child maskers), MBEA (score on the six subtests), Aprosodia (score on the five subtests), and Talker identification (exact and category score). Due to the multiple comparisons, a Bonferroni correction was used, and the significance level was set at 0.05/16 = 0.003. Table 2 provides the mean, standard deviation, and statistics of the comparisons. There was no significant difference between the mean scores of the bimodal and bilateral groups on any of the tests. At this point the main question of the study has been answered: the mean scores for bimodal users on pitch-related tasks are not different from the mean scores of bilateral users. However, it is clear from the error bars in Figures 3 to 6 that there is large inter-subject variability on the tests, and the number of subjects tested is not large.

Figure 3.

Mean SRT as a function of masker type in 11 bimodal (squares) and 12 bilateral (circles) cochlear implant users. Target material was HINT sentences spoken by a male; the maskers were a female (f), male (m), or child talker. Error bars represent ± one standard deviation. For clarity only the upward bar is shown for the bimodal group, and only the downward bar for the bilateral group. The normal-hearing (NH) data on seven subjects (triangles) were not age-matched, and were obtained in a previous study with an identical test procedure (Cullington & Zeng 2008).

Figure 6.

Mean scores (out of 50) for Talker identification in 13 bimodal (black bars) and 13 bilateral (gray bars) cochlear implant users. The dashed lines represent chance performance. Error bars represent ± one standard deviation. The normal-hearing (NH) data (open bars) on five subjects were not age-matched.

Figure 4.

Mean scores (out of 30) for the six subtests of the Montreal Battery of Evaluation of Amusia in 13 bimodal (black bars) and 13 bilateral (gray bars) cochlear implant users. The dashed line represents chance performance. Error bars represent ± one standard deviation. The age-matched normal-hearing (NH) data (open bars) on 18 subjects were not obtained in this study.

Figure 5.

Mean percentage correct for the five subtests of the Aprosodia Battery in 13 bimodal (black bars) and 13 bilateral (gray bars) cochlear implant users. The dashed lines represent chance performance. Error bars represent ± one standard deviation. The age-matched normal-hearing (NH) data (open bars) on 27 subjects were not obtained in this study.

Table 2.

Mean results for bimodal and bilateral cochlear implant users on HINT sentences with a competing female (f), male (m), or child talker, MBEA, Aprosodia Battery, and Talker identification. Also shown are the standard deviation (sd), number of subjects (n), F value, degrees of freedom (df), and p value for the ANOVA analyses.

Test		Bimodal			Bilateral			F	df	p
		mean	sd	n	mean	sd	n
HINT SRT (dB)	f	2.515	4.705	11	2.167	4.359	12	0.034	1,21	0.855
	m	6.333	3.412	11	7.639	2.560	12	1.090	1,21	0.308
	child	8.758	4.313	11	9.000	3.573	12	0.022	1,21	0.884
MBEA score	Scale	18.000	2.082	13	17.539	2.696	13	0.239	1,24	0.630
	Contour	17.692	3.172	13	17.308	2.463	13	0.119	1,24	0.733
	Interval	17.308	2.359	13	15.615	4.093	13	1.668	1,24	0.209
	Rhythm	23.923	1.891	13	25.385	2.959	13	2.252	1,24	0.147
	Meter	23.231	5.674	13	24.154	5.064	13	0.192	1,24	0.666
	Memory	18.077	3.622	13	16.846	2.609	13	0.974	1,24	0.334
Aprosodia %	Word	77.244	19.141	13	68.910	18.989	13	1.242	1,24	0.276
	Monosyllable	63.462	20.281	13	52.564	14.686	13	2.462	1,24	0.130
	Asyllable	46.154	15.817	13	40.064	14.686	13	1.035	1,24	0.319
	Discrimination	78.205	10.779	13	69.872	10.086	13	4.143	1,24	0.053
	Attitudinal	74.231	11.699	13	76.539	10.535	13	0.279	1,24	0.602
Talker id score	exact	17.385	3.863	13	15.462	3.992	13	1.558	1,24	0.224
	category	36.000	5.385	13	33.923	5.737	13	0.906	1,24	0.351

Correlations between the tests

It is reasonable to assume that those subjects who performed well on one pitch-related task may also perform well on others although this was not the case. Correlation coefficients were computed among the variables SRT with female, male, and child maskers, an average of the first three (pitch) MBEA subtests, Aprosodia Word, and Talker identification exact score. Using the Bonferroni correction to control for Type I error across the six comparisons, a p value of less than 0.008 (0.05/6) was required for significance. Apart from correlations among the HINT tasks, the only significant finding was that Aprosodia Word was correlated with Talker identification exact score (correlation = 0.565, p = 0.003). This suggests that those subjects with better perception of affective prosody were also better at recognizing different talkers. There were no other significant correlations, suggesting that subjects tended to be good at one particular test for whatever reason, but they would not necessarily be good at other pitch-related tasks. Examining the correlation between HINT word score in quiet with the other tests revealed that there were no significant correlations except with HINT in noise. Not surprisingly, those subjects who are good at HINT in quiet are good at HINT in noise. A HINT word score does not predict performance on any of the other tests. It is worth remembering that these subjects were preselected to be good cochlear implant users on the basis of their HINT word score in quiet. The range of values for this variable (65-100%) does not represent the whole spectrum of cochlear implant users, and there may be ceiling effects involved.

Correlations with hearing thresholds

Aided thresholds were obtained in the sound field on eight bimodal subjects; unaided audiograms were obtained on eleven subjects. The others were not tested due to time constraints. Figure 7 shows the mean aided and unaided thresholds for the contralateral ear of the bimodal subjects. If no response was recorded at the maximum output of the audiometer, the value was inputted as 5dB greater than the maximum level tested (e.g., > 90 would be entered as 95). Correlation coefficients were performed among the variables SRT with female, male, and child maskers, an average of the first three (pitch) MBEA subtests, Aprosodia Word, and Talker identification exact score with aided and unaided thresholds at 250, 500, 1000, 2000 and 4000 Hz. Using the Bonferroni correction for multiple comparisons, there were only two significant findings. The average of the first three subtests of the MBEA was significantly correlated with the aided threshold at 1 kHz (correlation coefficient = -0.871, p = 0.005). The HINT SRT with a child masker was correlated with the aided threshold at 2 kHz (correlation coefficient = 0.933, p = 0.002).

Figure 7.

Mean aided and unaided thresholds for the contralateral ear of the bimodal subjects. The squares represent mean sound-field aided thresholds; the diamonds represent mean unaided thresholds. Aided data were available for eight subjects; unaided data were available for 11 subjects. Error bars represent ± one standard deviation.

Music training and education level

Subjects were given a rating depending on how much music training they had received. They were divided into the categories 0 (no formal training), 1 (music training as a child or a little music training as an adult), and 2 (much music training as adult). Only two subjects fell into category 2, so categories 1 and 2 were collapsed to give a rating of either 0 (no formal music training) or 1 (some formal music training). Eleven subjects (six bimodal, five bilateral) indicated that they had received no music training; 15 subjects (seven bimodal, eight bilateral) had received some music training. The subject groups were thus well matched on their music training.

The bimodal and bilateral subjects were analyzed together to assess whether music training influenced test results. An ANOVA was performed between the two groups of ‘no formal music training’ and ‘some formal music training’. There was no significant difference in any of the test results between the two groups, suggesting that extent of music training did not influence performance any of these pitch-related tasks.

Subjects were also given a rating for their educational level. The categories were 0 (graduated high school or less), 1 (college degree), and 2 (Masters degree or PhD). Nine subjects fell into category 0 (four bimodal, five bilateral), eight subjects in category 1 (five bimodal, three bilateral), nine subjects were in category 2 (four bimodal, five bilateral). The subject groups were therefore reasonably well matched on their educational status. The bimodal and bilateral subjects were analyzed together to assess whether the educational status influenced test results. An ANOVA was performed between the three groups of ‘graduated high school or less’, ‘college degree’, and ‘Masters degree or PhD’. There was no significant difference in any of the test results between the two groups, suggesting that extent of education did not influence performance on any of these tasks.

Discussion

Thirteen bimodal and thirteen bilateral cochlear implant users underwent testing on speech recognition with a competing talker, music perception, affective prosody discrimination, and talker identification. Figures 3 to 6 show that the bimodal group performed better than the bilateral group on almost all of the tests; however, the difference was not statistically significant on any of the measures. Although these subjects were pre-selected to be good cochlear implant users (HINT word score ≥ 65% in quiet), as is often found in cochlear implant studies, there was large inter-subject variability in performance.

Speech recognition with competing talker

Both the bimodal and bilateral cochlear implant users performed equally poorly on speech recognition with a competing talker, which is somewhat surprising. Both groups performed at an equivalent level to unilateral cochlear implant users tested with an identical protocol in a previous experiment (Cullington & Zeng 2008). Cochlear implant speech processing does not explicitly provide F0 information, although with a hearing aid it may be available. Previous work using a cochlear implant subject with normal hearing in the other ear had suggested that the addition of acoustic information significantly improved speech recognition with a competing talker (Cullington & Zeng 2010). It was expected, therefore, that the bimodal users would perform better than the bilateral users when the tasks involved separating two talkers due to their presumed better access to F0. This was not the case.

Paired t tests were performed on the SRTs with a female, male, and child masker to see if there was a significant difference between the SRTs with different maskers within each group. A significant p value of 0.01 was used to reflect the Bonferroni correction for three comparisons. The bimodal users' SRT with a female masker was significantly better than with a male masker (two-tailed t = -4.085, df = 10, p = 0.002) and with a child masker (two-tailed t = -6.935, df = 10, p = 0.000); the SRT with a male masker was significantly better than with a child masker (two-tailed t = -3.533, df = 10, p = 0.005). For the bilateral users, the SRT with a female masker was significantly better than with a male masker (two-tailed t = -6.690, df = 11, p = 0.000) and with a child masker (two-tailed t = -7.629, df = 11, p = 0.000). There was no significant difference in SRT between a male and child masker (two-tailed t = -1.368, df = 11, p = 0.199). The three maskers have very different mean F0 values (214 Hz for the female, 101 Hz for the male, and 246 Hz for the child); the F0 of the target male is 109 Hz. A difference between the SRTs for different talker maskers suggests that the cochlear implant users can use a difference between target and masker F0 to help segregate the talkers, although there may have been other characteristics of the speech which assisted segregation. The bilateral users' results are in agreement with those obtained from unilateral cochlear implant users in a previous study using the same test (Cullington & Zeng 2008). Although F0 is not explicitly coded in the speech processor, it can be obtained via weak envelope cues. However, the bimodal users also showed a difference in SRT for the male and child masker. The use of a contralateral hearing aid presumably provides better low-frequency spectral resolution, thus enabling the bimodal users to have slightly better F0 perception. The HINT test is used extensively in cochlear implant research, usually with a steady-state masker. Because there are few occasions in real life when a listener encounters a steady-state masker, the authors suggest that a competing talker offers more validity.

Music perception

The pattern of results obtained is similar to that obtained in a previous study using the MBEA in unilateral cochlear implant users (Cooper et al. 2008). It was expected that the bimodal users would perform better than the bilateral users due to their better spectral resolution at low frequencies, especially as research has shown that hearing aid users who are cochlear implant candidates perform better on pitch tests than cochlear implant users (Looi et al. 2008b). Although Figure 4 shows a slight benefit to bimodal users on the pitch-related tasks, it was not statistically significant. Figure 7 suggests that the frequency range of the MBEA melodies (247 Hz to 988 Hz) should be audible to the bimodal users through their hearing aids.

The bimodal group performed better than chance on all subtests (p ≤ 0.01 using the Bonferroni correction). The bilateral users performed at chance on Interval and Memory, but better than chance on the other subtests. Although the MBEA does have advantages of not requiring musical memory and knowledge (expect perhaps for subtest Meter), it has the disadvantage that the pitch subtests are too difficult for cochlear implant users. It is, therefore, providing limited information about their pitch discrimination abilities. The difficulty of the test may explain the lack of difference in performance between the groups. Further work should use a more appropriate musical pitch test, perhaps based on the work by Looi and colleagues (2008b).

Affective prosody discrimination

This study introduced the use of the Aprosodia Battery in cochlear implant users. It has extensive normative data and is easy to administer. Because F0 is the main cue to affective prosody, one may predict that the low-frequency information provided by the contralateral hearing aid assists in the recognition of affective prosody. However, the mean bimodal users' results were not significantly different from those of the bilateral users. Interestingly, the mean score on the Word subtest for the bimodal group was not significantly different from the age-matched normal-hearing group (t = 1.692, df = 38, p = 0.099). The bilateral cochlear implant users performed worse on average than the normal-hearing group (t = 3.339, df = 38, p = 0.002). As more information is removed for the Monosyllabic and Asyllabic tests, the bimodal group does become significantly worse than the normal-hearing group. For the Discrimination subtest, the bimodal group is again not significantly different from the normal-hearing group (t = 1.359, df = 38, p = 0.182). However, the bilateral group is again significantly worse than the normal-hearing group (t = 3.560, df = 38, p = 0.001). For the Attitudinal subtest, where subjects had to decide if the female talker was using a genuine or sarcastic tone of voice, there was no significant difference between any of the three groups. The bimodal and bilateral cochlear implant users on average performed this task as well as the normal-hearing people (bimodal and normal-hearing comparison t = 0.727, df = 38, p = 0.472; bilateral and normal-hearing comparison t = 0.114, df = 38, p = 0.910). The perception of sarcasm is thus a task that cochlear implant users can perform well and probably relies more on stress or timing than pitch cues. Confusion matrices of the results for the Aprosodia subtests were examined. Considering Aprosodia Asyllabic, where the listeners are assumed to rely most on F0, both the bimodal and bilateral group scored best on the ‘surprised’ emotion and worst on the ‘sad’ emotion. Examining the F0 contours in 1a-1f, the ‘surprised’ contour is clearly different from the others.

Talker identification

There was not a significant difference between the bimodal and bilateral groups for the Talker identification task. Discrimination of F0 is the main characteristic of talker identification; it was therefore expected that the bimodal users would have greater access to F0 and thus perform better. Figures 2a to 2d show the formant frequencies for the ten talkers used in the task. There is some overlap in F0; for example, the girls, boy1, and woman1 are quite similar. Considering only the results from the adult talkers and simply a male or female judgment, the bimodal group mean score was 75% correct and the bilateral group mean score was 71%. This difference was not statistically significant (unpaired two-tailed t test t = 0.784, df = 20, p = 0.442). Fu et al (2005) had achieved a gender discrimination score of 94% when the male and female F0 values were very different, dropping to 68% when there was F0 overlap. Figure 2a shows that there is overlap in F0 between man1 and woman3; otherwise, the F0 values are quite distinct.

Correlation between tests

It was assumed that those subjects performing well on one pitch-related task would also perform well on others. This was not the case. The only significant correlation between test results was between Aprosodia Word and Talker identification exact score. These scattered scores show that the four tasks selected for this study are not simply measuring pitch ability. A test of pitch difference limen may well show a difference between the groups, but this study aimed to use real-world tasks. Clearly there is much more to these real-world tasks than pitch discrimination. This is also demonstrated by the fact that the normal-hearing results have similar standard deviation to the cochlear implant users' results. Because there was minimal correlation between the tests and speech perception in quiet, valuable information may be obtained by extending this protocol to those subjects who are considered poor performers on the basis of the speech recognition score.

Hearing threshold levels

Intuitively it seems that those bimodal subjects with more residual hearing would perform better on pitch-related tasks. The only correlations found were between the aided threshold at 1 kHz and the average of the MBEA pitch subtests, and the aided threshold at 2 kHz and the HINT SRT with a child masker. These analyses were performed on very limited data, but may suggest that better aided hearing in the mid frequencies may be associated with better music perception and better recognition of speech with a competing talker. It is worth remembering though that all these subjects have a severe to profound hearing loss in the contralateral ear, so the range of values for the correlation is limited. Interestingly, previous studies have shown mixed results related to extent of residual hearing. Ching (2005) and El Fata (2009) found that there was greater bimodal benefit in subjects with better hearing, and Mok et al (2006) found that subjects with poorer aided thresholds in the mid to high frequencies demonstrated greater bimodal benefit. Finally, Luntz et al (2005) found that the extent of residual hearing did not predict bimodal success.

Music training and education level

It has been suggested that subjects with more music training may perform better on the MBEA test (Cooper et al. 2008). Indeed this would seem logical, especially on subtest Meter, which required discrimination of a march and a waltz. However, the extent of music training in these subjects did not influence any test results on the four tasks. In addition, the level of education achieved was not related to any of the test results.

Limitations of present study

The numbers in this study were small, and although the subjects were good at speech recognition in quiet, there was large inter-subject variability on the other tests. This study assessed a group mean comparison, and considering the variability shown, a large difference between the groups would be required to show statistical significance. There was no attempt to measure binaural advantage (i.e., the difference in performance between subjects wearing two devices and wearing one device). This was because the authors believe that acclimatization plays an important role in cochlear implant benefit; therefore subjects were tested only in the mode they were accustomed to using. Studies where bilateral or bimodal users are tested in the cochlear implant only condition may falsely depress these implant only scores due to the patient being unaccustomed to listening in this mode; the binaural advantage may therefore appear greater than it really is. An ideal solution would be to test crossover patients, where they start bimodal and then receive a second cochlear implant. They could then be reevaluated after they had become accustomed to the bilateral condition.

The bimodal subjects in this study all received their hearing aids at their local centers; no attempt was made to assess or adjust their hearing aid characteristics. Ching and colleagues have suggested a protocol for fitting a hearing aid to a cochlear implant user in order to optimize performance (Ching et al. 2004b); it is unlikely that this protocol was followed at the local centers. It is therefore possible that some of the bimodal subjects would benefit from a balancing of both devices together.

The question of whether a patient will benefit more from a contralateral hearing aid or a second cochlear implant remains unanswered. In cases where there is no residual hearing in the unimplanted ear, the decision is more straightforward, as many studies have demonstrated the advantage of two implants over one. Studies comparing bimodal and bilateral implantation are fewer and less conclusive. The current study compared group means for bimodal and bilateral cochlear implant users. This study aimed to assess real-world performance on speech recognition with a competing talker, music perception, affective prosody recognition, and talker identification. Accordingly, there were no specific adjustments to the stimuli to control for F0, intensity, duration, and timing cues. It is worth noting that the subjects in this study were all good cochlear implant users; they all had good open-set sentence recognition in quiet. The results cannot necessarily be generalized to the whole cochlear implant population.

Summary

This study was designed to test the hypothesis that bimodal cochlear implant users would perform better on tasks requiring good pitch perception due to the better spectral resolution at low frequencies provided by the residual hearing. This hypothesis was rejected. Although the bimodal group did perform better than the bilateral group on most tasks, this difference was not statistically significant. The tasks used in this study were selected specifically to test the hypothesis. Bilateral cochlear implant users would not be expected to gain an advantage over unilateral cochlear implants on these tasks, as they are not evaluating binaural effects. Bilateral cochlear implant users were used in the comparison so that a two-ear condition was used in both groups. The lack of correlation between test results and the large variability in the test results for normal-hearing listeners shows that the tasks used are not simply providing a measure of pitch ability. Even if the bimodal users have better pitch perception, the real-world tasks used are reflecting more diverse skills than pitch. This research adds to the body of existing speech perception, language, and localization studies that show no significant difference between bimodal and bilateral cochlear implant users.

Thirteen bimodal and thirteen bilateral adult cochlear implant users were evaluated on tasks requiring good pitch perception: speech recognition with a competing talker, music perception, affective prosody discrimination, and talker identification. No significant differences were found between the mean scores of the bimodal and bilateral groups, although the bimodal group mostly scored better. Performance on the tasks was not correlated, suggesting that they were not providing a simple measure of pitch ability, but rather reflected more diverse real-world skills. This adds to existing speech perception, language, and localization studies showing no significant performance difference between bimodal and bilateral implant users.