The Effects of Training on Recognition of Musical Instruments by Adults with Cochlear Implants

Abstract

This study examines the efficiency and effectiveness of three types of training on recognition of musical instruments by adults with cochlear implants (CI). Seventy-one adults with CIs were randomly assigned to one of three training conditions: feedback on response accuracy, feedback-plus (response accuracy plus correct answer), and direct instruction. Each participant completed three training sessions per week over a five-week time period in which they listened to recorded excerpts of eight different musical instruments. Results showed significant pre-to-posttest improvement in music instrument recognition accuracy for all three training conditions (22.9–25.7%, p < 0.0001). Time when tested (week), bilateral CI use, and age were significant predictors of performance. Participants who wore bilateral implants scored significantly higher than participants with unilateral implants at all three time points; hearing aid use was not a significant predictor. These results may have practical implications for numerous types of auditory rehabilitation for persons who use CIs.

Learning Outcomes: As a result of this activity, the participant will be able to (1) describe the three different training protocols for musical instrument recognition; (2) describe the effects of different training approaches on participant recognition; and (3) discuss the predictor variables that can affect training outcomes on musical instrument recognition.

Despite new developments in cochlear implant (CI) technology that have resulted in significant improvements in speech perception, recipients of CIs are still significantly less accurate than listeners with normal hearing in music perception tasks such as the recognition of musical instruments (timbre recognition).^{1 2 3 4 5} In addition, most CI recipients do not enjoy significant improvement in music perception and enjoyment as a result of incidental exposure over time.^{6 7 8} However, a growing body of research suggests that various forms of rehabilitation may help CI recipients to improve music perception.^{6 9 10 11}

Auditory Rehabilitation

In auditory rehabilitation for persons with postlingual hearing loss, the primary objective is to relearn and associate new sounds and cues to restore previously acquired skills. This includes maximizing usage of the acoustic signals transmitted by a hearing aid or CI. Arthur Boothroyd¹² suggests several components for formal auditory training following a change in hearing (i.e., change in hearing thresholds, using a hearing aid or cochlear implant). Along with the expectation that the participant has the desire and motivation to participate, the process should be non-threatening. A computer-based program is one option for effective training.¹² Boothroyd recommended that: a) trainees should receive general feedback on their progress, as well as trial-by-trial feedback when possible; b) training should be interesting, meaningful and rewarding enough to ensure sustained participation; and c) a variety of possible stimuli should be offered to learn the range of the target sound.¹² While Boothroyd's recommendations were made in relation to speech training, these same factors could be applied to music training as well.

Music rehabilitation has been found to be effective in improving perception of various aspects of music listening for individuals with CIs,^{2 6 9 11 13 14} although the extent of improvement varies depending upon several factors such as age of participant, life experiences, and stimuli used in training. Age can affect performance on multiple levels, including auditory temporal processing, with increasing age resulting in declines in the interhemispheric transfer of auditory information and central processing.^{15 16} While no training studies have specifically examined age as an independent variable, studies of various training approaches have found a difference in outcomes as they relate to increasing age.^{6 10 11 13 17} Other trainee factors such as life experiences or cognitive skills also can affect the rate of learning and rehabilitative benefit. This includes experiences in music listening and the knowledge of musical instruments prior to and throughout training. Gfeller and colleagues^{7 8} found that music training prior to hearing loss was a significant predictor of pure-tone pitch ranking, familiar melody recognition, musical excerpt recognition with and without lyrics, and timbre recognition.

As noted previously, rehabilitation is likely to be more effective when the individual is interested and motivated to participate.¹² Using survey data, Looi and She¹⁸ found that 54% of 100 CI users in their study would be interested in some form of music training if one became available. The ability to recognize commonly-known instruments was ranked as the third most desired musical skill. Survey data of 40 CI users (unilateral, bimodal, and bilateral users) from Philips, et al,¹⁹ stated that bimodal and CI-only users believed that music training would be helpful in maximizing their CI benefit, while only 33% of those who completed the survey actually received music training during their rehabilitation. The authors recommended that training should include different levels of difficulty and should be provided early in the rehabilitation process.

The listening task (e.g., stimuli, feedback, response task requirements, etc.) can influence the effectiveness of auditory rehabilitation. Some studies indicate that as the complexity of the acoustic signal increases (such as multiple talkers, unfamiliar speech patterns, or complex music), the demands on memory and attention increase, resulting in greater levels of difficulty.^{5 16 20} Consequently, the learning period required to achieve significant benefit also increases.²⁰ In addition, the greater the similarity between the listening task and prior listening experiences, as well as training and testing conditions (i.e., format, layout, interaction, etc.), the more the user is able to apply the learned information from the testing session.²⁰

Specific to training of music instrument recognition, a handful of studies indicate that timbre perception of CI users can improve as a result of feedback or other forms of training.^{5 6 10 11 13} Pressnitzer et al,⁵ examined the influence of attack time and spectral center of gravity on timbre recognition, incorporating adaptive procedures with some initial training and feedback in an alternate forced-choice paradigm. CI recipients enrolled in the study were offered a training period in which they could ask for feedback prior to providing their answers. The participants showed a “good level of performance” on timbre discrimination following the training; no exact scores were provided. The authors concluded that the ability of CI recipients to discriminate changes in attack time and spectral center of gravity should allow for musical instrument recognition after training.

Gfeller and colleagues^{6 10 11 13} compared musical instrument recognition for CI recipients enrolled in a 12-week home-based computer training program with CI users who had only incidental experience (control group) to music during that same time period. Participants in the experimental group were trained on eight musical instruments presented with video, audio and written descriptions of the instruments, with ∼10 minutes of training on instruments for each lesson. Results revealed that the training group showed significant improvement (p < .01) in instrument recognition following 12 weeks of training, while those in the control group did not.

To assess the relative efficiency and effectiveness of different types of feedback in timbre training, Driscoll and colleagues¹⁷ evaluated the ability of 60 adults with normal hearing to identify musical instruments presented through CI simulation. Participants were placed into three treatment groups differentiated by the type of information provided: repeated exposure to timbre excerpts with no feedback; repeated exposure to timbre excerpts with feedback on response accuracy; and direct instruction on music instrument timbre. Although participants in the repeated exposure group showed significant improvement from pretest to the third week of testing (∼7% improvement), their scores were significantly lower than those of the other two groups (with 29% and 39% improvement, respectively); the repeated exposure group showed no significant improvement beyond the third week of testing. The other two conditions resulted in continued improvement over the entire training period, which sustained two weeks post training for the direct instruction group.

Although the simulation study suggests that normal hearing listeners can improve their perceptual accuracy in identifying degraded acoustic signals, generalization of these simulation results to CI users must be made with care, given a variety of individual factors that can influence perception of sounds such as auditory health, cognitive processing, environment and hearing background.^{7 8 21} Therefore, the purpose of this study was to evaluate whether the training program developed by Driscoll and colleagues,¹⁷ and trialed with normal hearing adults using CI simulations would show similar improvement when used by adults with CIs. Because several studies have already confirmed a lack of improvement in timbre perception as a result of mere exposure over time,^{6 10 11 13 17} the training conditions for this study did not include repetition only, but rather compared two types of feedback as well as direct instruction. That is, the study compared the efficacy of three approaches to training: feedback, feedback plus, and direct instruction on the ability of CI recipients to recognize different musical instruments. We hypothesized that the rate of improvement may be greater for the training conditions providing the most specific and informative feedback.

Method

Participants

Postlingual deaf adults using traditional long-electrode CIs (unilateral and bilateral) were recruited through (a) a database of patients enrolled in existing research studies, (b) a brochure provided to audiologists throughout the US, and (c) invitations to participate posted on social media (Twitter). Participants were required to be age 18 or older, regularly use a cochlear implant, read/understand written English, and to have access to a computer with internet and sound capabilities. Individuals who were interested in the study contacted our research team at which time they received a letter containing the elements of consent. Any participants who were prelingually deaf were not included in the analyses.^a Seventy-one listeners with CIs (21 males, 50 females) who ranged in age from 26 to 88 years (M = 62.59, SD = 12.61) completed the study. Twenty of the participants used bilateral CIs. Of the 50 unilateral users, 16 wore contralateral hearing aids. Forty one participants used devices manufactured by the Cochlear Corporation, 28 participants used implants from Advanced Bionics Corporation, and two individuals used the Ineraid device from Med-El. Upon agreeing to take part in the study, participants were provided the web site address, instructions, user name and a password to access lessons. Each participant was randomly assigned to one of the three experimental conditions. Participants received no compensation for participating in the study. The research protocol was approved by the institutional review board.

Stimuli

Stimuli consisting of recordings of solo instrumental performances were prepared for testing and training. Both testing and training included the same sound clips from eight musical instruments commonly known by non-musicians within the musical culture of the United States, as confirmed through prior studies.^{11 17} The instruments were trumpet, trombone, violin, cello, flute, clarinet, saxophone, and piano. These instruments represented a range of low, middle and high frequencies. The sound clips were recorded by professional musicians in a sound studio. A full description of the recording method can be found in Driscoll, et al.¹⁷

The timbral stimuli in this study were the same recordings as described in Driscoll, et al.¹⁷ In brief, 5 melodies were prepared for each instrument, except for piano in which 10 melodies were used to target the high, middle and low frequency ranges it covers. The melodies for all of the instruments included one seven-note tune which used equal-duration notes, adopted from studies conducted by Gfeller et al,^{2 3 11} and two standardized melodic phrases of 14 and 15 notes each, with minimal rhythm cues, composed by the researcher. These two melodies included the range of a full octave, intervals of one to 9 semitones and ascending and descending contours. In addition to these standardized melodic phrases used for all the instruments, two 6–12 second musical phrases considered stylistically unique or idiosyncratic for each instrument (e.g., “Trumpet Voluntary” by Purcell for the trumpet) also were recorded. The idiosyncratic selections were made with the assistance of professional musicians and university instructors who are specialists on their respective instruments and performed by the same professors or another professional performer. The stimuli were normalized to equal RMS amplitude.

Training Program Lessons

The training program was developed to provide consistent and repeated exposure to the eight instruments representing different instrumental families (based upon sound production principles, i.e., woodwind, brass, string, and pitched percussion) and frequency ranges (low, medium, high). The training program was a modified version of the training described in Driscoll et al,¹⁷ in which NH listeners were trained listening to CI simulations. From that study, it was determined that timbre recognition does not improve significantly as a result of mere repeated exposure, but can improve as a result of feedback or direct instruction. This study utilized similar computer-based training but the stimuli were unprocessed instrumental training stimuli; training consisted of 15 lessons presenting recordings of the eight target instrumental sounds, completed over five-weeks.

The lessons were delivered online via a web site, which the participants accessed at home using their own personal computers. Participants were asked to complete all the lessons and testing in the same listening format they would for other music experiences (i.e., headphones, direct connection or speakers). Each individual was provided a unique user name and password, and instructed to access the program 3 times per week; each lesson lasted ∼10 minutes. The computer program was formatted to require elapsed time between sessions because spaced rehearsal has been documented as more effective than massed rehearsal, and because consolidation of new information has been associated with sleep after learning.^{22 23} Specifically, a 12-hour clock began at the completion of each lesson and the participant could not log in to complete the next lesson until that time had expired.

Training programs were created using Adobe Flash; sound files used in the programs were MP3 audio files stored within the web site. A randomization list was stored within a structured query language (SQL) database to ensure each instrument was presented an equal number of times. In the lessons, each instrument was represented with one melody presented four times per lesson, over the five-week period, resulting in a total of 60 presentations per instrument. For example, in lesson one all the instruments played the 7-note melody and in a subsequent lesson all instruments presented with the 14-note melody. Thus, each melody group was performed 3 times per training; one melody group every five days.

Training Conditions

The 71 participants were randomly assigned to one of three experimental conditions (1) Feedback (FB), (2) Feedback Plus (FB + ) and (3) Direct Instruction (DI). Both FB and DI were replicated from the Driscoll, et al¹⁷ study to assess their effectiveness. FB+ was added as a third training condition to evaluate the effect of one piece of additional information. Each treatment condition included the same instrumental samples, and the same number of exposures to each item. Because the randomization to each training condition was not stratified, and because some individuals who agreed to participate did not begin the program (and are not included in the final report), there are unequal numbers of participants in each group.

In the FB condition (n = 24), correct/incorrect feedback was provided. The participant listened to a series of short melodic excerpts played by the 8 different musical instruments. After listening to each short melody, they guessed which instrument they had just heard by selecting, from a closed set of 12, the picture of the instrument on the computer interface. Following their selection, they were provided with feedback as to whether they were correct or incorrect. No additional information regarding which instrument had actually played was given.

The FB+ condition (n = 22) provided a nearly identical version of the FB condition, which was correct/incorrect feedback, with one modification, the addition of the name of correct instrument. In this condition, the participant made their selection (via the computer interface) regarding the instrument they believe had been presented. In addition to the correct/incorrect statement following their response, the program presented the name of the instrument that had been played (e.g., “Incorrect. That was a clarinet.” or “Correct. That was a clarinet”). It was hypothesized that information regarding the correct answer would increase the rate of improvement.

Direct Instruction (DI) consisted of directed instruction on the instrument being played, followed with correct/incorrect feedback. While the DI condition (n = 25) included exposure to the same short melodies, each melodic excerpt was accompanied by instructional information during the initial presentation of the instrument. The descriptions provided during DI were identical to those used in Driscoll, et al¹⁷ and were based upon reports provided by implant users as well as by normal-hearing listeners describing CI simulations. In brief, information regarding the typical use, history, and mechanics of the instruments were also provided during the initial presentation to aid in the learning and recognition. The participants were provided with a picture of the instrument while listening to the melody for that lesson. Specific characteristics of the instrument and the timbral components that might be heard through the implant during each audio excerpt were also provided (e.g., “The sound of the flute is thin and airy.” “The trumpet has a crisp start to the notes and each is well-focused.”). After the first presentation of each instrument, the remaining three exposures were identical in format to the FB condition and the participants received only “correct” or “incorrect” in response to their selections.

Because the DI condition provided opportunities to associate the new sound quality with prior knowledge (thus facilitating top-down processing), it was predicted that the participants who received the direct instruction would show the greatest improvement on the instrument identification task and perform significantly better than the other groups. Additionally, it was hypothesized the provision of the correct answer (FB+ condition) would be superior to feedback on accuracy alone (correct/incorrect, FB condition). To ascertain the amount of training required to achieve significant gains, the study included multiple measures over the entire training period to examine the trajectory and rate of change for the three training approaches over the 5 week period. It was hypothesized that, similar to the Driscoll, et al¹⁷ study, the greatest rate of improvement would be observed by the midpoint test (after 3 weeks of training) in all three groups, after which DI would show continued improvement.

Measurement of Training Benefit

Instrument recognition test. An instrument recognition test was administered prior to, during, and after training to ascertain rate and extent of learning as a result of the training program. This test consisted of a 12-alternative forced-choice (12AFC), using the eight trained instruments and four foils (drums, xylophone, harp, organ) and was scored as both a total number correct and percent correct. Five different melodies were presented one time for each instrument, for a total of 45 presentations (piano was presented twice for each melody grouping). Test stimuli were presented at participants' preferred levels of loudness and everyday mode of music listening (free field, headphones, direct input). All participants were instructed to use the same listening conditions for the duration of the study including testing and training (to which all reportedly obliged). This method of presentation was encouraged to guarantee identical testing and training situations and equipment.

The computer interface consisted of pictures (photograph or realistic drawing) of each instrument, as well as the written name. During testing and training, participants were asked to click the mouse on the picture of the instrument they believed had played the melody they just heard. No feedback was provided during the testing conditions. The same test was administered to all participants at mid-point (week 3) and post-training (week 5).

Data Collection

Upon completion of the study, each participant was provided with a summary of their improvement over the 5 week training, which included percentage of improvement as well as information regarding the training condition to which they were assigned. They also completed a questionnaire about their musical background (MBQ) and familiarity with the 12 musical instruments presented in the response form prior to training (i.e., the 8 tested and trained items and four foils). Number of instruments known prior to training was evaluated as a potential predictor variable. Participants were also encouraged to offer anecdotal feedback on the content, format, and efficacy of the training program.

Results

The dependent variable in this study was the percentage of musical instruments correctly identified in the recognition test (percent-correct score). Instrument recognition was measured at three points in time: pre-training, the mid-point of training (week 3), and after completion of training (week 5) (Table 1). A linear mixed model (PROC MIXED) was developed for the analysis. Variables included as predictors in the analyses were training condition, time of testing (pre, week 3 & week 5), gender, musical instruments known prior to testing, age of participant, use of hearing aid, unilateral or bilateral device usage, musical background (as measured by a questionnaire completed online following training) and location (1: participant from our clinic or 2: recruited through brochure/online posting).

Table 1. Percent Correct Scores for each Group at Times of Testing.

Time	Group	% Correct	Standard Deviation
Pretest	FB	47.01	15.45
	FB+	51.72	16.42
	DI	51.91	15.71
Week 3	FB	67.48	16.11
	FB+	72.22	15.28
	DI	73.16	12.75
Week 5	FB	69.91	15.28
	FB+	74.85	15.43
	DI	77.60	12.50

In addition to testing for differences between the three conditions, we examined improvement as a function of time. We hypothesized that the rate of improvement may be greater for the training conditions providing the most specific and informative feedback (FB, FB + , DI). Thus, an interaction between condition and time was included in the statistical analysis. An interaction between time and location also was tested. In this model, participant was treated as a random effect to account for within subject variability.

RESULTS FOR THREE TESTING TIMES

Of the 9 predictor variables, 3 were significant at the level of 0.05: time when tested (week), age of participant, and device usage (unilateral or bilateral) (see Table 2). From parameter estimates, we found that prior music experience (as measured by the MBQ) had a positive relationship with the percentage of correct answers (p = 0.02) at pretest only. Age of the participant was significant (p = 0.043), but the parameter estimate was negative, indicating a tendency for older participants to perform more poorly in the instrument recognition test. There was no significant difference between the three groups at pretest (p = .37). As hypothesized, a significant improvement was observed from pretest to week 3 test (p < .0001) with an additional significant improvement observed from week 3 to week 5 (p = .0114) for all three groups. Fig. 1 shows no evidence of a condition by time interaction as all three conditions showed a similar pattern of change. For all three conditions we see the probability of correct response increasing from pretest to week 3 and week 3 to week 5. Individuals who had bilateral CIs scored significantly higher than those with unilateral implants (p = 0.02) with and without hearing aids at all three testing times; hearing aid use did not predict performance.

Table 2. Tests of Fixed Effects.

Variable	DF	F Value	P value
Condition	2	1.67	0.14
Week	2	213.06	<0.01
Music Experience	1	3.21	0.10
Age	1	3.76	0.04
Gender	1	0.68	0.34
Instruments Known	1	0.47	0.35
Location	1	2.08	0.11
Hearing Aid	1	1.92	0.11
Bilateral CI	1	7.89	<0.01

Figure 1.

Percent correct for each group measured at each testing period. Error bars indicate one standard deviation. The lines show the trajectory of each group and the markers represent the average percent of instruments identified correctly for the group. The corresponding error bars display one standard error.

DISCUSSION

The purpose of this study was to compare three types of training conditions for timbre recognition with postlingually deaf adult CI users. This study did not include a control (test-only) group. However, longitudinal timbre recognition data from 21 participants in our clinic showed no significant improvement in timbre recognition over time as a result of everyday listening experience over a two year time period.²⁴ Further, research by Gfeller et al¹¹ found that the control group showed no significant improvement on timbre recognition over time when compared with the training group. This is also consistent with the no-feedback training condition in simulation study by Driscoll et al,¹⁷ which resulted in significantly less improvement over time and only minimal initial improvement. The results of this study indicate that training which includes feedback, whether detailed to include additional information or including only whether one was correct or incorrect, and/or direct instruction can improve perception of music timbre, despite the degraded signal transmitted by the CI. In addition, significant improvement can occur with as little as 3 weeks (90 minutes) of training. Thus, these data suggest that even a modest amount of training can help CI recipients improve perception for recognition of musical instruments that have been systematically trained.

These results are consistent with prior studies,^{11 17} which indicate that musical timbre is amenable to rehabilitation, despite the limitations in CI technology. Whereas direct instruction results in higher scores than feedback alone, even correct/incorrect feedback on what one is hearing while listening to sounds in everyday life may assist CI listeners in understanding and recognizing sounds in their environment. The results have implications for the sorts of information that should be included in auditory rehabilitation for timbre recognition. Because age was found to be a significant predictor, it is possible that older CI users may need more extensive, longer, or different forms of training to achieve benefits similar to those of younger CI users. This supposition, however, requires direct testing.

It is important to acknowledge that participants were trained on the same items included in the tests, generalizing to stimuli outside of the training would have shown a greater effect of training and that these results may simply be displaying a learning effect for those items; however, anecdotal reports from many participants reveal they were able to apply the skills gained in the study to their preferred styles of music and reported an increase in recognition and appraisal. Because the testing and training materials were similar, future research is required to determine whether training would generalize to other instrumental timbres that were familiar prior to deafness, but not included in the training protocol.