Simultaneous Grouping in Cochlear Implant Listeners: Can Abrupt Changes in Level Be Used to Segregate Components from a Complex Tone?

Abstract

A sudden increase in the amplitude of a component often causes its segregation from a complex tone, and shorter rise times enhance this effect. We explored whether this also occurs in implant listeners (n = 8). Condition 1 used a 3.5-s “complex tone” comprising concurrent stimulation on five electrodes distributed across the array of the Nucleus CI24 implant. For each listener, the baseline stimulus level on each electrode was set at 50% of the dynamic range (DR). Two 1-s increments of 12.5%, 25%, or 50% DR were introduced in succession on adjacent electrodes within the “inner” three of those activated. Both increments had rise and fall times of 30 and 970 ms or vice versa. Listeners reported which increment was higher in pitch. Some listeners performed above chance for all increment sizes, but only for 50% increments did all listeners perform above chance. No significant effect of rise time was found. Condition 2 replaced amplitude increments with decrements. Only three listeners performed above chance even for 50% decrements. One exceptional listener performed well for 50% decrements with fall and rise times of 970 and 30 ms but around chance for fall and rise times of 30 and 970 ms, indicating successful discrimination based on a sudden rise back to baseline stimulation. Overall, the results suggest that implant listeners can use amplitude changes against a constant background to pick out components from a complex, but generally these must be large compared with those required in normal hearing. For increments, performance depended mainly on above-baseline stimulation of the target electrodes, not rise time. With one exception, performance for decrements was typically very poor.

Introduction

Concurrent mixtures of sounds have to be analyzed by our auditory system in many everyday listening situations, such as when following speech in the presence of competing speech or when listening to polyphonic music (Bregman 1990; Darwin and Carlyon 1995). It is well established that the ability of cochlear implant (CI) listeners to understand speech is often greatly impaired by competing background sounds (e.g., Nelson et al. 2003; Stickney et al. 2004). Despite this, the psychophysical abilities of CI listeners to segregate simultaneous sounds perceptually (concurrent sound segregation) have received little attention. Furthermore, it should be noted that there is currently no consensus on the extent to which CI listeners can perform the related task of sequential grouping. Although some studies have claimed to demonstrate auditory stream segregation in CI listeners (Chatterjee et al. 2006; Hong and Turner 2006, 2009), others have disputed these claims (Cooper and Roberts 2007, 2009; Cooper 2008).

A number of cues for concurrent sound segregation are used by normal-hearing (NH) listeners, but some of them cannot be expected to be available to CI listeners. In particular, cues based on harmonic relations are unlikely to be useful because individual harmonics are not effectively conveyed by any currently available combinations of electrode arrays and speech-processing strategies. Individual harmonics are generally not resolved, and the log spacing of implants channels does not provide any means of signaling harmonic relations between components. Similarly, fundamental (F0) frequency is poorly transmitted in implant listening. Indeed, unlike NH listeners, CI listeners are unable to take advantage of gender mismatches between target and masker speech when listening in a “cocktail party” situation (Nelson et al. 2003). In contrast, when separating target speech from competing speech sources, NH listeners can use differences in F0 between talkers to improve intelligibility (Brokx and Nooteboom 1982; Bird and Darwin 1998).

Some cues for concurrent segregation are available, in principle, to CI listeners and might therefore be demonstrable in an experimental setting. One of these is onset asynchrony, which has been shown to be a robust cue in NH listeners (e.g., Darwin 1984; Roberts and Moore 1991). However, a recent study by Carlyon et al. (2007) has shown that a delay in the onset of stimulation on one electrode with respect to that on a set of others (analogous to an onset time difference on one component within a complex tone) provides only a small benefit for concurrent segregation in CI listeners. Their study is considered further in the “Methods” and the “Discussion.”

Abrupt increases in sound mixtures tend to signal the introduction of new sources, while gradual changes are more likely to be caused by changes in ongoing sounds (Bregman 1990). Many natural sounds, in particular percussive sounds, have sharp onsets but gradual offsets. This sudden-change cue can be used by NH listeners for concurrent sound segregation. For example, a single pure-tone component can be caused to “pop out” from a complex tone if its intensity is momentarily reduced and then returned to its original level (Kubovy 1981). In a follow-up study, Kubovy and Daniel (1983) presented listeners with a chord made up of eight pure tones. When all the tones had the same intensity, none of them dominated the perceived pitch of the chord. When each of the tones in turn was reduced in level for 80 ms and then abruptly restored to its former level, the pitch of that tone became dominant, allowing a scale or melody to be played. This was a robust effect; it could still be perceived even for amplitude notches as shallow as 3 dB. Kubovy and Daniel (1983) referred to this as segregation by momentary amplitude disparity; there had been no overall increase in energy in the tone, but a sudden rise in its intensity led to its perceptual dominance.

The effect of sudden change on concurrent sound segregation in NH listeners was explored in more detail by Bregman et al. (1994a), using incremented target components. In one experiment, they used a 3.5-s-long complex tone consisting of the first five harmonics of an F0 of 500 Hz. The amplitudes of an adjacent pair of two of the “inner” tones (1,000, 1,500, or 2,000 Hz) were incremented in succession, without any temporal overlap, in the middle of the complex while the rest of the components remained at a steady state. This led to the pitch of each incremented tone becoming salient and standing out from the rest of the complex as a pure-tone-like percept. The increments were 1 s long and had triangular amplitude envelopes with rise times of 30, 90, 270, 730, 910, or 970 ms; the sizes of the increments were 1, 3, or 6 dB. Listeners reported the order of the two target tones, by indicating whether the pitch pattern had gone up or down.

Bregman et al. (1994a) found a strong effect of rise time, with the best discrimination scores obtained with the most sudden onset (30-ms rise time) and declining with increasing rise times up to the maximum of 970 ms. There was also a small but significant improvement in performance as the increment size was increased, indicating that a temporary increase in energy can also be used to pick out a target component. Note, however, that a rapid onset could be discriminated almost as easily at 1 dB as at 6 dB. The explanation for the sudden-change effect given by these authors was that the increments activated the same sort of onset responses in the auditory system as are seen for true onsets. Their hypothesis was that this onset response does two things; it supplies frequency information enabling identification of the tone, and it “flags up” a narrow spectral region for enhanced analysis by other processes, such as pitch computation, enabling auditory scene analysis. The sudden-change effect described by Kubovy (1981), Kubovy and Daniel (1983), and Bregman et al. (1994a) for NH listeners has not yet been explored in CI listeners; the aim of our experiment was to do so.

Relatively few studies have explored systematically the abilities of CI listeners to perform simple intensity discrimination tasks, and the available data relate primarily to older types of implant. However, there is evidence that intensity discrimination by CI listeners is generally quite good. For example, Zeng and Shannon (1999) measured the difference limen (DL) in percentage of dynamic range (DR) for users of the Ineraid percutaneous implant. The average DL was about 8% DR for the stimulus type (100-Hz pulses) and baseline level (50% DR) most comparable to those used in our experiment. Kreft et al. (2004) collected similar data for users of Clarion transcutaneous implants across a range of pulse rates. For a baseline level of 50% DR, the average DL was about 4% DR for users of the standard spiral electrode array and about 10% DR for the perimodiolar array. These results informed the choice of the increment/decrement sizes used in our experiment. The aim was to measure how well CI listeners perform a task requiring them to use changes in level to segregate components from a background of stimulation. This kind of task is considerably more challenging than simply detecting a change in stimulus level, and so we selected a range of increment/decrement sizes for which the intensity DL would not limit performance.

Methods

Overview

Two cues potentially available to CI listeners for segregation of one sound from a mixture of other sounds were evaluated. First, a temporary increase in energy in the stimulation applied to one electrode, in the context of steady-state stimulation on other electrodes, should provide a “pop out” cue for the electrode to which it is applied and elicit an audible pitch if CI listeners are able to detect such a change in energy. If CI listeners perform in a similar way to NH listeners, an abrupt rise in energy should provide a stronger segregation cue than a slow rise when the total energy contained in each increment is identical. Second, a temporary decrement in the stimulation applied to one electrode in a mixture, followed by a return to its previous value, should also produce a sudden-change effect, similar to that seen in the experiment described by Kubovy and Daniel (1983). If CI listeners are able to use a sudden-change cue in this context, a pitch should again be elicited related to the electrode whose level has been manipulated, and this should be most effective when the offset (i.e., the rise to normal level at the end of the decrement) is more abrupt.

The experimental approach used was similar to that described by Bregman et al. (1994a). The main difference was that, instead of a complex tone made up of harmonics 1–5 of a 500-Hz F0, an analog of a complex tone was created by stimulation of five electrodes evenly spaced across nearly the whole array. Thus, the stimuli were more akin to a log-spaced set of components in NH listeners. Given the generally limited spectral resolution and problem of current spread known to occur with CI listeners (e.g., Throckmorton and Collins 2002; Fu and Nogaki 2005), the chance of resolving the individual components in the complex was optimized by using widely spaced electrodes.

The other key difference between this experiment and that of Bregman et al. (1994a) was the size of the increments and decrements used. They found that NH listeners could discriminate increments with abrupt onsets as small as 1 dB. However, we tested larger increment sizes to ensure that the task was manageable for CI listeners. Expressed in terms of the available DR, see below, the largest increment/decrement used was 50%. This is roughly equivalent to around 60 dB in an NH listener with a DR of 120 dB and so might reasonably be expected to provide a clear cue for segregation. The smallest increment/decrement size used, 12.5% DR, was chosen to be greater than the mean DL for a CI listener in a simple intensity discrimination task (range = 4–10%—Zeng and Shannon 1999; Kreft et al. 2004).

The investigation by Carlyon et al. (2007) of onset asynchrony as a cue for concurrent segregation in CI listeners explored the effect of a delayed onset of stimulation on the target electrode relative to that of the others comprising the stimulus complex. Prior to this onset, the target electrode was not stimulated at all, and the rise time was effectively instantaneous. What distinguishes our study from theirs is that we explored the effect of changing the level of ongoing stimulation on the target electrodes, using both increments and decrements, and also the temporal characteristics of these changes, using both fast and slow rise times.

Listeners

Eight experienced adult users of the 22-channel Nucleus CI24 device took part. All had normal electrode insertions and showed no evidence of electrode pitch reversals. Their demographic and clinical details are shown in Table 1, including their open-set (sound only) speech recognition score for sentences, recorded within 6 months of the experiment. All listeners gave informed consent, and the research was approved and overseen by the Aston University Ethics Committee.

TABLE 1.

Demographic and clinical details of listeners

Listener	Age	Gender	Duration of deafness (years)	Etiology of deafness	Open-set speech recognition score (% correct for BKB sentences in quiet)	Pulse rate in normal daily use (pulses per second)
L4	33	F	33	Congenital	88	250
L12	69	F	30	Unknown	93	1,200
L13	51	F	8	Ménières	100	900
L15	69	M	10	Unknown	68	900
L16	64	M	20	Ototoxicity	53	500
L17	79	F	10	Unknown	93	1,200
L18	73	M	60	Infection	27	900
L19	63	M	40	Unknown	74	2,400

Calibrating stimulation levels

Levels of stimulation provided by the Nucleus implant are divided into discrete steps, referred to as stimulus units, which represent current delivery (μA) on a log scale. Threshold (T) level is defined as the lowest stimulus level on an electrode that can be detected reliably and the maximum comfortable loudness (C) level is defined as the highest stimulus level on an electrode that remains comfortable. Prior to the experimental runs, and for each listener, T and C levels were measured for each of the electrodes to be stimulated, using the implant manufacturer’s programming software. This was achieved by a “sweep” across channels, as many times as required to obtain stable estimates. In this procedure, 500-ms stimuli, with 500-ms interstimulus gaps, were applied in turn to each electrode in the test set, starting near the apical end of the electrode array and progressing towards the basal end. The listeners reported the loudness of the stimulation on each electrode, and adjustments were made to individual levels as necessary to obtain estimates of the T and C levels. The DR for each electrode was defined as the difference between these levels. Stimulation levels used in the experiment were defined in terms of DR.

Stimuli and conditions

Part one of the experiment used increments in stimulus level, and part two used decrements, in a “complex tone” made up of stimulation on five well-separated electrodes (3, 7, 11, 15, and 19). The target electrodes to which the increments or decrements were applied were always an adjacent pair within the interior set of three. The changes were applied to two electrodes in succession, in a background of steady-state stimulation on the other electrodes. The steady-state stimulation was set at 50% DR for each electrode throughout; listeners reported that this evoked broadly similar sensation levels (medium loudness) on each of the stimulated electrodes. Increment sizes were an additional 12.5%, 25%, or 50% DR; the last of these resulted in an increase in level peaking momentarily at C level. Decrements were at 12.5%, 25%, or 50% DR; the last of these resulted in a decrease in level down to a momentary minimum at T level.

Each stimulus was 3.5 s long in total; individual increments and decrements were 1.0 s long. The first and second increments (or decrements) started 0.5 s and 2.0 s after the onset of the five-electrode complex, respectively (i.e., there was 0.5 s between the end of the first and the beginning of the second). Two amplitude envelopes were used for the increments and decrements—either an onset and offset of 30 and 970 ms or the reverse. On each trial, the same configuration of increment (or decrement) size and amplitude envelope was used for the two target electrodes. As the increments and decrements had linear onsets and offsets and no plateau, the 970-ms onset case was identical to the temporally reversed version of the corresponding 30-ms onset case, containing identical energy when the same-sized increment (or decrement) was added to the stimulation applied to a given target electrode. Note that both the 30-ms onset increments and 970-ms onset decrements are associated with an abrupt rise in stimulus amplitude (fast rise time). Conversely, the 970-ms onset increments and 30-ms onset decrements are associated with a gradual rise in amplitude (slow rise time).

Individual increment and decrement sizes were calculated for each listener and each electrode; an example is shown in Table 2. For all listeners, a constant pulse rate of 900 pps and a pulse width of 25 µs were used throughout; stimulation was monopolar (MP1 + 2), and pulses were interleaved across stimulated electrodes. Increments/decrements were applied only to electrodes 7 (H), 11 (M), or 15 (L). The H, M, and L notations were used to indicate the expected relative pitches for the electrodes concerned; for example, the location of e7 is more basal compared with that of e11 and so should produce the highest (H) pitch out of the three. Stimulation remained at 50% DR throughout on the remaining electrodes and also before and after the increments/decrements on the target electrodes. Based on pilot work with one CI listener, the spacing between successive targets (separation = four electrodes) was chosen to provide a clear change in pitch when the targets were presented as stimulation on two isolated electrodes.

TABLE 2.

Example of calculation of increment and decrement sizes for an individual listener

E	T	C	DR	50% point	50% DR	25% DR	12½% DR
3	100	155	55	128	28	14	7
7	110	165	55	138	28	14	7
11	120	172	52	146	26	13	7
15	120	173	53	147	27	13	7
19	115	175	60	145	30	15	8

All levels are quoted in stimulus units, as delivered by the nucleus CI24 implant. From moment to moment on each electrode, the level of the amplitude envelope was rounded up to the nearest integer value in stimulus units. Note that, in practice, dynamic ranges varied considerably across listeners and electrodes (minimum = 25; maximum = 65)

e electrode number, T threshold level, C maximum comfortable level, DR dynamic range, 50% point level used for “steady state”, 50% DR, 25% DR, 12.5% DR maximum increment/decrement sizes for each percentage of dynamic range tested

Specific manipulations of stimulus parameters were achieved using the “Impress” software for precision control of the Nucleus CI24 cochlear implant. This allowed the creation and presentation of various combinations of stimulation on multiple electrodes. In order to verify that the selected channels were activated as intended by the experimental software, all stimuli were routed through the experimental speech processor, and the output was analyzed using a “dummy” cochlear implant within the manufacturer’s computer interface; this allowed the generation of “electrodograms” which illustrate visually the output on each electrode over a selected time window. Representative examples of these outputs are shown in Figure 1.

FIG. 1.

“Electrodograms” displaying the output of the experimental speech processor and transmitter coil for different examples of experimental stimuli. Filled bars illustrate when electrical stimulation was present on each electrode. The vertical height of each bar indicates the amplitude of the stimulation applied to that electrode. The abscissa shows time elapsed in seconds. The ordinate shows electrode number; lower-numbered electrodes towards the top (i.e., more basal/higher pitched) and higher-numbered electrodes towards the bottom (more apical/lower pitched). The pitch sequences illustrated in these examples are therefore downward in the left-hand panels and upward in the right-hand panels. A Increments with 30-ms onsets; e7 then e11. B Increments with 970-ms onsets; e15 then e11. C Decrements with 30-ms onsets; e7 then e11. D Decrements with 970-ms onsets; e15 then e11.

Task and responses

The task was to attend to each complex sound and listen for two successive pitches, one higher than the other, occurring during the longer sound. Listeners were asked to identify the order of the pitches, i.e., whether the first pitch was higher (falling pattern) or the second pitch was higher (rising pattern). Assuming that the two pitches elicited differed clearly, this provided a measure of detection of the change applied to two of the electrodes and hence of the concurrent perceptual segregation of the stimulation on these electrodes from that on the others (equivalent to the background complex for NH listeners). It should be noted, however, that the percepts conveyed by electrical stimulation do not meet a strict definition of musical pitch but rather are probably more akin to pitch height or timbral brightness. Indeed, Moore and Carlyon (2005) suggested that a change in place of electrical stimulation for a CI listener may be similar to the pitch change experienced by NH listeners responding to a shift in the spectral shaping of noise components. This point is considered further in the “Discussion”.

The purpose of the experiment was explained to the listeners; they were instructed both verbally and in writing. Each listener was seated in front of a custom-built patient response box on which one button (marked S) was illuminated; they were instructed to press the lit button to hear the next sound, which was presented 500 ms after the button was pressed. After each sound, they responded by pressing one of two buttons (marked 1 and 2) to indicate whether the first or second pitch was higher, respectively. Having responded, the S-button lit up again ready for the next sound. This continued until the experimental run was complete. All listeners received training and practice on the task prior to the experimental runs. The experimental software recorded each response and whether it was correct or not, but no feedback was provided to listeners on the accuracy of their responses.

Procedure

A single-interval forced-choice design was used. Increments and decrements were tested separately; increments were always tested first because pilot testing showed that listeners found these stimuli much easier to detect. Larger increment sizes (in part 1) and decrement sizes (in part 2) were tested first as listeners were able to detect them more easily. In each case, testing only progressed to the next smaller increment (or decrement) when listeners reported that they could hear pitch changes for the current stimulus size. In each experimental block, the sequence of electrodes on which the increments/decrements were applied was randomized between four possible orders: LM, ML, MH, or HM (e7 = H, e11 = M, e15 = L; equal numbers of each order per block). Stimulus presentation was via the “Spear” experimental speech processor. Listeners’ own speech processors were not used.

The experimental block for the increment condition consisted of ten repetitions of each possible order, for each combination of onset time (30 or 970 ms) and increment size (50%, 25%, or 12.5% DR). Each data point for each listener was derived from one block of trials, and so represents the percentage of correct responses for 40 trials (i.e., quantized in steps of 2.5%). The experimental block for the decrement condition was created in the same way, substituting decrement size for increment size. All listeners were tested initially with training stimuli in which the steady-state level for all five electrodes was set at T level, and increments of 100% DR were applied. This provided familiarity with the task, using stimuli that were easily distinguishable owing to the (effective) absence of background stimulation. Listeners only proceeded to the experiment itself when they were able after training to achieve reliably near-perfect performance in this condition.

Results

Increments

Mean results for all eight listeners are shown in Figure 2. For the 50% DR condition, mean scores were significantly above chance for both 30-ms onset increments (fast rise time) and 970-ms onset increments (slow rise time); 95% confidence intervals below the means were 66.1% and 63.6%, respectively. A similar trend was evident for the smaller increment sizes, and this effect approached significance for the 25% DR condition (95% confidence intervals below the means were 46.3% and 47.1%, respectively).

FIG. 2.

Mean performance for each combination of increment size and onset time (n = 8). Dashed lines indicate chance level and perfect performance. Black and gray columns correspond to 30- and 970-ms onsets, respectively. NB: for the purposes of illustration, a value of 50% was inserted into the calculation of means where individual data were absent (see Fig. 3), and chance-level performance can be assumed. Intersubject standard errors (indicated by vertical lines above the means) were calculated only where data were available from at least half of the listeners. This criterion was met in all cases.

A two-way within-subjects analysis of variance (ANOVA) showed a significant main effect of increment size on the accuracy of pitch-change judgments; larger increments were associated with greater accuracy [F(2, 10) = 7.85, p = 0.009]. However, the main effect of onset time failed to reach significance [F(1, 5) = 2.94, p = 0.147], and there was no significant interaction between onset time and increment size [F(2, 10) = 0.396, p = 0.683]. Listeners L18 and L19 were unable to complete the task in the 12.5% DR condition, and so their data for that condition were not included in the analysis. The results for one listener (L13, see below) were quite different from those of the others, and so the ANOVA was repeated excluding their data. This analysis confirmed that the main effect of increment size was the only significant factor. As a final precaution, with the intention of reducing the heterogeneity of variance associated with percent correct scores, the ANOVA was also repeated using arcsine-transformed data. Again, the results confirmed the outcome of the original analysis.

Individual results for all eight listeners are shown in Figure 3. Considerable variability was apparent; only four out of the eight listeners showed performance well above chance for the largest increment size (50% DR), and performance generally deteriorated for the smaller increments. Only two listeners (L12 and L16) showed an apparent advantage for a fast rise time for all three increment sizes. Two others (L18 and L19), who were unable to perform the task at all for the smallest increment size, showed a reversal of the expected pattern in relation to rise time, with nominally better performance for 970-ms onset increments. Results for one listener (L13) differed markedly from the others; she showed near-perfect performance for both onset times at all increment sizes. For her, this near-ceiling performance may have obscured any underlying effect of rise time. Results of further testing of this exceptional listener are described below. Overall, the results suggest that the majority of listeners were only able to perform above chance with increments of 50% DR; mean performance fell quickly towards chance with smaller increments.

FIG. 3.

Performance of individual listeners for each combination of increment size and onset time. Dashed lines indicate chance level and perfect performance. Black and gray columns correspond to 30- and 970-ms onsets, respectively. Absent columns are where listeners were unable to hear any pitch changes and so were unable to continue with the task.

Decrements

Mean results for all eight listeners are shown in Figure 4. Performance for 30-ms onset decrements (slow rise time) was very close to chance for all three decrement sizes. Mean accuracy for 970-ms onset decrements (fast rise time) was somewhat above chance for all three decrement sizes; this effect approached significance for the 50% DR and 25% DR conditions, for which data were available from eight and four listeners, respectively (95% confidence intervals below the means were 46.3% and 48.9%). For the smallest decrement size (12.5% DR), data were only available for two listeners and so statistical analysis was not attempted.

FIG. 4.

Mean performance for each combination of decrement size and onset time (n = 8). Dashed lines indicate chance level and perfect performance. Black and gray columns correspond to 30- and 970-ms onsets, respectively. NB: for the purposes of illustration, a value of 50% was inserted into the calculation of means where individual data were absent (see Fig. 5), and chance-level performance can be assumed. Intersubject standard errors (indicated by vertical lines above the means) were calculated only where data were available from at least half of the listeners. This criterion was met in all cases except for the smallest decrement size.

Analysis of variance was not attempted in view of the overall amount of missing data across conditions, but two pairwise comparisons were made. For the largest decrement size (50% DR), data were available for both onset times for seven listeners. The mean score was higher for 970-ms onset decrements (fast rise time) than for 30-ms onset decrements (slow rise time), but this difference did not quite reach significance (10.7%; p = 0.085). For the 25% DR condition, data were available for both onset times for only four listeners; again, the mean score was higher for the fast rise time, but the difference was not significant (8.0%; p = 0.275). There were too few observations for an equivalent assessment of the 12.5% DR condition.

Individual results for all eight listeners are shown in Figure 5. It would appear that at least five of the eight listeners were unable to perform above chance even in the 50% DR condition. Furthermore, four out of eight listeners were unable to perform the task at all in the 25% DR condition (i.e., they responded entirely randomly and further testing was discontinued), and six out of eight were unable to perform at the smallest decrement size used (12.5% DR). The results for L13 again differed markedly from those for the others; for all three decrement sizes, she showed near-perfect performance for 970-ms onsets (fast rise time) and near-chance performance for 30-ms onsets (slow rise time). L16, and to a lesser extent L12, showed a similar advantage for the 970-ms onset case at the largest decrement size tested (50% DR).

FIG. 5.

Performance of individual listeners for each combination of decrement size and onset time. Dashed lines indicate chance level and perfect performance. Black and gray columns correspond to 30- and 970-ms onsets, respectively. Absent columns are where listeners were unable to hear any pitch changes and so were unable to continue with the task.

Further testing of the exceptional listener (L13)

For both increments and decrements, the results for one listener (L13) were very different from those for the other seven; she appeared to behave more like an NH listener. To explore her performance limits on this task further, she was tested with smaller increment and decrement sizes than were used in the main experiment. The task and procedure were identical to those used before. Increments and decrements of four, two, and one stimulus units were tested. These were equivalent to approximately 6%, 3%, and 1.5% DR (DRs for this listener were fairly uniform across the three target electrodes). For very small changes in amplitude, it should be noted that the difference between nominally slow and fast rise times essentially becomes meaningless. At the limit, an increment (or decrement) of one stimulus unit is a step function with no difference at all between the 30- and 970-ms onset conditions. However, the largest increments and decrements used (four stimulus units) provided a reasonably smooth approximation of the intended amplitude envelopes.

Results are shown in Figure 6. Once again, the performance of L13 with increments was near-perfect for increments of four or two stimulus units, and only slightly reduced (i.e., still well above chance) for an increment of one stimulus unit. As before, no clear effect of onset time was revealed. For the decrement task, her performance for 30-ms onsets (slow rise time) was again relatively close to chance for all three decrement sizes. For 970-ms onsets (fast rise time), however, a clear effect of decrement size was apparent; she produced scores of 100% and 81% correct for decrements of four and two stimulus units, respectively, falling close to chance (47% correct) only for the smallest decrement (one stimulus unit). Thus, the results for L13 were closest to what might have been expected for NH listeners, although any effect of rise time for increments was obscured by her remarkably good performance. She was apparently able to reliably detect decrements associated with rapid level increases (970-ms onsets, four or two stimulus units), almost as well as rapid-onset increments, but was much less able to detect decrements associated with slow rises in level (30-ms onsets, four or two stimulus units).

FIG. 6.

Further results for listener L13 in the increment and decrement conditions. Dashed lines indicate chance level and perfect performance. Black and gray columns correspond to 30- and 970-ms onsets, respectively. Upper and lower panels display the results for increments and decrements, respectively.

It is difficult to be sure why the performance of L13 differed so markedly from that of the other listeners on this task; perhaps she has unusually good neural survival in the cochlea. Further evidence of her exceptional discrimination abilities comes from her open-set speech recognition without lip reading, which was the best out of all eight listeners (100% correct).

Discussion

For NH listeners, a reduction in the amplitude of one component within a complex tone, followed by a sudden return to its original level, can cause the pitch of that component to pop out, even though there has been an overall decrease in energy (Kubovy 1981; Kubovy and Daniel 1983). An increase in energy on a single component within a complex tone can have the same effect, particularly if it has a sudden onset (Bregman et al. 1994a). The aim of this experiment was to evaluate the ability of CI listeners to use either a temporary increase in energy or an abrupt increase in level (with or without an increase in energy) on an individual electrode as a cue for the perceptual segregation of the corresponding pitch from a complex comprising simultaneous stimulation on multiple electrodes.

The results indicate that some CI listeners are able to use temporary increases in stimulus level to perform the task, but they do not appear consistently to show the same benefit of suddenness of onset as seen in NH listeners. CI listeners generally require much larger increases in stimulus amplitude than NH listeners, who are able to discriminate amplitude changes as small as 1 dB in a similar task (Bregman et al. 1994a). Moreover, the increase in stimulus amplitude needed for CI listeners to perform this task successfully was generally much greater than typical intensity DLs for CI listeners. This suggests that the performance of CI listeners was limited by their ability to segregate the incremented components from a constant background rather than by their ability to detect the increments per se. Indeed, as discussed below, it is possible that some listeners may have performed the task without actually picking out the target increments from the complex.

Performance for decrements was typically very poor, with the exception of one listener (L13) who behaved more like an NH listener. She showed near-perfect discrimination of increments as small as one stimulus unit. She also showed a clear advantage for sudden offsets in decrements (i.e., rapid amplitude rises) on individual electrodes. Thus, she was apparently able to use either small, temporary increases in energy or (in the absence of an energy peak) abrupt rises in level, as cues for concurrent segregation.

Detection of sudden change in NH and CI listeners

The poor performance of most CI listeners in discriminating decrements in stimulus amplitude, combined with the absence of any clear overall effect of increment envelope, suggests that most CI listeners rely mainly on detecting peaks in energy and are largely unable to use abruptness of change as a cue for concurrent sound segregation. This is very different from the performance of NH listeners (Kubovy and Daniel 1983; Bregman et al. 1994a). Bregman et al. (1994a, b) interpreted the sudden-change effect that they observed in NH listeners as a result of processing being attracted to the frequency band of the target tones more strongly for rapid onsets than for slower ones, owing to the role of sudden changes in signaling the introduction of new sources of sound. Why might abrupt onsets fail to provide an effective segregation cue for most CI listeners? Three possible accounts are considered.

One candidate mechanism for the perceptual pop out associated with sudden onsets is a release from adaptation of the inner hair cells, which causes a transient onset response in the auditory nerve; this physiological effect would not be apparent in CI listeners. However, this explanation is not compatible with the evidence that exceptional CI listeners, such as L13, can discriminate target decrements far better when they are associated with rapid rise times. Another possible explanation concerns the limited spectral resolution available to CI listeners (e.g., Throckmorton and Collins 2002; Fu and Nogaki 2005). Specifically, it might be that the usefulness of onset cues on the target electrodes is largely eliminated because of spectral smearing through spread of excitation from those electrodes receiving the steady background stimulation. While this account cannot be ruled out entirely, it seems unlikely to provide a complete explanation given that a wide spacing of test electrodes was used and that the open-set speech recognition score for seven of our eight listeners was above 50%. Consistent with this view are the results of Bregman (1994b), who reported a study of NH listeners related to that of Bregman et al. (1994a) but using much more closely spaced harmonics. Bregman et al. (1994b) found that more abrupt onsets increased the clarity of pitches of individual pure tones in a cluster of four overlapping tones even when they were separated from one another by less than one equivalent rectangular bandwidth (Glasberg and Moore 1990).

A third possibility emerges from considering the implications of our results for the other experiment reported by Bregman et al. (1994a). The main difference from the one already discussed is that the targets were pure tones embedded in noise rather than in a harmonic complex tone. Specifically, two 1-s pure tones were presented in succession (an adjacent pair from the set 1,000, 1,500, and 2,000 Hz), in the middle of a 3.5-s burst of noise (band-pass-filtered with lower and upper cutoff frequencies of 500 and 2,500 Hz); signal-to-noise ratios were 1, 2, or 3 dB. The rise times of the tones were 30, 90, 270, 730, 910, or 970 ms. Again, listeners reported the order of the two tones by indicating whether they heard the pitch pattern as falling or rising. In contrast to the results obtained using a complex-tone background, only a small and inconsistent decline in discrimination was found for longer rise times.

Bregman et al. (1994a) argued that the effect of rise time was far smaller in a background of noise because onset responses in the auditory system are triggered not only by the target tones but also by the fluctuations inherent in the noise. However, the background for the targets used in our experiment with CI listeners was constant, not fluctuating, and yet in most cases we also observed little or no effect of rise time. This raises the possibility that Bregman et al.’s results for the noise background arose from its lack of tonality rather than from its inherent fluctuations. Two dimensions of musical pitch can be identified – (1) pitch height, which for a pure tone increases monotonically with frequency, and (2) pitch chroma, which is related to note class within the octave. As already noted, the sensations associated with stimulation of a single electrode are probably more akin to pitch height or timbral brightness than to pitch chroma (Moore and Carlyon 2005). Indeed, this probably relates to the poor perception of music commonly reported by CI listeners, particularly of melody (McDermott 2004).

There is evidence for NH listeners that only pure tones below 5 kHz have pitch chroma. For example, melody recognition becomes poor when the notes of a tone sequence are transposed in frequency to above 5 kHz (Attneave and Olson 1971), but listeners can still discriminate which one of a pair of these notes is higher in pitch. This abrupt change in performance probably relates to the loss of phase-locking cues in the auditory nerve (Rose et al. 1968). In principle, therefore, our conjecture might be tested in NH listeners using stimuli similar to the harmonic complex used by Bregman et al. (1994a) but where the components are shifted in frequency to above 5 kHz. If the absence of pitch chroma for the background components is the basis of the weak effect of rise time seen in most CI listeners, then this transposition should produce similar effects in NH listeners.

Whatever the reason why the onset cue could not be used effectively by most of our listeners, the consequence was that they generally found it difficult or impossible to segregate the target components from the background. How then were they able to perform the task at all? We propose that the increments required by our listeners to perform the task successfully were large compared with typical intensity DLs for CI listeners because they relied mainly on changes over time in the timbral brightness of the whole stimulus rather than on the perceptual segregation of the target stimulation from the background stimulation. For example, stimulus configurations HM and ML might each convey a sensation of descending pitch because, in both cases, the spectral centroid of the complex is lower during the presentation of the second target increment than during the first.

Other cues for concurrent segregation in CI listeners

A difference in pulse rate is a cue specific to CI listening that might in principle be used for simultaneous grouping. Indeed, CI listeners are able to discriminate pulse rate on single electrodes up to a frequency limit of around 300 Hz. However, Deeks and Carlyon (2004) suggested, based on the results of their study using simulations of CI listening, that differences in pulse rate were unlikely to be useful for the segregation of concurrent sounds. They found that although speech recognition performance could be improved by presenting target speech at a higher pulse rate (140 pps) than masker material (80 pps), compared with when they were both presented at the same rate (140 pps), performance was actually worse when the pulse rates for target and masker were reversed. They also found that presenting the target speech on only the odd-numbered channels of a six-channel simulation and the masker speech only on the even-numbered channels (or vice versa) gave worse performance than presenting both of them on all channels.

The role of pulse rate differences in concurrent sound segregation has been evaluated directly in CI listeners in a recent study (Carlyon et al. 2007). This study also evaluated the role of onset time differences as a cue for simultaneous grouping in CI listeners; this cue is often considered to be the most effective commonly available to NH listeners (Bregman 1990). Five listeners (implanted with the Nucleus CI24 device) were presented with a 400-ms duration “mixture” created by simultaneous stimulation of four electrodes, each separated by four electrodes from its neighbor, distributed across the electrode array. The actual electrodes used varied between listeners, but for each listener the same set of four electrodes was used throughout. Stimulation on three out of the four electrodes in the mixture was at a fixed percentage of the listener’s DR; the target electrode (always one or other of the two inner electrodes) was stimulated at a higher percentage of the DR. Each mixture was preceded, 200 ms earlier, by a 400-ms probe on one or other of the two possible target electrodes. On each trial, one interval contained a mixture preceded by a probe identical to the pulse train on the target electrode in the mixture. In the other, the probe was a pulse train with identical characteristics but applied to the other possible target electrode. Listeners had to identify which interval contained the mixture in which the probe was more clearly present (the mixture was actually identical in both intervals).

In a baseline condition, only the increased level on the target electrode distinguished it from the other three sounds in the mixture. Three other conditions evaluated the additional beneficial effects of: (a) a difference in pulse rate on the target electrode compared with the other electrodes in the mixture (77 vs. 100 pps); (b) an onset delay of 200 ms applied to the pulse train on the target electrode and to the preceding probe (NB: stimulation on all electrodes ended at the same time; the probe and target were both 200 ms long); and (c) a pulse-asynchrony condition in which the pulses on the target electrode were delayed by 5 ms relative to the other electrodes in the mixture. If any of these manipulations helped to segregate the target electrode from the rest of the mixture, performance should be better in that condition as the probe should be more salient in the four-electrode mixture. No feedback was provided to listeners during the test procedure.

Carlyon et al. (2007) found a small but significant beneficial effect of onset delay on performance, although the actual improvement (expressed in higher percent correct score) compared with the baseline condition was modest and below ten percentage points in four out of the five listeners (NB: listeners were able to perform above chance in the baseline condition because of the increased level applied to the target electrode; for example, one listener’s score in the baseline condition was around 88% correct and increased to around 95% correct in the onset-delay condition). There was no significant beneficial effect of pulse rate difference or of pulse asynchrony. Two listeners took part in a further experiment in which no increased stimulus level was applied to the target electrode, so that in the absence of any additional cue, performance would be expected to be at chance level (50% correct). Both listeners again showed performance above chance in the onset-delay condition, although for one listener the effect was quite small; their score was just under 60% correct.

Carlyon et al. (2007) again found a small benefit of a 200-ms onset delay in an analogous task using NH listeners. In this case, instead of just four stimuli quite well separated in frequency, as used with the CI listeners, the stimuli used were a mixture of harmonics 5 to 40 of a 100-Hz F0; the onsets of harmonics 13-17 or 26-30 were delayed by 200 ms in a 400-ms-long stimulus. Harmonics above the 12th were chosen to ensure that the individual components were not resolved (Shackleton and Carlyon 1994). They also showed that introduction of a spectral gap between the target and nontarget bands greatly enhanced performance. Based on this, they hypothesized that overlap of electrical stimulation caused by spread of excitation in the CI listeners had greatly reduced the benefit of the stimulus onset delay in concurrent sound segregation.

Conclusions

Overall, our results suggest that CI listeners can use changes in level to pick out components from a complex tone, but generally these must be large compared with those required by NH listeners. Indeed, it is possible that some of our listeners performed the task by monitoring changes over time in the timbral brightness of the whole stimulus, rather than by segregating the target components. For increments, performance depended primarily on the above-baseline stimulation of the target electrodes, rather than on rise time. Performance for decrements was typically very poor, although one exceptional listener performed very well when the decrements were accompanied by a rapid rise in amplitude back to baseline.

The study by Carlyon et al. (2007) suggested that onset asynchrony may be available to CI listeners as a cue for concurrent segregation; however, the effect they reported was small and they found no evidence at all that differences in pulse rate, or pulse asynchrony, could be used for simultaneous grouping. The results of our study further suggest that, for most CI listeners, a sudden rise in energy from (increments) or back to (decrements) a steady background is ineffective as a segregation cue for target components. Thus, the evidence so far suggests that most CI listeners may have access to few, if any, of the cues for concurrent sound segregation that NH listeners use in difficult listening situations with mixtures of competing sounds. Some recent studies have drawn a similar conclusion in relation to sequential grouping by CI listeners (Cooper and Roberts 2007, 2009; Cooper 2008).