Sound source segregation by goldfish: Two simultaneous tones

Abstract

The perception of two simultaneous tones was investigated in goldfish using classical respiratory conditioning and a stimulus generalization paradigm. Pairs of tones were used to make up a mixture of 150 Hz and a higher harmonic or a mistuned harmonic. Fish were conditioned to the two-tone mixture and then tested for generalization to several pure tones. The simultaneous tones tended to be segregated in perception, with the generalization gradient for single tones having two peaks corresponding to the frequencies of the tone pairs. There were no consistent differences in the generalization gradients following conditioning to harmonic or inharmonic tone pairs. In addition, experiments were carried out in which the two tones of the pair were heard on alternate trials, always as single tones, followed by generalization tests to single tones. There was more generalization in this experiment, reflecting the fact that conditioning and generalization test stimuli were both single tones. However, the shapes of the generalization gradients were similar to those in which fish were conditioned to two simultaneous tones, indicating that the simultaneity of the tones did not make them harder to segregate. As the frequency separation between the two components narrowed, segregation tended to fail.

INTRODUCTION

In 1964, van Bergeijk (1964) asked the following question about the structure of a fish’s auditory experience: “Given that a fish can discriminate between two sounds A and B when they are presented separately, can he still discriminate either one when both are presented simultaneously? Or do the two sounds blend to form a new entity (such as a chord)?” (p. 296). At that time, the concepts of analytic and synthetic listening, sound source segregation, and auditory scene analysis were not yet defined, so van Bergeijk’s question seems prescient as well as intelligent. But neither were there obvious methods for answering such a question, and, in fact, there was a general bias against the notion that a fish had any “auditory experience” at all. van Bergeijk (1964) went on to state that if fishes were not able to segregate source A from source B, then vocal communication among fishes would be of very limited use. Actually, it would seem that if fishes were not able to segregate sources, hearing itself would be of very limited use.

The source segregation question for fishes has recently been answered in the affirmative several times, both with respect to single-tone components (Fay, 1992) and more temporally and spectrally complex sources (Fay, 1998, 2000). But unanswered questions remain. For example, first, what is the role of harmonicity? In general, harmonic relationships play a role in human source segregation, and this formed an element of van Bergeijk’s question when he suggested that the two sources might “blend” to form a chord. Second, what are the limitations of source segregation with respect to frequency differences? Fay (1992) demonstrated that goldfish behaved as if they had segregated two simultaneous tones of 166 and 724 Hz, but it is not known how the animal performs in segregating more closely spaced tones. The 166–724 Hz segregation could have essentially a simple peripheral explanation in that these two frequencies activate differently tuned auditory nerve channels with center frequencies near these frequencies (Fay and Ream, 1984; Fay, 1997). What happens to segregation, and is it still possible when the two components fall essentially within the same channel? These and other questions were the motivations for the present experiments. The segregation of two simultaneous components was evaluated with respect to the harmonic or inharmonic relations between the components, and as a function of the frequency separation between a low-frequency component (150 Hz) and the higher harmonic component to be possibly segregated.

MATERIALS AND METHODS

Animals

The subjects were 96 common goldfish (Carassius auratus), about 8 cm in standard length, maintained in communal aquaria for from 2 weeks to over 1 year. The fish were obtained from commercial suppliers.

Acoustics and stimuli

The experimental test chamber was a water-filled plexiglass cylinder 23 cm in diameter and 28 cm high. A University Sound UW-30 underwater pool speaker projected upward from the bottom of the tank, and was buried in water-saturated sand with the speaker diaphragm about 2 cm below the sand surface.

In general, the acoustic conditioning and test stimuli were either pure tones between 50 and 900 Hz, or two simultaneous (mixed) tones consisting of 150 Hz plus a higher component at either harmonics of 150 Hz (300, 450, 600, or 750 Hz) or at near or mistuned harmonics (310, 460, 610, or 760 Hz). The two-tone mixtures were called harmonic when the higher frequencies were integer multiples of the 150 fundamental frequency component and were called inharmonic when the harmonics were mistuned by 10 Hz. In all cases, the tones were presented at a level about 40 dB above absolute threshold for the component(s) (Fay, 1969). During conditioning (40 trials), the level of the tone(s) was varied from trial to trial by ±10 dB. During generalization testing, all stimuli were presented at a constant 40 dB sensation level.

In all experiments, conditioning and test stimuli were 6 s duration signals with 20 ms rise-fall times synthesized in advance, stored on disk, and read out of a 16 bit digital-to-analog converter (DAC1) from Tucker Davis Technologies (TDT) at 5 kHz. The DAC1 output was low-pass filtered at 1500 Hz, led to a TDT programmable attenuator, and then to a Crown 100 W power amplifier and the UW-30 loudspeaker.

The acoustic signals were measured by a Bruel and Kjaer 8103 calibrated miniature hydrophone placed in the fish restrainer (described below). The hydrophone output was amplified, bandpass filtered between 10 and 2500 Hz, and digitized at 5 kHz. Samples were spectrally analyzed using the fast Fourier transform. Levels (in decibels with regard to 1 μPa) at selected frequencies were also measured using an HP 3040A wave analyzer. The experimental tank rested on a vibration-isolated limestone slab inside an Industrial Acoustics single-walled audiometric booth.

Experimental design

A total of 12 experiments were carried out, with eight animals per experiment. In each experiment, animals received 40 conditioning trials, and then 40 generalization test trials without shock, but in which every fifth trial was the conditioning stimulus terminated by shock. These reinforced trials served to maintain the conditioned response in the face of the generalization testing procedure that tended to extinguish the conditioned response. In experiments 1–4, animals were conditioned to a two-tone mixture and tested for generalization to eight single tones between 50 and 900 Hz. The two-tone mixtures always had a 150 Hz, fundamental frequency component plus a higher, harmonic frequency component at 300, 450, 600, or 750 Hz. Experiments 5–8 had identical stimuli except that the higher of the two conditioning components were mistuned with respect to perfect harmonics by 10 Hz (310, 460, 610, or 760 Hz). A comparison of generalization behavior between experiments 1–4 and 5–8 helped to evaluate the role that harmonicity might play in two-tone segregation. In experiments 9–12, the conditioning stimuli were two harmonic tones presented alternating with one another on successive trials, rather than simultaneously. Generalization testing stimuli were as in experiments 1–4. This last group of experiments helped to evaluate what was to be expected if the simultaneous tones in experiments 1–4 were perceived as segregated as possible.

Conditioning and generalization testing

For respiratory conditioning, a goldfish was restrained in a cloth bag about 2 cm from the water surface, centered in the test tank. Slits in the bag allowed respiratory movements of the gill covers and mouth. A thermistor near the mouth measured respiration. Water flow cooled the thermistor producing a fluctuating voltage proportional to respiration. This waveform was filtered between 1 and 4 Hz and digitized at 5 kHz. Respiratory activity was calculated by summing the piecewise lengths of the respiratory waveform in arbitrary units minus the length expected without respiratory activity (i.e., a flat line) over the same period of time. The response (suppression ratio) during a 6 s conditioning or test stimulus was defined as the ratio of the respiratory activity during the last 4 s of the stimulus to the sum of the latter and the respiratory activity 4 s preceding the stimulus. Complete respiratory suppression results in a suppression ratio (SR) of zero, and no change in respiration results in a SR of 0.5.

Respiratory suppression lasting several seconds was an unconditioned response to a 100 ms ac electric shock delivered through steel electrodes placed near the animal’s head and tail. A single conditioning trial included a 4 s pretrial period during which respiration was measured, and a 6 s stimulus presentation that terminated with the shock. Conditioned responses to sound tend to occur after 10–15 trials.

Animals were conditioned and tested in two separate sessions that were between 30 min and 24 h apart. A conditioning session consisted of 40 conditioning trials (random intertrial intervals averaging 3 min). Generalization testing consisted of 40 test trials (average intertrial interval of 1.5 min) of eight test stimuli presented four times each in random order without shock. The conditioning stimulus was presented with shock every fifth trial in order to maintain the conditioned response. Shock voltage was adjusted occasionally in order to maintain the conditioned response. Often, conditioning and testing was suspended for 20–40 min in order to aerate the test tank water and maintain slow respiration. Animals failing to show robust, sustained respiratory suppression during the conditioning period were eliminated as subjects.

A median SR was calculated for each of the stimuli presented four times during each generalization testing session. Generalization was normalized with respect to the median SR to the conditioned stimulus measured during the test session (eight trials) and expressed as a percentage. Percentage generalization was defined as ((0.5−med_T)∕(0.5−med_C))×100, where med_T is the median SR to the test stimulus and med_C is the median SR to the conditioning stimulus obtained from eight conditioning stimulus presentations (with shock) during generalization testing. Generalization values above 100% occurred when suppression to a test stimulus was greater than that to the conditioning stimulus. Values below 0% occurred when the SR was greater than 0.5, indicating an acceleration of respiratory activity during the stimulus.

The care and use of animals in this work was approved by the Institutional Animal Care and Use Committee of Loyola University Chicago, and was supported by the NIH NIDCD Grant No. 1 R01 DC005970 “Sound source segregation and determination,” R. Fay P.I.

RESULTS

The group mean generalization gradients for all 12 experiments are shown in Fig. 1. As an example, Fig. 1a shows the gradients for animals conditioned to 150 and 750 Hz (harmonic)—filled squares, animals conditioned to 150 and 760 Hz (inharmonic)—filled triangles, and animals conditioned to 150 and 750 Hz tones alternating on successive trials—open circles. These three groups were always tested identically using single tones. All groups produced the greatest response at 150 and 750 Hz, suggesting that these two components had been segregated in perception. The maximum amount of generalization for the two groups conditioned to the tonal mixtures reached about 60%, indicating that no single tone used in generalization testing was fully equivalent to the two-tone mixture on which the fish were conditioned. The gradient for the inharmonic test stimuli is essentially identical to that for the harmonic group, suggesting that source segregation for tones does not depend on strict harmonic relationships, at least for these two most widely spaced tones. The gradient for the group conditioned to the two tones presented successively on alternate trials is generally higher (reaching 100%) than for the other two groups, indicating that the single test tones were judged to be essentially equivalent to the single tones used in initial conditioning. There are no other obvious differences between the generalization behavior of the simultaneous and alternating conditioning groups.

Figure 1.

Group mean generalization gradients for 12 independent groups of eight animals conditioned to two tones. (A) 150+750 Hz, (B) 150+600 Hz, (C) 150+450 Hz, and (D) 150+300 Hz. For all panels, filled squares: two-tone, harmonic complexes; filled triangles: two-tone mistuned complexes; and open circles: two single tones presented on alternate trials. Vertical error bars indicate ±1 standard error at each mean. The two large arrows in each panel indicate the two frequencies making up the conditioning stimuli.

In general, the other panels in Figs. 1b, 1c, 1d can be described similarly. However, as the two-tone separation narrows, the evidence for source segregation also declines. Figure 1d shows that at the smallest component separation (150 and 300 Hz), there is no sign of source segregation. This is due to the limited resolution of the generalization test and the limited resolution of generalization behavior as a sign of source segregation (see Sec. 4). However, the amount of generalization after conditioning to the tonal mixtures is higher in this group (80%–90%), suggesting that the response to the two-tone complex (150 and 300 Hz) was nearly equivalent to the response to the single tones used in the generalization test. Also notice that at this small separation, there is still little sign of the influence of harmonic versus mistuned components on generalization behavior, and no obvious differences between the gradient shapes for simultaneous or successive tone conditioning.

The degree of tone source segregation as revealed by the generalization gradients can be estimated using a peak-valley contrast metric. Contrast was defined as P-T, where P is the average generalization response at the two frequencies used in conditioning and T is the lowest single generalization value that fell between the two peaks. Figure 2 shows this contrast for the four groups and indicates that contrast declines approximately equally and linearly as the frequency separation between the two components declines for both the harmonic and mistuned groups and regardless of whether the goldfish were conditioned to simultaneous or successive tones.

Figure 2.

The degree of segregation indicated by a contrast metric. Contrast (see text) as a function of the frequency of the higher harmonic. Filled squares: two-tone, harmonic complexes; filled triangles: two-tone mistuned complexes; and open circles: two single tones presented on alternate trials.

DISCUSSION

These results are generally consistent with the results of a previous experiment (Fay, 1992) on tonal source segregation in goldfish. This and the earlier experiment suggest that two-tone mixtures are naturally analyzed by the goldfish into their individual tonal components and do not tend to be perceived synthetically as a unique chord or auditory “chimera.” Rather, the goldfish auditory system tends to operate analytically and forms a “scene” made up of the collection of individual auditory components or sources. This ability to segregate sources, even those with no obvious biological significance, is presumed to be an adaptation of all auditory systems for analyzing the soundscape for general orientation purposes (Slabbekoorn and Bouton, 2008).

The generalization gradients for the simultaneous and successive groups are substantially similar, indicating that the simultaneous occurrence of the two components did not affect or reduce the source segregation behavior to a greater degree that occurs when the components are heard separately or as single tones. Therefore, there appears to be little or no effect of simultaneity itself on source segregation as measured by this generalization procedure. The frequency selectivity revealed by these generalization experiments seems to be quite acute, and the results are generally consistent with previous generalization experiments using single tones (Fay, 1969, 1970a, 1992).

In order to compare single-tone generalization behavior with the present gradients after conditioning to mixtures, Fig. 3 was constructed. The present gradients for the harmonic mixtures (filled triangles connected by lines) are plotted along with results abstracted from the single-tone gradients (Fay, 1970a, see also Fig. 4). Since the exact frequencies used presently (150, 300, 450, 600, 750 Hz) were not tested with single tones previously, a procedure was developed to predict what these generalization gradients would likely have been if centered on the frequencies used in the present experiments. First, the upper and lower frequencies of each single-tone generalization gradient corresponding to 50% generalization were measured and were plotted as a function of center frequency. The frequencies at 50% generalization corresponding to the present center frequencies were determined by linear interpolation. Then the predicted gradients were plotted in Fig. 3 as three points—the interpolated lower and upper frequencies at 50% generalization, and the center frequency. It can be seen that for the two widest tone separations (panels A and B), the lower leg of the 150 Hz gradients and the upper leg of the higher harmonic gradients are coincident with those for single tones and the present two-tone mixtures. At test frequencies between the two components, the mixture gradients lie well above the single-tone gradients. For the two narrowest frequency separations (panels C and D), there is more excess generalization, especially above the center frequencies of the higher, single-tone gradients. Thus, we conclude that for the 150 Hz plus 600 and 750 Hz gradients, the generalization behavior is predicted from the single-tone gradients at frequencies above and below the respective tonal frequencies, but between them generalization is greater than predicted. This means that the two-tone mixtures were well analyzed, but this analysis tends to be degraded or uncertain at frequencies between the two mixed tones. As noted above, this uncertainty is not due to the simultaneous presentation of the two tones, but rather with the performance in recalling two, rather than one, tone in the generalization test. For the other two gradients (panels C and D), analysis of the mixtures generally declines and fails completely for the 150+300 Hz gradients. The failure to demonstrate segregation in these cases is due, at least in part, to the combination of the limited resolution of the generalization procedure as a definition of source segregation and the limited resolution of the frequencies chosen for the generalization test.

Figure 3.

Generalization gradients for harmonic complexes (filled triangles) along with predictions for generalization gradients to single tones (the two, inverted “V” shapes in each panel—see text). (A) Generalization gradient following conditioning to 150+750 Hz with predicted single-tone gradients to these same frequencies. (B) 150+600 Hz, (C) 150+450 Hz, and (D) 150+300 Hz.

Figure 4.

Single tone generalization gradients for goldfish replotted from Fay, 1970a on a linear frequency scale.

Figure 4 plots the pure tone generalization gradients that have been obtained from goldfish previously (Fay, 1970a). These gradients are remarkable in suggesting that frequency analysis is very acute, and that it deteriorates monotonically with frequency between 40 and 1600 Hz. For example, the left-most gradient indicates that a 5 Hz deviation from the 40 Hz conditioning frequency results in a significant failure to generalize. Not only is this a discriminable change of frequency (Fay, 1970b) but it is a meaningful one to the goldfish in that it easily controls behavior. What are the mechanisms for this high degree of frequency selectivity, and for the degree of source segregation revealed by these generalization gradients? In other words, what evidence permits the goldfish to decide that there are two sinusoids that make up the mixture stimuli used here, or that a 100 Hz sinusoid has little in common with a 150 Hz sinusoid (see Figs. 45)?

Figure 5.

The gradients of Fig. 4 (dotted lines) along with the two fundamental filter shapes determined for single cells of the goldfish saccular nerve using the revcor method (Fay, 1997). The low-frequency filter (lighter solid line) and the high-frequency filter (heavier solid line).

The first hypothesis that comes to mind is that the segregation behavior is based on peripheral filters (the tuning curves of primary afferents). This hypothesis, in its simplest form, can be quickly rejected. Fay (1978a, 1997) and Fay and Ream (1986) investigated the frequency selectivity of saccular afferents in goldfish and concluded that the hearing range is divided into 2–4 wide-band frequency channels. Figure 5 shows the two major peripheral frequency channels revealed using the reverse correlation (revcor) method in goldfish (Fay, 1997), superimposed on the generalization gradients of Fig. 4. These two channels are quite wide and overlapping, and cannot account in any simple way for the behaviorally determined frequency selectivity, especially at the lowest frequencies. Thus, the hypothesis that the two-tone source segregation behavior of goldfish (Fay, 1992) is simply based on the selective and independent analysis of the two mixed components in peripheral channels can also be rejected.

The remaining possibilities include that this frequency selectivity and source segregation behavior are a result of computations in the brain, begun either with the place-coded peripheral selectivity of Fig. 5 or with the temporally coded waveform, presumably based on phase-locking in the periphery. Computations based on the timing of spikes in the auditory nerve is the hypothesis that has received the most experimental and theoretical attention (e.g., Fay, 1978b; De Cheveigné, 2005).

The temporal coding hypothesis is relevant to the present question of the effects of harmonicity on source segregation. There appears to be no apparent effect of harmonicity on segregation: the generalization gradients are substantially similar for harmonically related and for mistuned components. For human listeners, the mistuning of a harmonic within a harmonic complex tends to make the mistuned component stand out as a separate entity (enhances segregation or analytic listening) (Hartmann, 1988). This phenomenon is often discussed in the context of pitch theory; the frequency of a mistuned harmonic is not consistent with the pitch or fundamental frequency of the rest of the complex and thus is perceptually segregated. Goldfish exhibit a sort of pitch perception for harmonic complexes (Fay, 2005) but differ from humans in apparently not demonstrating repetition noise pitch (Yost et al., 1978), and in their perceptions that, for example, a 100 Hz pure tone and a harmonic complex with a fundamental frequency (f0) of 100 Hz have no perceptual elements in common (Fay, 2005). The evidence so far is that the goldfish is well aware of the acoustic spectrum but unaware of a spectrum’s periodicity or internal structure that can determine pitch for human listeners. Guttman (1963) came to a similar conclusion about the visible spectrum after studying stimulus generalization across single wavelengths in pigeons.

It is noteworthy that in Fay, 2005, goldfish did not generalize at all between a single tone of 100 Hz and a harmonic complex with 20 successive harmonics having a f0 of 100 Hz or vice versa. But in the present experiment, there was significant generalization between a two-tone harmonic complex with f(0)=150 Hz and a 150 Hz single tone. We could predict, therefore, that as the number of higher harmonics of a complex increases, the magnitude of generalization to a single tone at the fundamental frequency declines, in spite of the fact that the 100 Hz periodicity of the complex’s waveform becomes better and better defined as more harmonics are added. Apparently, a signal consisting of many successive harmonics is difficult to analyze for the goldfish and is perceived synthetically as quite distinct from a single tone. In any case, the often-stated conclusion that organisms tend to analyze the low-frequency acoustic waveform in the time domain (e.g., Capranica, 1992) may be true to some extent, but fundamental structural features such as waveform periodicity do not appear to strongly determine the resultant perceptions in goldfish. It is not yet clear whether this tentative conclusion applies as well to all fish species or may apply only to hearing specialists (e.g., otophysans) such as goldfish and zebrafish that have a wide hearing range extending to several thousand hertz.

For human listeners, the pitch of complex tones is determined primarily by higher-frequency harmonic components that are not resolved by the peripheral auditory system (the “residue”). Perhaps for goldfish, this unresolved residue (comprised of the fundamental and all audible harmonics) determines aspects of perception, but there may be no pitch or periodicity pattern processor to make sense out of it. Perhaps for goldfish (and other fishes), complex pitch, per se, is not an important aspect of hearing or source segregation.

The present study contributes to this issue by revealing the frequency selectivity of the internal processes (e.g., filtering computations) that perform this segregation. Lu and Fay (1995, 1996) demonstrated the existence of complex and selective tuning curves in the goldfish midbrain (torus semicircularis) and thalamus (central posterior nucleus). Some single-unit tuning curves showed responses to narrow frequency bands in an “island” pattern, where excitatory responses were flanked by suppressive bands, apparently caused by inhibitory interactions. Thus, narrow frequency selectivity of central units appears to be synthesized through excitatory and inhibitory interactions among frequency-selective inputs, ultimately originating in the periphery. Therefore, we tentatively conclude that the segregation behavior revealed here can be best understood in terms of the frequency selectivity synthesized in the brain and ultimately derived from peripheral filtering.

Regardless of the specific mechanisms, however, these experiments indicate an essential similarity between goldfish and human auditory perception. Both species easily segregate many sources in mixtures, and for both species, the spectrum plays a dominant role. For humans and some other vertebrates, the analysis of the spectrum at the periphery seems to be adequate to account for many aspects of source segregation. For goldfish (and possibly other fishes), it appears to be central computations based on a rather crude peripheral frequency selectivity that accounts for the fundamental aspects of source segregation. The absence of a fine peripheral frequency selectivity and tonotopicity in fishes has not been an important limitation on the sense of hearing or its performance in source segregation.