Do we hear what birds hear in birdsong?

Abstract

Peter Marler’s fascination with richness of birdsong included the notion that birds attended to some acoustic features of birdsong, likely in the time domain, which were inaccessible to human listeners. While a considerable amount is known about hearing and vocal communication in birds, how exactly birds perceive their auditory world still remains somewhat of a mystery. For sure, field and laboratory studies suggest that birds hear the spectral, gross temporal features (i.e. envelope) and perhaps syntax of birdsong much like we do. However, there is also ample anecdotal evidence that birds are consistently more sensitive than humans to at least some aspects of their song. Here we review several psychophysical studies supporting Marler’s intuitions that birds have both an exquisite sensitivity to temporal fine structure and may be able to focus their auditory attention on critical acoustic details of their vocalizations. Zebra finches, Taeniopygia guttata, particularly, seem to be extremely sensitive to temporal fine structure in both synthetic stimuli and natural vocalizations. This finding, together with recent research highlighting the complexity of zebra finch vocalizations across contexts, raises interesting questions about what information zebra finches may be communicating in temporal fine structure. Together these findings show there is an acoustic richness in bird vocalizations that is available to birds but likely out of reach for human listeners. Depending on the universality of these findings, it raises questions about how we approach the study of birdsong and whether potentially significant information is routinely being encoded in the temporal fine structure of avian vocal signals.

Birdsong has served as an extremely productive behavioural and neurobiological model of vertebrate learning in general and as a model of human speech development and acoustic communication specifically. But compared to humans, it is fair to say we know considerably more about vocal production in birds and much less about perception of species-specific vocal signals. Furthermore, while there are many parallels in the learning and production of vocalizations between these two communication systems, there are surprisingly few demonstrations that these parallels extend to the perceptual systems of humans and birds. This raises a simple question: does birdsong sound to birds like it does to humans? What we do know about basic hearing in birds comes mostly from psychophysical studies using simple sounds such as tones and noises. From such studies, we know that birds hear best between about 1 and 5 kHz and show discrimination thresholds for changes in frequency, intensity and temporal envelope generally approaching the values typically reported for humans (reviewed in Fay, 1988; Dooling et al., 2000), although species differences in salience sometimes emerge when birds are tested with species-specific vocalizations (Dooling et al., 1992).

Students of ornithology often describe birdsong in terms of its pitch, tempo, complexity, structural organization and stereotypy. Indeed acoustic correlates of these common perceptual dimensions are how we make judgments about whether a song has been learned or altered in some significant way. While we can describe speech in these terms, we usually do not. Instead, when listening to speech, we typically focus on well-learned acoustic patterns, reflexively attending to critical acoustic features necessary for communication. This combination of well-learned acoustic patterns and sharp attentional focus is part of what leads to the notion that speech perception is special for humans. It is possible that these advantages are available to birds. Anecdotal field observations over the years, coupled with well-known differences between birds and mammals in the anatomy and physiology of the peripheral and central auditory systems, has led to speculations that birds must have extremely fine temporal processing abilities (Carr & Friedman, 1999; Greenewalt, 1968; Konishi, 1969; Pumphrey, 1961; Schwartzkopff, 1968).

Zebra finches, Taeniopygia guttata, have become an extremely popular model for studying song learning, bioacoustics and vocal behaviour and are a good species for investigating these phenomena (Braaten et al., 2006; Brainard & Doupe, 2001; Bolhuis & Everaerts, 2013; Elie & Theunissen, 2015). These birds are closed-ended learners that have a single sensitive period for song learning, after which new songs cannot be learned. The result of this sensitive period is a single, highly stereotyped song consisting of an ordering of syllables, termed a motif, that is repeated several times throughout the song bout. Motifs are typically composed of five to eight notes or syllables. Each syllable is an acoustically distinct harmonic complex, which contains multiple cues that result in a unique sound (Zann, 1996). The simple and repetitive nature of these songs has allowed for extensive study of the behavioural and neurobiological basis of song development, song learning and song production (see, for example, Doupe & Konishi, 1991; Brainard & Doupe, 2001; Troyer, 2016; Glaze & Troyer, 2006; Margoliash & Fortune, 1992).

Zebra finch contact calls are some of the most obvious and ubiquitous vocalizations given by these birds in captivity (Blaich et al., 1995) and in the wild (Zann, 1996). Male zebra finches learn their songs and perhaps some aspects of calls (Zann, 1984; Simpson & Vicaro, 1990). Peter Marler himself identified bird calls as an underutilized model for the neurobiology of acoustic communication (Marler, 2004). Indeed, zebra finch calling behaviour is proving more complex than was previously thought. Importantly, not only do male and female zebra finches produce a wide range of acoustically distinguishable calls (Elie & Theunisen, 2015), but these calls occur in distinct social circumstances (Elie & Theunisen, 2015; Gill et al., 2015; Elie et al., 2010; Williams, 2001). Furthermore, variation in calling behaviour between birds can have surprising effects on mating and parental behaviour (Boucaud et al., 2016).

The case to be made here is that a combination of species-specific attentional advantages and acute temporal resolving power in birds could very well put communicatively relevant acoustic details of complex song-like stimuli out of the reach of human hearing. Auditory psychophysical studies in birds that have manipulated attention are rare, but results from one psychophysical study are tantalizing. In this study, brief tonal sequences were concatenated to create a synthetic model of the species-specific contact call of budgerigars, Melopsittacus undulatus. Birds were then tested on frequency changes in these sequences, the location of which was both fixed and random from trial to trial. Results showed that budgerigars attended to the tonal complex as a whole while human listeners did not. This suggests that the influence of top–down attentional processes is available at least to budgerigars, but not to humans, when listening to these call-like tonal complexes (Dent et al., 2000).

For study of temporal fine structure, zebra finches make an excellent species. Zebra finch vocalizations are extremely complex and provide a multitude of acoustic cues for discrimination, including amplitude envelope modulations, spectral structure and temporal fine structure. In considering these features, it is important to distinguish between two types of temporal cues. Most discussions of the temporal features of birdsong focus on the envelope changes. Temporal envelope is a global timing that occurs over many milliseconds to seconds, and accounts for the global rhythm and timing of song including, motif, syllable, and note durations and intersyllable intervals. Fine structure, on the other hand, is a local timing that occurs over milliseconds, and includes amplitude, spectral and temporal cues within individual harmonic syllables. While both temporal envelope and fine structure cues are present in vocalizations, especially in harmonic zebra finch vocalizations, fine structure has historically been ignored in part because it is not apparent in traditional sonographic analysis that has been the mainstay in birdsong research for decades.

Whether birds can actually hear and discriminate temporal fine structure in complex sounds is another matter. Here we discuss research that addressed this issue using synthetic stimuli and three species of birds with very different vocalizations — two songbird species, one of which was an open-ended learner (canary, Serinus canaria) and one of which was a close-ended learner (zebra finch), and a nonsongbird species (budgerigar). The perceptual thresholds for these species were directly compared to those of humans. All three perceptual experiments below used identical standard psychophysical methods: birds were trained by operant conditioning to discriminate a change (a target) against an ongoing, repeating sound (the background). Correct responses were rewarded with food and false alarms were punished with a brief blackout. The fact that birds were all trained and tested using the same behavioural procedures and that psychophysical threshold estimates were obtained using the same method in birds and humans strengthens the comparisons.

DISCRIMINATION OF TEMPORAL FINE STRUCTURE: SCHROEDER HARMONIC COMPLEXES

Some bird vocalizations, like those of the zebra finch, can be described as predominantly complex harmonics (Zann, 1984), making them more difficult to describe and characterize than more tonal or whistled bird vocalizations. Modern signal-processing techniques can be used to manipulate complex harmonic sounds and to create synthetic harmonic models for testing, which can closely mimic some of the natural properties of these harmonic bird sounds. This ability allows for perceptual threshold measurements of the fine details in complex harmonic sounds that escape notice in more traditional sonographic analysis.

Evidence for an extreme sensitivity to temporal fine structure in birds comes from a study looking at the discrimination of Schroeder waveforms (Schroeder, 1970; Dooling et al., 2002). These stimuli were constructed of harmonically related pure tones with the phases of the individual tonal components adjusted so that they were monotonically increasing (positive Schroeder complex) or decreasing (negative Schroeder complex) with harmonic number, resulting in instantaneous frequencies that fell or rose monotonically across each period. Figure 1 shows examples of negative and positive Schroeder-phase waveforms with three different fundamental frequencies. The acoustic differences between members of a pair of these complexes were limited to temporal fine structure: all waveforms had a flat envelope and, within a pair, defined by the fundamental frequency, had identical long-term spectra. These waveforms were 260 ms in duration, including 20 ms rise/fall times. While the envelope and overall spectrum were constant across stimuli to be discriminated, the temporal fine structure was reversed. Test stimuli consisted of seven pairs of these harmonic complexes with fundamental periods ranging from 6.6 ms (fundamental frequency of 150 Hz) to 1 ms (fundamental frequency of 1000 Hz) in duration. Finches, budgerigars, canaries and humans were tested on their ability to discriminate a forward Schroeder complex from a reversed Schroeder complex. Birds were tested on a Go-NOGO task, while humans were tested in a two-alternative, forced-choice task, with values ranging from chance performance at 50% correct to perfect performance (100%), so the bird data were scaled to the range of the human data in Fig. 1 (Dooling et al., 2002).

Figure 1.

Temporal waveforms of three pairs of negative and positive Schroeder-phase harmonic complexes. These harmonic sounds were generated using the Schroeder algorithm to minimize envelope cues (Schroeder, 1970). These Schroeder-phase waveforms have fundamental frequencies of 100 Hz, 200 Hz and 400 Hz corresponding to period durations of 10 ms, 5 ms and 2.5 ms, respectively (from Dooling & Lohr, 2006).

Birds were able to discriminate between positive and negative Schroeder harmonic complexes at fundamental frequencies up to at least 600 Hz. Budgerigars and canaries showed some difficulty discriminating at the highest fundamental frequencies (800 and 1000 Hz), while the zebra finches discriminated easily between the positive and negative Schroeder waveforms even at the highest fundamental frequency of 1000 Hz as shown in Fig. 2. Human listeners begin having difficulty making these discriminations when the fundamental period became shorter than about 3 ms, budgerigars and canaries did much better, and zebra finches had little difficulty even over periods as short as 1 ms. These results show unequivocally that birds have better resolution of temporal fine structure in acoustic stimuli than humans and, among birds, that zebra finches are especially sensitive to these features (Dooling et al., 2002). Interestingly, Belgian Waterslager canaries with an inherited peripheral auditory pathology resulting in poor frequency discrimination and large filter bandwidths also show extraordinarily good temporal resolution, as expected from other studies showing an inverse relationship between bandwidth and temporal resolving power (Lauer et al., 2006; Lauer et al., 2009; Henry et al., 2011).

Figure 2.

Results for zebra finches, canaries, budgerigars and humans tested on positive/negative Schroeder waveform discrimination at different fundamental frequencies ranging from 150 Hz to 1000 Hz (periods of 6.7 to 1.0 ms, respectively). Error bars represent standard errors between subjects. Human thresholds begin to fall to chance levels at fundamental frequencies around about 300 Hz (periods less than 3.3 ms), while zebra finch thresholds remain high at fundamental frequencies up to 1000 Hz (periods of 1.0 ms) (replotted from Dooling et al., 2002).

DISCRIMINATION OF TEMPORAL FINE STRUCTURE IN ZEBRA FINCH SONGS AND CALLS

While zebra finches clearly show an impressive sensitivity to temporal fine structure that does not prove that fine structure changes are more salient than envelope changes in natural song. In the following experiment, zebra finches were trained to discriminate between a normal song motif and either an altered song motif with one syllable reversed (largely a fine structure manipulation) or one with a doubling of the interval between two syllables (an envelope manipulation) as shown in Fig. 3 (Vernaleo & Dooling, 2011). Changes to syllable fine structure in the motif consisted of reversing single syllables in time while keeping the order of syllables in the motif intact, and changes to envelope consisted of doubling in the interval between two syllables. In reversing syllables, the overall level and spectral content remained the same, whereas fine structure was changed. Interval doubling was done by adding the same amount of silence to the interval as the length of the interval itself. Targets consisted of the original motif, with only a single syllable reversed or only a single interval doubled as shown in Fig. 3.

Figure 3.

Example of natural motif with seven syllables (A–G) and where the last syllable has three parts (1–3). This stimulus was played as a background stimulus (top). Two targets are shown below. In one case the duration of the interval between syllables A and B is doubled (left) and in the other case the last syllable G is reversed (from Vernaleo & Dooling, 2011).

Human listeners could easily hear a doubling of the interval regardless of where it occurred in a motif and they reported that it sounded like there was a glitch in the stimulus. But humans usually could not detect a reversed syllable in the motif even with considerable practise. Surprisingly, birds did just the opposite. Birds discriminated single interval doublings (i.e. the envelope change) on average less than 5% of the time and single syllable reversals (i.e. the fine structure change) on average greater than 90% of the time. Birds were much better at discriminating single syllable reversals than they were at discriminating single interval doublings, regardless of the song motif (Vernaleo & Dooling, 2011). In some ways this is consistent with Glaze and Troyer (2006), who examined the durations of intervals and syllables in many renditions of a set of songs. They found that the coefficient of variation was larger for intervals than it was for syllables and that tempo changes in song affected the durations of intervals more than it did syllables. In other words, when songs were sped up or slowed down, the intervals tended to stretch and compress, whereas syllable durations were more stable. Perhaps because intervals are normally sung with some amount of variability, changes to interval duration are not particularly salient to birds (Glaze & Troyer, 2006). Given the harmonic nature of these zebra finch syllables, it is likely that birds are relying on temporal fine structure in making their discriminations since duration and overall spectral cues are the same for forward and reversed syllables, although clearly these syllables also provide spectral cues that change over the duration of the syllable.

Is it just the temporal fine structure in their vocalizations that zebra finches are using in making discriminations? A more definitive test, and a much closer parallel to the Schroeder experiment described above, used variations in the contact or long call of female zebra finches. The long call of the female zebra finch is a harmonic complex with a fundamental typically around 600–700 Hz (a period of 1–2 ms). It is quite easy to create a synthetic version of this harmonic complex by adding together, in phase, a set of harmonically related pure tones. Mistuning one of the harmonics, ever so slightly, creates myriad interesting changes including spectral changes and changes in the time waveform, specifically in the temporal fine structure. Earlier experiments examining the sensitivity of zebra finches to such frequency mistunings in vocalizations sought to explain the birds’ performance as a sensitivity to spectral changes (Cynx et al., 1990; Lohr & Dooling, 1998). But zebra finches were so extraordinarily sensitive to such frequency changes, it became more likely that they were discriminating on the basis of something else like temporal fine structure (Dooling & Lohr, 2006; Lohr et al., 2006). To test whether this temporal acuity operates on call-like stimuli, independent of the concurrent variation in overall spectral and temporal changes that normally occur in such calls, we designed test stimuli using single periods of these long calls (Lohr et al., 2006). The time waveform of a female zebra finch long call is shown in Fig. 4a.

Figure 4.

(a) Time waveform of a female zebra finch contact call showing regions of individual periods that were excised and concatenated to produce 200 ms synthetic calls consisting of repeated single periods of a natural call. (b) The fine structure of both forward and reversed versions of these calls is shown below each corresponding synthetic call. (c) Results for zebra finches and humans tested on forward/reversed synthetic call discrimination. Zebra finches performed at much higher levels in discriminating between forward and reversed versions of such calls compared with humans. Fundamental frequencies ranged from 690 Hz to 816 Hz corresponding to periods of 1.45 ms to 1.225 ms, respectively (from Dooling & Lohr, 2006).

The harmonic nature of these calls means that the fine details of periodic waveforms are very complex. But, it is possible to select a single period from different locations of this call and repeat them to create synthetic stimuli with the duration of a natural call but with no variation in envelope or spectrum and which approximates the duration of natural calls (as shown in Fig. 4b, top row). The middle row of Fig. 4b shows periods extracted from the beginning, middle and end of the natural call (middle row), and the bottom row shows these same stimuli reversed. Casual inspection of these waveforms shows that the temporal fine structure of these stimuli differs between the beginning, middle and end of the call and, of course, that it differs depending on whether the waveform is played in the forward or reverse direction.

Birds and humans can easily discriminate between stimuli created from call periods drawn from different locations in the call. Humans report subtle differences in pitch, timber and roughness between these stimuli. However, zebra finches can discriminate the forward from the reverse version of this stimulus while humans cannot (Fig. 4c). The forward and reverse versions of this stimulus differ only in the ordering of temporal fine structure (Lohr & Dooling, 2006). Taken together, these psychophysical experiments with synthetic harmonic complexes, natural calls and song syllables show that zebra finches are exquisitely sensitive to changes in temporal fine structure in their vocalizations, possibly at the expense of sensitivity to overall envelope cues, at least in their song motifs. Both behavioural and single-unit studies using forward and reversed song syllables or motifs (where duration and overall spectrum are the same) also suggest that fine structure cues in these harmonic vocalizations could be important (Braaten et al., 2006; Margoliash & Fortunes, 1992; Theunissen & Doupe 1998; Williams et al., 1989).

DISCUSSION

There are several ways to consider temporal fine structure in vocalizations. One is whether it occurs in some systematic fashion that allows discrimination of different vocal signals, different individuals, different emotional states, etc. Another is whether such fine structure can be faithfully encoded in the auditory system. A third is whether fine structure differences in vocalizations alone, without myriad other accompanying acoustic cues, can be heard or discriminated by the listener. Temporal fine structure is generally defined as rapid variations in amplitude within the more slowly varying envelope of sound. Temporal fine structure is encoded in the pattern of phase locking by auditory nerve fibres (Moore, 2014). In humans, temporal fine structure is becoming increasingly important in models of pitch perception, masking, speech discrimination and recognition in noise, and the perception of complex sounds (Moore, 2014). The extraordinary sensitivity to fine structure in birds leads us to hypothesize that birds are able to communicate using information within the fine structure of calls. While the psychoacoustic research demonstrating the capability to hear this information has been present for over a decade, there have been relatively few studies investigating the potential function of fine structure in vocal communication or identifying fine structure in natural vocalizations. Zebra finches show extraordinary sensitivity to fine structure, even among birds. Additionally, these birds also have more harmonic vocalizations, rich in fine structure, compared to many other songbirds (including budgies and canaries, the other two species tested above). Casual observation suggests that zebra finch vocalizations have the most temporal fine structure, canaries have less and budgerigars have the least, which corresponds to their psychophysical thresholds. It is possible that sensitivity to fine structure variation and temporal resolution has coevolved with the richness of fine structure in vocalizations and it is worth considering what subtle information zebra finches might be communicating.

Zebra finches are highly social songbirds: they live in large groups and maintain life-long pair bonds. Pairs are typically sexually monogamous and engage in biparental care (Birkhead et al., 1988; Zann, 1996). The formation and maintenance of these long-term bonds are highly dependent on vocal behaviour. During courtship and bond formation, song appears to be crucial. Female zebra finches do not court males whose songs have been experimentally disrupted (Tomaszycki & Adkins-Regan, 2005). However, following bond formation, song is not necessary to maintain pair bonds (Tomaszycki & Adkins-Regan, 2006). What is often underappreciated is how essential calling behaviour is for zebra finch pairs. Once a pair bond is formed, movement, breeding and activity appear to be largely coordinated through emergent and dynamic calling behaviours.

Male and female zebra finches have a large repertoire of calls (Zann, 1996; Elie & Theunissen, 2015). Variation in calling is behaviourally relevant (Elie et al., 2010; Perez et al., 2015) and can function to synchronize a pair’s behaviour (Boucaud et al., 2016). For example, partners use distance calls to stay in touch while travelling and while visually isolated (Zann, 1996; Perez et al., 2015). Additionally, complex vocal duets, made up of several call types, are used at the nest to coordinate parenting behaviour (Elie et al., 2010; Boucaud et al., 2016). Behavioural synchrony within a pair is critical towards maximizing reproductive success and likely reflects the quality of a pair bond (Mariette & Griffith, 2012). It would not be surprising to find that temporal fine structure cues in vocalizations contribute to these behaviours.

Relatively recent research clearly demonstrates that bird calls, which are typically assumed to be innate and fixed, are highly plastic and can reflect internal state and experience. This is true for zebra finches as well as many other bird species (Marler, 2004). The acoustic structure of calls can be mediated by (1) environment (e.g. noise) (Nemet & Brumn, 2009; Villain et al., 2016), (2) social context and/or prior experience (Mundinger, 1970; Hile & Striedter, 2000; Villain et al., 2015; Boucaud et al., 2016), and (3) hormones and motivational state (Cynx et al., 2004; Hetrick & Sieving, 2011; Perez et al., 2012; 2016). The impact of these observations is something that Marler anticipated decades ago: that birds may be communicating important information through their vocalizations in ways we still do not understand. If so, temporal fine structure may be a part of this story. Interestingly, there is some evidence supporting this notion from recent studies showing that there can be seasonal plasticity in the coding of fine structure by the auditory periphery (e.g. Henry et al., 2011; Gall et al., 2013; Velez et al., 2015).

Finally, the relative importance of temporal fine structure in human speech perception, hearing in noise, the effectiveness of hearing aids and cochlear implants, the overall experience of listening to complex sounds and the quality of information encoded in speech is significant (Moore, 2014). This observation, along with the evidence above that birds are especially sensitive to temporal fine structure, suggests more attention should be paid to the possibility that important information is carried in the temporal fine structure of bird vocalizations. This is not to minimize the importance of other cues in birdsong such as the strict ordering of harmonic syllables in song, which also invites investigations into sequential structure or phonological syntax (Marler, 2000). There is considerable interest in the rules that govern the learning and perception of these syllable sequences and whether there could be parallels with human speech (ten Cate & Okanoya, 2012).

Summary

In summary, in spite of the fact that human listeners are sensitive to temporal fine structure in speech (Moore, 2014; Hopkins & Moore, 2011), the ability of birds and zebra finches, in particular, to greatly surpass humans in the discrimination of temporal fine structure in their vocalizations most certainly means that bird vocalizations sound much different to birds than it does to humans. There are several implications of this. First, it suggests there is a foundation for a deeper level of acoustic communication occurring among birds. Second, it suggests caution is warranted when restricting our analysis of bird communication signals to envelope and spectral or spectrographic cues as is traditionally done. Third, it remains unclear what other functions this enhanced temporal resolving power serves. But, it is not hard to imagine that it serves many purposes other than acoustic communication. For instance, there are many binaural phenomena such as sound location, binaural release from masking, etc., where birds do much better than they should given their small heads and closely spaced ears (Dooling et al., 2000). Enhanced time resolution in comparing signals between ears could provide the foundation for precision in these and a host of other auditory behaviours as well.

Highlights.

• Birds have impressive temporal sensitivity to auditory stimuli.

• Zebra finches have particularly good sensitivity to temporal fine structure.

• The communicative potential of fine structure in vocalizations needs to be explored.