Auditory Signal Processing in Communication: Perception and Performance of Vocal Sounds

Abstract

Learning and maintaining the sounds we use in vocal communication require accurate perception of the sounds we hear performed by others and feedback-dependent imitation of those sounds to produce our own vocalizations. Understanding how the central nervous system integrates auditory and vocal-motor information to enable communication is a fundamental goal of systems neuroscience, and insights into the mechanisms of those processes will profoundly enhance clinical therapies for communication disorders. Gaining the high-resolution insight necessary to define the circuits and cellular mechanisms underlying human vocal communication is presently impractical. Songbirds are the best animal model of human speech, and this review highlights recent insights into the neural basis of auditory perception and feedback-dependent imitation in those animals. Neural correlates of song perception are present in auditory areas, and those correlates are preserved in the auditory responses of downstream neurons that are also active when the bird sings. Initial tests indicate that singing-related activity in those downstream neurons is associated with vocal-motor performance as opposed to the bird simply hearing itself sing. Therefore, action potentials related to auditory perception and action potentials related to vocal performance are co-localized in individual neurons. Conceptual models of song learning involve comparison of vocal commands and the associated auditory feedback to compute an error signal that is used to guide refinement of subsequent song performances, yet the sites of that comparison remain unknown. Convergence of sensory and motor activity onto individual neurons points to a possible mechanism through which auditory and vocal-motor signals may be linked to enable learning and maintenance of the sounds used in vocal communication.

1 Central Importance of Vocal Signals in Human Communication

Accurate perception and imitation of the sounds we hear performed by others are fundamental to human communication through spoken language, and the neural basis of those abilities has fascinated researchers for centuries. At a basic level, the neural mechanisms of vocal communication must accomplish two tasks. First, a continuous stream of acoustic signals must be broken into behaviorally relevant segments, and each of those segments must be accurately perceived. This process is especially challenging because acoustic signals cannot be resampled as they can in the case of fixed stimuli such as this text. Vocal communication is a temporally dynamic process, and information will quite literally pass the listener by unless robust mechanisms are in place to facilitate rapid and reliable perception. Second, auditory perception must be linked to motor performance to enable imitation of the sounds that compose our spoken language. The sounds we use in speech are initially learned as sensory signals performed by others, but over the course of development we use sensory feedback to refine our performance of those sounds. The quality of imitation that we typically achieve is extraordinary, as evident in the preservation of specific features that define regional dialects.

In considering how auditory input is used to shape motor output, some researchers have speculated that the sensory framework in which speech is processed as an auditory signal and the motor framework in which speech is processed as a vocal signal must be similar, and in order for communication to occur then at some point in neural processing those sensory and motor representations must be the same (Liberman et al., 1967; Rizzolatti and Arbib, 1998). An attractive idea is that individual neurons are active in association with specific vocal signals both when they are heard and when they are spoken, and it is through that sensorimotor correspondence that auditory perception may be translated into vocal motor performance (Arbib, 2005; Ferrari et al., 2003; Gallese et al., 1996; Iacoboni et al., 1999; Iacoboni et al., 2005; Rizzolatti and Craighero, 2004; Rizzolatti et al., 2001). Given these challenges faced by the brain in perception and imitation, it is somewhat astounding that communication typically proceeds as fluidly as it does. Fluency of vocal communication is a testimony to the robust effectiveness of the underlying brain mechanisms, and this commentary will highlight recent insights into a possible basis of perception and imitation of the sounds used in vocal communication.

Specialized regions of the human cortex have been implicated in perception and performance of the sounds used in speech. These regions, such as Wernicke's and Broca's areas, are well known, however they are by no means alone in their contribution to the behavioral complexity of vocal communication. Other sites, including primary and secondary cortical regions associated with sensory and motor processing, as well as subcortical regions through cortical-striatal-thalamocortical loops have been identified as important in speech (Alm, 2004; Doupe and Kuhl, 1999; Fox et al., 1996). A challenge to understanding the neural basis of human vocal communication is that discerning the contributions of those sites is complicated by the necessity of high-resolution information in both the temporal (millisecond) and spatial (microns) domain and difficulty disambiguating speech-related activity from other information processed by mammalian corticostriatal projections. Simply put, gaining the high-resolution insight necessary to define the cellular and circuit mechanisms underlying human vocal communication is presently impractical. In precious few cases in which human neurons can be sampled during hearing and speaking, neurons have been found to be active in each state, but it is not clear the degree to which that activity is sensory, motor-related or some combination (e.g., Creutzfeldt et al., 1989). Therefore, it has been a topic of considerable debate whether individual neurons in the human brain express a precise sensorimotor correspondence and whether such cells may play a role in linking the perception and execution of vocal signals (Brass et al., 2007; Hari et al., 1998; Mukamel et al., 2010). It has been a priority for many years to study a diverse array of songbird species so that we can discover ideal model species in which to investigate the possible existence of such neurons and to consider how they may contribute to the neural basis of learned vocal communication.

2 Songbirds as a Model System to Understand the Neural Basis of Vocal Communication

Songbirds provide an excellent animal model of human speech. In a form of learning that is rare across animals, songbirds acquire their songs by imitating other members of their species, just as humans imitate others to learn the sounds we use in speech (Doupe and Kuhl, 1999; Jarvis, 2004). Songbirds and humans are both critically dependent on auditory input to learn and maintain their vocalizations, and the pattern of song development is strikingly similar to babbling and subsequent vocal refinement in humans (Bolhuis and Gahr, 2006; Doupe and Kuhl, 1999; Mooney, 2009). Briefly, sensory representations of the signals that will eventually be imitated are memorized during a juvenile sensitive period (Funabiki and Konishi, 2003; Marler and Peters, 1981; Marler and Peters, 1982; Marler and Peters, 1987; Peters et al., 1992). Later in juvenile development, vocalizations begin and auditory feedback is used to refine those vocalizations to achieve accurate imitation and maintain those signals in adulthood (e.g., Marler and Sherman, 1983; Woolley and Rubel, 1997). Parallelism in the learning, performance and communicative significance of speech and birdsong make songbirds a uniquely advantageous animal model to understand how vocal signals are perceived and transformed into a motor framework to enable communication.

In seeking to understand the neural basis of vocal communication, songbirds provide two additional important advantages. First, song can be easily quantified and manipulated to facilitate experimentation and assessment of the relation between behavior and the associated neural activity. Second, and most importantly, song behavior is associated with dedicated regions in the songbird brain. This interconnected network, collectively called the “song system”, comprises nuclei of the forebrain, striatum, thalamus and brainstem, and the presence of a song system distinguishes songbirds from avian species that do not learn their vocalizations (Kroodsma and Konishi, 1991; Mooney et al., 2008; Wild, 2004). The song system contains two major pathways, the song motor pathway (SMP) and an anterior forebrain pathway (AFP), each of which emerges from the sensorimotor nucleus HVC (Figures 1 and 2). An intact SMP is necessary for song production, with lesions resulting in severe degradation or elimination of song (Nottebohm et al., 1976; Simpson and Vicario, 1990). In contrast, the AFP is not necessary for adult song (Nottebohm et al., 1976), but it is necessary for juvenile song learning and adult song plasticity (Andalman and Fee, 2009; Bottjer et al., 1984; Brainard and Doupe, 2001; Olveczky et al., 2005; Scharff and Nottebohm, 1991; Sohrabji et al., 1990; Williams and Mehta, 1999). Furthermore, activity in HVC and the AFP has been implicated in song perception, as lesions to either HVC or its synaptic target in the AFP (Area X) disrupt perception of songs performed by self or others (Gentner et al., 2000; Scharff et al., 1998). Each of these pathways is active during singing (Hahnloser et al., 2002; Hessler and Doupe, 1999; Kao and Brainard, 2006; Kozhevnikov and Fee, 2007; McCasland and Konishi, 1981; Olveczky et al., 2005; Olveczky et al., 2011; Prather et al., 2008; Sakata and Brainard, 2008; Schmidt, 2003; Troyer and Doupe, 2000; Yu and Margoliash, 1996), and cells in each pathway receive auditory input (Katz and Gurney, 1981; Margoliash, 1986; Mooney, 2000).

FIGURE 1. The microcircuit of HVC provides one possible circuit through which singing-related motor activity could be recurrent onto HVCX cells.

HVCRA cells are active when the bird sings (Amador et al., 2013; Hahnloser et al., 2002) and project into a song motor pathway that is essential for song production (HVCRA cells, red). HVCX cells are also active when the bird sings (Fujimoto et al., 2011; Kozhevnikov and Fee, 2007; Prather et al., 2008), and the anterior forebrain pathway into which they project is not required for song but has been implicated in song perception and plasticity (details in the text). An important future direction will be to define the pathways through which singing-related activity of HVCX neurons emerges, and it is likely that recurrent activity from HVCRA cells plays an important role. In addition to brainstem pathways through which activity originating in HVCRA cells could affect HVCX cells directly (shown in Figure 2), the HVC microcircuit is another place where HVCRA cells could influence HVCX cells. HVCRA cells excite HVCX cells directly and inhibit them indirectly through intervening projections onto HVC interneurons (Mooney and Prather, 2005). HVCRA cells also excite other HVCRA cells directly, but no connections from HVCX cells onto HVCX cells were detected (Mooney and Prather, 2005). The contributions of this network to the generation of precisely timed auditory responses in HVCRA and HVCX cells have been described (Mooney, 2000; Rosen and Mooney, 2003; Rosen and Mooney, 2006), but whereas these connections are well documented, it remains unclear what role they play in the generation of precisely timed auditory and vocal motor activity in association with singing. All neurons in HVC receive auditory input (green) (Rosen and Mooney, 2006). Excitatory inputs are indicated by open circles. Inhibitory inputs are indicated by filled triangles. Networks form which HVC neurons receive auditory input and to which HVC neurons project are elaborated in Figure 2. Figure is adapted from Mooney and Prather 2005.

FIGURE 2. HVC is an important component of auditory (green) and premotor (red) networks in the male songbird brain.

(A) HVC is a high-level auditory processing site with exceedingly selective auditory responses among the most selective described for any sensory system to date. Of special importance to the themes addressed here, several brain sites project directly to HVC. Among those, Bauer and colleagues (2008) described a projection from CM to HVC, and Akutagawa and Konishi (2010) clarified that finding by showing that a subset of CM, the nucleus Avalanche (Av), is the source of monosynaptic projections to HVC. Furthermore, CM is interconnected with NCM, and those sites are collectively considered the avian analog of mammalian auditory association cortex (Jarvis, 2004; Karten and Shimizu, 1989; Vates et al., 1996). Understanding the functional contributions of those high-level auditory processing centers and others in this network, with special emphasis on how they shape the activity of HVC neurons, is an important goal of ongoing research. The dashed line between auditory input and VTA indicates that the connection has been described in unpublished data (Las and Fee 2008, Society for Neuroscience poster). Determining the strength and prevalence of that possible connection will require further investigation. (B) HVC is also an important piece of the song motor pathway devoted to singing. If HVC is lesioned, then the bird can still vocalize but the complex learned behaviors that define song are no longer possible (Aronov et al., 2008; Nottebohm et al., 1976). Singing-related activity emerges from HVC and drives the activity of RA neurons (Hahnloser et al., 2002; Hahnloser et al., 2006; Leonardo and Fee, 2005), and singing-related action potentials in RA neurons drive activation of brainstem neurons (nXIIts) that control the muscles of the vocal organ (Nottebohm et al., 1976; Vicario, 1991; Wild, 1993). Important for the present consideration of possible mechanisms through which singing-related activity emerges in HVCX neurons, activity emanating from RA enters several additional pathways that are eventually recurrent onto HVC (Ashmore et al., 2005; Coleman et al., 2007; Jurgens, 2002; Roberts et al., 2008; Striedter and Vu, 1998). An important goal of future research will be to define the degree to which these loops contribute to the singing-related activity detected in HVCX neurons. In the stylized parasagittal section of each panel, arrows indicate the directionality of each connection, and gray dotted lines indicate prominent lamina that serve as useful landmarks. Information depicted here is taken from many primary articles describing each connection. For the sake of brevity only a selection of that literature is cited here (Ashmore et al., 2005; Bolhuis and Gahr, 2006; Jarvis et al., 2005; Mooney et al., 2008; Nixdorf-Bergweiler and Bischof, 2007; Reiner et al., 2004; Vates et al., 1996). Importantly, not all connections are shown. For example, there are important distinctions between projections emanating from dorsal RA versus those from ventral RA, and brainstem sites implicated in control of respiration are intermediate between RA and nXIIts and between RA and Uva, but those sites are summarized here as connections from RA to nXIIts and from Ra to Uva (e.g., Ashmore et al., 2005).

Research into the functional significance of the song system has been wide ranging, but many studies have focused on two broad themes. First, how is auditory perception encoded in the brain, and how is that perception stored and recalled as a model of imitative behavior? Second, how are auditory perception and vocal motor commands compared to enable feedback-dependent learning? Early investigations of those themes revealed the importance of auditory experience in juvenile learning and adult maintenance of song. Peter Marler and colleagues showed that sparrows raised in social isolation and therefore deprived of a song model developed songs lacking the acoustic complexity of song-tutored birds (e.g., Marler and Tamura, 1964). These results revealed that adult song is shaped by a tutor song that is experienced and memorized in early life (Funabiki and Konishi, 2003; Marler and Peters, 1981; Marler and Peters, 1982; Marler and Peters, 1987; Marler and Peters, 1988). Around that same time, Mark Konishi (1965) performed another classic experiment in which he deafened juvenile sparrows after exposure to a tutor song but before they began to sing. He found that birds were unable to reproduce the song that they had memorized, revealing that both early auditory experience of a model and ongoing auditory feedback throughout development are necessary for vocal development. In those experiments, Konishi also deafened adult sparrows and saw little or no change even long after deafening, leading to the idea that feedback was of relatively less importance in adult song maintenance than in juvenile learning. Years later, additional experiments revealed that this was not the case in all species. Adult zebra finches that are deafened (Lombardino and Nottebohm, 2000; Nordeen and Nordeen, 1992) or exposed to chronic alteration of auditory feedback (Leonardo and Konishi, 1999) express deteriorated songs in days or weeks, and vocal deterioration in Bengalese finches can occur even more rapidly (Okanoya and Yamaguchi, 1997; Woolley and Rubel, 1997). This dependence on auditory feedback for learning and maintenance is a hallmark of vocal learning, as preservation of human speech is similarly dependent on auditory feedback (Cowie and Douglas-Cowie, 1992). Together with the discovery of a song system specialized for song perception, performance and plasticity, these data reveal that comparative studies of species that express different degrees of sensitivity to altered auditory feedback (e.g., white crowned sparrows and Bengalese finches) afford an excellent opportunity to understand not only the circuit and cellular basis of feedback-dependent learning but also the mechanisms through which we acquire and preserve the signals we use to communicate through speech.

In the years since the discovery of the song system, additional forebrain regions have also been identified as important contributors in auditory processing. Those regions, including the caudal mesopallium (CM) and the caudomedial nidopallium (NCM), are not involved in song production and are present in both male and female birds (Figure 2) (Bauer et al., 2008; Jarvis and Nottebohm, 1997). Therefore, they are not considered part of the canonical song system, however they are active in association with auditory processing of features far beyond the acoustic song structure, such as the familiarity of a song or its role as the model from which the bird learned its song during development (Bolhuis and Eda-Fujiwara, 2003; Bolhuis et al., 2000; Chew et al., 1996; Chew et al., 1995; Phan et al., 2006; Terpstra et al., 2004). CM and NCM (collectively called the auditory lobule) are well positioned to serve such a role, as they receive input directly and indirectly from the avian primary auditory cortex (Field L) and project to nucleus HVC and to other sites in the forebrain (Bauer et al., 2008; Vates et al., 1996). Thus, activity of cells in the auditory lobule and the sensorimotor nucleus HVC to which they project have been implicated in auditory perception. These findings recommend CM, NCM and HVC as excellent sites in which to investigate the neural basis through which auditory perception shapes vocal performance.

3 Correlates of Auditory Perception in Secondary Auditory Cortical Areas

Neural correlates of auditory cognition have been detected in a variety of sites in the songbird brain. Patterns of gene expression and results from electrophysiological recordings have revealed song-evoked activation of neurons in the avian analog of secondary auditory cortex (CM and NCM) (Jarvis, 2004; Karten and Shimizu, 1989; Vates et al., 1996). Initial studies reported gene expression in CM and NCM that was selective for song of their own species, drawing attention to those areas as potentially important in processing complex auditory stimuli (Mello et al., 1992). Subsequent studies using electrophysiological methods have sought to define the behaviorally relevant song features that are encoded in the activity of those neurons. The earliest recordings from neurons in CM and NCM confirmed that those cells are indeed selective in their auditory responses (Chew et al., 1996; Chew et al., 1995; but see Stripling et al., 1997). Specifically, auditory responses of neurons in CM and NCM are selective not only for song, but also for songs of the bird's own species versus songs of other species (Mello et al., 1992; Phan et al., 2006; Stripling et al., 2001; Terleph et al., 2007). In contrast, primary auditory cortical neurons (Field L) are very broadly responsive to not only song but also to many other stimuli such as calls, pure tones or white noise (Amin et al., 2004; Grace et al., 2003; Lewicki and Arthur, 1996; Margoliash, 1986; Meliza and Margoliash, 2012). In further support of the idea that activity in the auditory lobule represents more complex features than the primary auditory cortex, CM and NCM neurons in different species are tuned to species-typical song features, but that degree of selectivity is not evident in the activity of thalamorecipient cells in the primary auditory cortex (Fields L1 and L2) (Meliza and Margoliash, 2012; Terleph et al., 2007). This selectivity for species-typical song features among cells in the auditory lobule may facilitate integration of song elements in service of recognition of song and singer identity.

Cells in the auditory lobule also express another intriguing aspect of auditory selectivity in that the responses of those cells to a given song are dependent on the context in which that song is presented. Auditory responses become progressively weaker with repeated presentation of the same song, but even in that habituated state, presentation of new songs is capable of driving a strong response (Chew et al., 1996; Chew et al., 1995; Stripling et al., 1997). Habituation to a specific song can persist even without ongoing presentation of that song or even if other songs are played in the interim (Chew et al., 1996; Chew et al., 1995; Stripling et al., 1997). The context-dependent auditory responses of cells in the auditory lobule have been interpreted as a means of representing familiarity with a specific song stimulus. Together, these complex patterns of auditory response among cells in the auditory lobule led to the idea that those neurons are active in association with perception of song features beyond the physical structure of the stimulus.

The speculation that cells in the auditory lobule contribute to song perception has been further bolstered by results from the past decade showing that activity of those cells can encode behaviorally relevant categories of song stimuli. In starlings trained to recognize different groups of songs, activity of CM neurons recorded under anesthesia revealed that individual CM neurons are selectively active in association with songs belonging to one or the other of those groups, and cells can extend their categorization to novel songs that conform to the specifications that define the training group for which those cells are selectively active (Gentner and Margoliash, 2003). Thus, information about learned categories of song perception is encoded in the activity of cells in the auditory lobule, and recent results have identified a subregion of the auditory lobule (CMM, the medial portion of CM) in which cells are extremely selective for specific song components and information about learned categories is especially well-represented in the activity of individual cells (Jeanne et al., 2011; Meliza et al., 2010). These data implicate neurons in the auditory lobule, especially those in CM, as part of a network that can be profoundly shaped by experience and that likely plays an important role in the emergence of song perception.

Recent results also indicate that cells in CM and NCM may contribute to not only song perception but also to memorization of those songs in service of vocal learning. The earliest such hints came from studies of ZENK gene expression in which male zebra finches were played the songs that served as the tutor song for their own song learning. Gene expression in NCM is greatest in birds that copy their tutor song most accurately, suggesting a possible link between NCM activity and the precision of tutor song representation (Bolhuis et al., 2000; Terpstra et al., 2004). Later experiments extended those findings by recording auditory responses in NCM of awake, restrained zebra finches and finding that those cells are selectively responsive to the song of a tutor heard during early development, and properties of that auditory response were correlated with the fidelity of the bird's imitation of that model (Phan et al., 2006). Those tantalizing results led to the speculation that the auditory lobule could play an important role in the memorization of the tutor song, and that was tested in an elegant experiment by Sarah London and David Clayton (2008). In a series of behavioral experiments coupled with studies of gene expression in CM and NCM, cells in those sites were reversibly inactivated only during tutor exposure but not during alternate days when the pupil was allowed to rehearse in the absence of tutor instruction. In those birds, the pupil's imitation of the tutor song was compromised. In contrast, inactivation of CM and NCM during rehearsal but not during tutor song exposure had little or no effect on the quality of imitation. Together, these results make a strong case that cells in the auditory lobule play central role in auditory processing and the formation of a tutor memory that guides imitative learning (Gobes and Bolhuis, 2007). In another set of electrophysiological recordings in very young zebra finches that had been exposed to tutor song but had not yet begun to rehearse, Adret and colleagues (2012) found neurons in the auditory lobule that were selectively active in association with hearing the tutor song. Although those cells were not especially common, they nonetheless demonstrate the presence of cells in songbird auditory cortical areas that are modified by perceptual and/or social experience during vocal learning and development. Studies of female birds have also suggested a role for auditory lobule neurons in song memorization even without any attempt at vocal imitation. Those cells are active in association with presentation of the familiar song of a female bird's father (Terpstra et al., 2006), or the song or the call of a female bird's mate (Menardy et al., 2012; Vignal et al., 2008; Woolley and Doupe, 2008). Together with results from male birds, these data support the idea that cells in CM and NCM play important roles in perceptual grouping and memorization of the sounds used in vocal communication.

4 Correlates of Auditory Perception in Sensorimotor Cortical Areas

In light of the present consideration of perception and its relation to imitation of the signals used in vocal communication, an important question is whether neural correlates of perception are preserved or perhaps even refined at the level of sensorimotor structures. To test that possibility, Rich Mooney, Steve Nowicki, Susan Peters, Rindy Anderson and I performed a series of studies using swamp sparrows (Melospiza georgiana). Just as humans perceive categorical differences among specific sounds used in speech, swamp sparrows perceive categorical differences between song notes of different duration (Diehl et al., 2004; Nelson and Marler, 1989). Below a categorical boundary, notes are perceived as short regardless of their actual duration; above that boundary, notes are perceived as having long duration. We investigated whether categorical perception of song notes was evident in the activity of individual neurons in the sensorimotor nucleus HVC (Prather et al., 2009). The most informative approach in seeking to understand the neural basis of auditory perception is to investigate the activity of neurons in awake and freely behaving birds as they are engaged in song perception. To achieve that in our experiments, we used a miniature, motorized recording device (Fee and Leonardo, 2001) to sample the activity of individual HVC neurons that project into the forebrain pathway implicated in song perception and plasticity (HVCX cells, Figure 3). In swamp sparrows, individual HVCX neurons express categorical auditory responses to songs containing notes of different duration, and the categorical boundary evident in the activity of individual neurons predicts the boundary evident in song perception (Figure 4) (Prather et al., 2009). Importantly, song stimuli can evoke robust activity in HVCX cells regardless of whether the bird produces the song or the same song is produced by another bird (Figure 5) (Prather et al., 2008). Auditory responses of swamp sparrow HVCX cells are selective for specific song features, and they are responsive to those features regardless of whether they are part of the bird's own repertoire or part of the repertoire of a nearby conspecific. Therefore the selective auditory responses of these cells are not simply a self-tuning mechanism. Instead, they provide a substrate for perception of signals used in communication between individuals. Additional studies have also provided evidence that activity of individual HVC neurons reflects complex features of auditory perception. Specifically, HVC neurons of very young birds recorded during song learning are selectively responsive to the tutor song that the bird is engaged in imitating (Nick and Konishi, 2005; Volman, 1996). Furthermore, HVC auditory responses to songs heard or rehearsed only during juvenile development can persist into adulthood, and auditory responses to those developmentally relevant songs are commonly as strong as or stronger than responses evoked by anything in the adult repertoire (Prather et al., 2010). Because neurons in CM and NCM have also been implicated in song perception and those sites project directly and indirectly to HVC (Bauer et al., 2008; Pinaud et al., 2008; Vates et al., 1996) including important connections through the nucleus interface of the nidopallium (NIf, Figure 2) (Bauer et al., 2008; Cardin and Schmidt, 2004; Coleman and Mooney, 2004; Hosino and Okanoya, 2000; Roberts et al., 2012) and the nucleus Avalanche (Av, Figure 2) as the portion of CM that projects monosynaptically to HVC (Akutagawa and Konishi, 2010), it remains unknown whether neural correlates of perception emerge in HVC or whether HVC activity reflects information present in synaptic inputs from the auditory lobule. Nonetheless, auditory perception is reflected in the activity of individual neurons in HVC of awake and freely behaving birds, providing a locus where perception and motor performance may be linked. Important in our consideration of possible translation of perception into vocal performance, all of the HVCX neurons that express categorical auditory responses are also active when the bird sings (Figure 6). This colocalization of activity related to perception and vocalization in one and the same neuron establishes HVCX cells as very attractive candidates for understanding how auditory perception may influence vocal performance.

FIGURE 3. Robust, phasic and highly selective auditory responses are common in HVCX neurons across freely behaving male songbirds of different species.

(A–C) Strong auditory responses were evident in HVCX neurons sampled from swamp sparrows (Melospiza georgiana) (Prather et al 2008), Bengalese finches (Lonchura striata domestica) (unpublished data, see also Prather et al 2008), and canaries (Serinus canaria) (unpublished data). A representative cell from each species is shown here, but auditory responses were ubiquitous across HVCX cells in these species (swamp sparrows: significant responses in 69/71 cells from 9 birds; Bengalese finches: 32/32 cells from 4 birds; Canaries: 12/12 cells from 2 birds; top: raster of action potentials; middle: histogram of action potentials across all trials, 10 ms binsize; bottom: oscillogram of the bird's own song played in each trial). (D) In contrast, HVCX neurons in zebra finches (Taeniopygia guttata) (unpublished data) were not responsive to song in the awake state (detailed records were not kept regarding negative results, but failure was detected in dozens of zebra finch HVCX cells from many birds; plots arranged as in A–C). This stark difference suggests that different species may have solved the challenge of sensorimotor comparison in different ways, and the apparent absence of a sensorimotor correspondence in zebra finch HVCX neurons – or at least one that is broadly expressed – may at least partly account for previous findings that ablating those cells had little effect on adult song (Scharff et al., 2000). Taken to the extreme, these comments may sound like a call to arms against using zebra finches, but that is not at all my intent. Although broad features of songbird neuroanatomy are conserved across all species studied to date (e.g., the existence of a song system), it is important to exercise caution in extrapolating finely detailed features of song-related neural function across species that express such a variety of vocal behaviors (e.g., repertoires, fixed versus variable syntax, etc.). A comparative approach within and across laboratories will be essential in future efforts to discern general principles of the neural basis of vocal communication (Brenowitz and Beecher, 2005; Konishi et al., 1989; Schmidt, 2010). Discerning such principles will be of great value to our understanding of birdsong and to clinicians seeking to understand how the human nervous system may encode speech and its disorders.

FIGURE 4. HVCX neurons in adult male swamp sparrows express categorical responses to changes in note duration, and auditory activity predicts the bird's categorical perception of those same stimuli.

(A) Auditory responses of HVCX neurons occurred at a restricted phase in the syllable of the song type in the adult repertoire to which the cell was selectively responsive (the “primary song type” for each cell; top: histogram of auditory response, 1 ms binsize; bottom: spectrogram of one syllable in the primary song type for this cell). (B) Manipulation of song note duration reveals categorical auditory responses in HVCX neurons (Prather et al., 2009). Individual notes in each syllable of the primary song type (note B in the top spectrogram) were replaced with another swamp sparrow song note. Replacement notes always had similar spectral characteristics (e.g., a frequency down-sweep) but were of different duration. Categorical responses were evident as strong responses to one group of replacement notes (4, 8, 16 ms for this cell) but little or no response to another group of replacement notes (27, 31 ms). Note durations were chosen to match values used in field studies of swamp sparrow song perception (Nelson and Marler, 1989) we controlled for other note characteristics such as frequency modulation and bandwidth (Prather et al., 2009) (top: spectrogram of one syllable of the primary song type with individual notes labeled; middle: durations of replacement notes and action potential histograms recorded from the same cell in response to a synthetic song composed of syllables with note B replaced by a note of the corresponding duration, 1 ms binsize; bottom: stimulus syllables, gray boxes indicate replacement notes).

FIGURE 5. HVCX selective auditory responses extend to songs performed by other birds.

(A, B) Swamp sparrow HVCX neurons respond to specific acoustic sequences in each syllable of the primary song type, and similar note sequences in the songs of other birds are also capable of activating those cells (Prather et al., 2008) (top panel: auditory response histogram and syllable spectrogram of the primary song type; bottom panel: response histogram and syllable spectrogram of the song of another swamp sparrow (conspecific); different cells from different birds are shown in A and B, 1 ms binsize in each histogram). (C, D) Alignment of syllables in the primary song type (top in each panel) and the effective conspecific song (bottom in each panel) revealed very similar acoustic features (syllables aligned according to the mean latency of auditory action potentials, filled triangles). Swamp sparrows can perceive differences between not only different song types performed by self or other birds (Searcy et al., 1981) but also between acoustically distinct variants of the same song type performed by different birds (DuBois et al., 2011). Differences between auditory responses of the sort shown here may underlie that perceptual distinction.

FIGURE 6. HVCX neurons that express categorical auditory responses also express a precise and selective sensorimotor correspondence.

(A) Using the same approach as described in Figure 4, categorical responsiveness was detected in this HVCX neuron (different cell and different bird than in Figure 4). (B) A precise sensorimotor correspondence was also evident in this neuron (same cell as in panel A). That correspondence was evident as precise alignment activity of this cell when the bird sang the primary song type (top: singing-related action potentials; bottom: song spectrogram, 15-syllable trill clipped for clarity) and when the bird heard the primary song type played through a speaker (middle: auditory action potentials). This sort of sensorimotor correspondence was detected in all cells in which it was tested in swamp sparrows (7/7 cells in 3 birds) and Bengalese finches (7/7 cells in 2 birds) (Prather et al., 2008). Fujimoto and colleagues (2011) also found this sort of sensorimotor correspondence in HVCX cells of Bengalese finches. The prevalence of this correspondence is greater in our hands (100%) than in their data (31%), likely due to our assessment of each cell's auditory response using exactly the song that was recorded in association with singing-related activity in that cell. A detailed consideration of such features as the latency of action potentials in the hearing and singing states, as well as the production of single action potentials in the hearing state and bursts in the singing state can be found in our previous publication (Prather et al., 2008).

5 Translating Sensory Perception into Vocal Performance

Consistent with a role in linking auditory perception and vocal motor performance, individual HVCX neurons are active in association with a specific song both when it is heard and when it is sung. Using each swamp sparrow's adult song repertoire as a means of exploring auditory response selectivity, we found that individual neurons are selective for one song type in the bird's repertoire. The song type that is represented in action potential responses varies among cells, but each HVCX neuron is selective in a nearly all-or-none fashion for what we called its “primary song type” (Prather et al., 2008). Selectivity was also observed when the bird sang. Invariably, the primary song type as defined in the auditory domain was also the primary song type as defined by activity of that cell when the bird sang the elements of its repertoire (Figure 6B). Importantly, tests using transient distortion of auditory feedback indicated that this activity during singing was a motor-related signal as opposed to the bird simply hearing itself sing (Prather et al., 2008). Therefore, individual HVCX neurons in the swamp sparrow brain express precise representations of both auditory perception and song performance. Notably, a correspondence like that found in swamp sparrows is also evident in Bengalese finches (Lonchura striata domestica) (Fujimoto et al., 2011; Prather et al., 2008). These species are of distant phylogenetic relation, and the swamp sparrow song system grows and regresses across seasons but the Bengalese finch song system does not (Tramontin and Brenowitz, 2000). The presence of a temporally precise sensorimotor correspondence and selective representation of individual vocal elements in HVCX neurons of such distantly related species with such different patterns of brain plasticity suggests that sensorimotor correspondence is a fundamental feature of perception and performance of learned vocal signals. It remains to be seen through what network and over what time scales song perception may influence performance of the signals used in vocal communication, but this finding of neural activity related to auditory perception and vocal motor performance co-localized in one and the same neuron provides an especially advantageous animal model in which to investigate how those signals can collectively guide imitative vocal learning.

The presence of auditory and singing-related activity in individual HVCX neurons provides a mechanism that could facilitate song learning and maintenance. Conceptual models of song learning typically involve the comparison of vocal motor commands against the associated auditory feedback to compute an error signal that is used to guide refinement of subsequent song performances (Keller and Hahnloser, 2009; Leonardo, 2004; Mooney, 2009; Solis et al., 2000; Troyer and Doupe, 2000). Therefore, sites that receive both motor-related and auditory input and thus could be sites of sensorimotor comparison have been a topic of great interest in our field. But simply receiving input from both motor-related and auditory sources is not sufficient to perform this comparison because of the potential difference in the timing of activity from each of those sources regarding one and the same vocal signal. It is not clear how the initial motor command from HVC into the song motor pathway, which lasts only milliseconds and precedes the vocalization (Hahnloser et al., 2002), could be preserved for a sufficient duration to enable it to be compared against auditory feedback arriving tens of milliseconds later. One mechanism through which the initial motor command could be preserved and delayed in its recurrence onto sensory-recipient cells is called a corollary discharge. A corollary discharge is essentially a copy of the initial motor command that is diverted along a pathway that does not result in motor activation (Bell, 1989; Crapse and Sommer, 2008a; Crapse and Sommer, 2008b; Sommer and Wurtz, 2008). If the arrival of a corollary discharge onto its target cells occurred at that same time as the arrival of the associated auditory feedback, then such an arrangement could enable direct comparison of neural representations of the motor command and the sensory feedback (Mooney, 2009). Intriguingly, in the species in which this sensorimotor correspondence has been studied to date, the singing-related activity in HVCX neurons is delayed such that it occurs in exact temporal register with the latency of auditory responses in the same neuron (Figure 6B) (Prather et al., 2008). These data support the idea that HVCX singing-related activity may provide a prediction of the expected auditory feedback, and that prediction may be compared to the actual auditory feedback to compute an error signal. The sensorimotor correspondence that we and others have described in HVCX neurons provides a locus where such a comparison may occur in service of learning and maintaining the signals used in vocal communication.

Cells in which activity changes in response to a mismatch between the bird's vocalization and the associated auditory feedback have been described in two sites in the songbird brain. Specifically, cells in the primary auditory cortex of zebra finches (Field L) (Keller and Hahnloser, 2009) and some HVC interneurons in Bengalese finches (Sakata and Brainard, 2008) are selectively active in association with such mismatches. These data confirm that auditory-vocal comparisons occur in the songbird brain, and their conservation across species lends additional support to the idea that such comparisons are a fundamental feature of vocal learners. Michael Brainard and colleagues have demonstrated a functional role for feedback in shaping real-time vocal plasticity (Sakata and Brainard, 2006; Sakata and Brainard, 2008; Sakata and Brainard, 2009; Sober and Brainard, 2009; Sober and Brainard, 2012; Tumer and Brainard, 2007) and the activity of specific neurons (Sakata and Brainard, 2006; Sober et al., 2008), however it remains unknown how neural correlates of distorted auditory feedback are manifest as changes in vocal behavior. Interestingly, not all cases of distorted feedback induce changes in vocal output (e.g., trials recorded soon after the onset of distortion in Leonardo and Konishi 1999 and cases of transient distortion in Prather et al. 2008), and in cases where changes are evident, many of those changes are very subtle (Kozhevnikov and Fee, 2007; Sakata and Brainard, 2006; Sakata and Brainard, 2008). This general resistance to changes in song note sequence and spectral properties in the face of transient mismatches between vocal behavior and auditory feedback is advantageous because it preserves learned behavior in the presence of ambient sounds or songs of other birds nearby. In the case of manipulations that are sufficient to induce changes in note sequence and spectral properties, such as deafening or prolonged exposure to distorted feedback (Leonardo and Konishi, 1999; Nordeen and Nordeen, 1992), one idea is that prolonged error results in the sustained alteration of activity in real-time error detectors such as neurons in Field L and HVC interneurons. Such a paradigm would suggest that chronic alteration of activity of those cells eventually induces changes in the activity of another set of neurons, and it is those cells that are the agents of vocal change. Those putative integrators of prolonged error have not yet been identified, but HVCX neurons are attractive candidates because they receive monosynaptic input (HVC interneurons, Figure 1) and polysynaptic input (Field L, Figure 2) from cells implicated in real-time detection of sensorimotor mismatches and they are the origin of an anterior forebrain pathway implicated in vocal plasticity (Figure 2).

Presently, the possible role of HVCX neurons as sensorimotor comparators remains unknown. On the one hand, the sensorimotor correspondence expressed by HVCX neurons makes them ideal candidates to act as sensorimotor comparators, yet on the other hand our earlier tests indicate that the singing-related activity of HVCX neurons is not affected by acute distortions of auditory feedback. One possible explanation for our earlier results is that the singing-related activity of HVCX neurons is a corollary discharge of song performance, such that comparison of that signal against sensory feedback occurs at some downstream location, presumably in the AFP. This seems unlikely in light of data showing that auditory activity in the AFP is silenced when HVC is inactivated (Roy and Mooney, 2009). Those data suggest that there is little or no additional auditory input beyond that point, however those data are from anesthetized birds and auditory responses can be quite different in the awake bird (e.g., HVCRA cells express strong auditory responses in the anesthetized bird but little or no response in the awake bird) (Cardin and Schmidt, 2003; Dave et al., 1998; Mooney, 2000; Nick and Konishi, 2001; Raksin et al., 2012; Schmidt and Konishi, 1998) (personal observations in awake zebra finches). Intriguingly, preliminary findings reveal auditory responses in VTA neurons that project to Area X of awake zebra finches (Las and Fee, 2008). Those data indicate that the latency of those responses is too short for activity to have passed through the AFP to Area X and then to VTA (Gale et al., 2008; Las and Fee, 2008), suggesting an alternative pathway through which auditory information may enter the AFP to be compared with singing-related activity that enters via HVCX neurons.

An alternative explanation for our previous finding that HVCX neurons are unaffected by short-term changes in auditory feedback is that the strength of corollary discharge that HVCX cells receive from recurrent motor pathways may be much greater than the strength of auditory feedback, such that effects of distorted feedback would be very difficult to detect using extracellular recordings and difficult to detect even with intracellular recordings. The manipulation of auditory input that we used (overlaying a second copy of the primary song type at a random phase delay) was quite effective at eliminating auditory action potentials in those cells, yet there was no detectable effect on HVCX singing-related activity or the bird's song performance, even when the distortion signal was played quite loudly to compete with bone conduction of the self-generated vocalization (Prather et al., 2008). Our manipulation was not sufficient to evoke changes in song behavior, but song changes are commonly observed with other, more prolonged changes in auditory feedback (Andalman and Fee, 2009; Charlesworth et al., 2011; Kozhevnikov and Fee, 2007; Leonardo and Konishi, 1999; Lombardino and Nottebohm, 2000; Sober and Brainard, 2009; Tumer and Brainard, 2007). This difference leaves open the possibility that any possible comparison occurring in HVCX cells is specific to other forms of distorted auditory feedback or may require distortion over a much longer duration in order to affect song behavior. Some cells in HVC, which are unidentified but are thought to be interneurons, are sensitive to acute alterations of auditory feedback, but feedback-induced changes in the activity of those cells are relatively subtle (Sakata and Brainard, 2008). Fundamental challenges in gaining further insight will be the need to record intrasomatically from individual neurons over many song performances (Long et al., 2010), the need to record from identified populations of neurons over durations sufficient for behavioral changes to emerge, and the need to selectively manipulate neural activity to ask whether altered activity of HVCX neurons plays a causal role in altering auditory perception or vocal performance (Roberts et al., 2010; Scharff et al., 2000; Tschida and Mooney, 2012). Attention should be given to the possibility that not only real-time error detection but also offline changes such as those that occur during sleep may play important roles in adaptive vocal plasticity (Dave and Margoliash, 2000; Deregnaucourt et al., 2005; Hahnloser and Fee, 2007; Hahnloser et al., 2006; Margoliash and Schmidt, 2010; Peigneux et al., 2004; Rasch et al., 2007; Roberts et al., 2010; Shank and Margoliash, 2009). Continued refinement of our approaches to achieve those goals will be invaluable in our quest to understand the information encoded in the activity of HVCX neurons and their role as possible sensorimotor comparators.

6 Central Questions and Important Future Directions

The findings reviewed here highlight several questions that are of central importance in our field. First, where in the brain does auditory perception emerge, and how is that perception stored and recalled as a template to guide vocal learning? As noted in section 2, CM and NCM represent features of song perception far beyond the acoustic song structure, and activity in those reciprocally interconnected sites is essential in the formation of a tutor song memory that guides juvenile learning. Therefore, it seems likely that activity in the auditory lobule is an important component of song perception, but additional studies using fluorescent imaging to examine the effect of tutoring on neurons in the song system have also revealed effects of tutor experience on sensorimotor cells in HVC. Specifically, dendritic spines on HVCX neurons are rapidly stabilized within 24 hours following a young male bird's first exposure to tutor song (Roberts et al., 2010), and focal disruption of activity in HVC also impairs song copying (Roberts et al., 2012). As noted above, cells in the auditory lobule have also been implicated in song perception and memory, and those cells receive synaptic input from HVC neurons (Akutagawa and Konishi, 2010). Together, those results indicate that a process as complex as perception, memorization and recall of a song memory in service of imitation is distributed across several brain sites. Future investigations will focus on CM, NCM and HVC as important contributors to the emergence of song perception and the utilization of song memories.

A second set of centrally important questions is: through what networks are sensory and motor signals compared, and how does activity emerging from that comparison modify subsequent motor performances to guide juvenile learning and adult maintenance of vocal signals? Initial investigations indicate that when the bird is singing, HVCX activity is not affected by disruption of auditory feedback (Kozhevnikov and Fee, 2007; Prather et al., 2008). As noted in section 4, that does not rule out the possibility that a much longer-duration mismatch between the motor command signal and the associated auditory feedback could result in the emergence of feedback-sensitivity in those cells. Such conservative responses to brief sensorimotor errors would be very beneficial to prevent plasticity in the face of transient mismatches induced by environmental noise. Sites downstream of HVCX cells have been implicated in song learning and behavioral plasticity, supporting a possible functional role for sensorimotor comparison in HVCX cells or at some downstream location (Bottjer et al., 1984; Olveczky et al., 2005; Scharff and Nottebohm, 1991; Sohrabji et al., 1990). If subsequent tests reveal that HVCX neurons are insensitive to auditory feedback under any condition, then the presence of a motor-related signal in the AFP would raise the possibility that sites in that pathway may also be part an auditory-vocal comparator circuit. Identifying the circuit(s) through which motor commands and sensory feedback are compared will also open the door to exploring how the output of sensorimotor comparator neurons is harnessed to refine juvenile motor performance as it becomes progressively more similar to a memorized song model. Presently it remains unknown which brain regions participate in sensorimotor comparison and whether the output of that hypothetical comparator may simply free the system to take on a new state, or whether it may instruct the system in a directed transition to minimize the difference between performance and model (Solis et al., 2000). Those topics will remain an important focus in our field, and songbirds will continue to be an excellent animal model to understand the circuitry and cellular mechanisms that underlie perception and performance of the sounds used in vocal communication. Because of the growing body of evidence that structures in the songbird brain are analogous, and in some cases homologous, to structures in the human brain, defining the neural basis of vocal communication in songbirds holds the promise of profoundly improving clinical therapies for human communication disorders.

HIGHLIGHTS

• Songbirds are an excellent model for studying the neural basis of vocal communication

• Insight into a basis of auditory perception and sensorimotor integration is reviewed

• Individual neurons are activated during both song perception and song performance

• Comparison of performance vs. feedback guides vocal learning and maintenance

ABBREVIATIONS

A discussion of many of these brain sites and their analogous or homologous relation to sites in the mammalian nervous system has been published in papers published by a consortium of researchers led by Anton Reiner (Reiner et al., 2004) and Erich Jarvis (Jarvis et al., 2005).

Av

Nucleus avalanche

Area X

Area X (specialized region of the avian striatum)

CM

Caudal mesopallium

DLM

Nucleus dorsolateralis anterior, pars medialis

DMP

Nucleus dorsomedialis posterior thalami

Field L

Field L (avian primary auditory cortex)

HVC

HVC (abbreviation used as a proper noun)

LLv

Ventral nucleus of the lateral lemniscus

LMAN

Lateral magnocellular nucleus of the anterior nidopallium

MLd

Nucleus mesencephalicus lateralis, pars dorsalis

MMAN

Medial magnocellular nucleus of the anterior nidopallium

NCM

Caudomedial nidopallium

NIf

Nucleus interface of the nidopallium

nXIIts

Hypoglossal nucleus, tracheosyringeal nerve (12^th cranial nerve nucleus)

Ov

Nucleus ovoidalis

RA

Robust nucleus of the arcopallium

UVA

Nucleus uvaeformis

VP

Ventral pallidum

VTA

Ventral tegmental area