Neural correlates of auditory scene analysis and perception

Abstract

The auditory system is designed to transform acoustic information from low-level sensory representations into perceptual representations. These perceptual representations are the computational result of the auditory system's ability to group and segregate spectral, spatial and temporal regularities in the acoustic environment into stable perceptual units (i.e., sounds or auditory objects). Current evidence suggests that the cortex--specifically, the ventral auditory pathway--is responsible for the computations most closely related to perceptual representations. Here, we discuss how the transformations along the ventral auditory pathway relate to auditory percepts, with special attention paid to the processing of vocalizations and categorization, and explore recent models of how these areas may carry out these computations.

1. Introduction

Imagine, for a moment, that you are at a cocktail party and surrounded by sounds: music plays in the background; your conversation partner is telling you a story; beyond you, a group of fellow party-goers are engaged in a lively debate; and a cell phone is ringing. Despite the fact that these sound sources (e.g., the phone and the stereo speaker) are happening in close temporal and spatial proximity, with many likely having similar frequency components, you are readily— and seemingly effortlessly—able to differentiate between these different sounds. But how is our auditory system able to transform the acoustic information that is a mixture of the stimuli produced by each sound source, into distinct perceptual representations (i.e., sounds, such as the music or the cell phone's ring)?

The study of this problem is often referred to as ‘auditory scene analysis’, referencing Bregman's seminal book (1990; van Noorden, 1975b). Sounds or “auditory objects” are thought to be formed as a result of the auditory system's ability to detect, extract, segregate, and group the spatial, spectral, and temporal regularities in the acoustic environment into distinct perceptual units (i.e., auditory objects or sounds) (Bregman, 1990; McDermott, 2010; Shinn-Cunningham, 2008; Sussman et al., 2005; Winkler et al., 2009). Moreover, because audition is inherently temporal, a sound can span multiple acoustic events that unfold over time and a sequence of auditory objects forms an auditory stream (Bizley and Cohen, 2013; Bregman, 1990; Fishman et al., 2004; Micheyl et al., 2007; Sussman et al., 2007a). Auditory objects and streams are the basis from which, through categorization, we can reason about and respond adaptively to the auditory environment. In the following review, we explore the conceptual framework for auditory perception and delve into the role of the cortex in mediating auditory-object and stream formation, as well as its role in auditory categorization.

2. Auditory scene analysis, streaming, and predictive regularity

According to Bregman's (1990) theory, auditory scene analysis occurs in two stages: (1) the formation of alternative organizational schemes for incoming acoustic information, and (2) the selection of one of the alternative organizational schemes to be perceived. A simple example of this two-stage process is the sound sequence displayed in Fig. 1a, which consists of tone bursts A and B, each having a different frequency. There are two ways this sequence is generally heard: (1) group all acoustic events into a single stream (Fig. 1b top) or (2) segregate the tones of different frequency into two separate streams (Fig. 1b bottom). In the first scheme, a listener would hear a galloping rhythm of tones. In the second scheme, a listener would hear two distinct streams: one consisting of tone sequences at tone A's frequency and one at tone B's frequency. Indeed, humans can hear this sequence as either one or two streams (Cusack, 2005; Denham and Winkler, 2006), suggesting that both organizational schemes can be formed.

Figure 1.

Competition between alternative organizational schemes. Consider the sequence above sequence of tones (A), which consists of a repeating low-high-low pattern. (B) One potential organizational scheme could be to group all tones into a single group, which would be heard as a “gallop” (top). Alternatively, another scheme could be two segregate the tones of the same frequency into separate groups, which would lead to the perception of two isofrequency patterns (bottom). Depending on the timing and frequency separations between the tones, humans are capable of hearing both (Cusack, 2005; Denham and Winkler, 2006), consistent with the hypothesis that auditory perception functions on the basis of competing organizational schemes (Bregman, 1990).

When multiple organizational schemes are formed, one must be selected for perception. Studies using alternating tone sequences, or those featured in Fig. 1, suggest that initially sequences tend to be perceived as a single stream but that over time the percept tends to change to hearing two distinct streams (Bregman, 1990; Cusack et al., 2004; Micheyl et al., 2005). This suggests that early in scene analysis, the default organizational scheme is to tend to integrate all events into a single stream and only over time does evidence ‘build up’ in support of an alternative scheme that segregates events into multiple streams. Perhaps a more direct example of active competition between competing organizational schemes is that of a perceptually bistable stimulus, which can be heard as either one or two streams. The perception of this stimulus as either a single or multiple streams tends to fluctuate spontaneously over time (Denham and Winkler, 2006), which suggests that both organizational schemes are represented simultaneously and compete for perception.

These organizational schemes reflect the potential ways in which the brain can parse the acoustic environment into different auditory streams. Auditory streams are formed on the basis of detected patterns, or regularities, from within the acoustic environment, under the reasonable assumption that auditory stimuli produced from a single sound source are likely to be more similar than those produced by different sources. For example, a male's voice can be differentiated from a female's voice based on differences in the voices’ formant structures. Indeed, a large literature of psychoacoustic studies has demonstrated that auditory stimuli can be segregated into unique streams based on distinct regularities in frequency (van Noorden, 1975a), spatial location (Hill et al., 2011), timbre (Singh, 1987), and others (Bregman, 1990; Grimault et al., 2002; Vliegen and Oxenham, 1999). Thus, organizational schemes reflect the detection of distinct regularities within the acoustic environment. Moreover, these detected regularities are also inherently predictive of future acoustic events that the putative sound sources are likely to produce. Thus, auditory objects and streams can be thought of as predictive-regularity representations (Denham and Winkler, 2006; Winkler et al., 2009). In this sense, alternative organizational schemes compete on the basis of how well they predict future acoustic events.

Neurophysiological evidence in support of such a predictive-regularity hypothesis has come from the numerous electroencephalographic (EEG) and magnetoencephalographic (MEG) studies on deviance detection (Winkler et al., 2009). In a typical experiment, a sequence of tones (e.g., a sequence of tone bursts at 1 kHz) or other auditory stimuli is presented in a regular pattern. This regularity is disrupted by an occasional stimulus that deviates from this pattern (e.g., a tone burst at 2 kHz). If the brain was creating predictive-regularity representations, then it is reasonable to hypothesize that the brain should differentially encode events that deviate from the expected pattern. Indeed, the ‘mismatch negativity’ (MMN) is a frontally negative event-related response that reflects the differential response to deviant versus non-deviant stimuli (Winkler et al., 2009). The MMN can be measured in response to various types of deviations, ranging from simple changes in the acoustic features of a tone sequence (Alain et al., 1999; Horvath et al., 2001; Kisley et al., 2004; Schröger et al., 1992), to more abstract deviations, such as changes in the temporal order of an acoustic pattern (Korzyukov et al., 2003; Tervaniemi et al, 1994) and omissions (Yabe et al., 1997; Yabe et al., 2001). Critically, the MMN is elicited on the basis of the perceived organization of the stimulus (Ritter et al., 2000; Sussman et al., 2007b; Winkler et al., 2005; Yabe et al., 2001), which suggests that it reflects processes that follow the selection of a predictive-regularity representation and may signal the need for the auditory system to adjust its representation of the auditory environment (Winkler et al., 2009).

Current-source density analyses have distinguished two distinct neuronal generators of the MMN signal: one located within auditory cortex along the superior temporal gyrus (Giard et al., 1990; Hari et al., 1984; Molholm et al., 2005), and another located in frontal cortices (Doeller et al., 2003; Giard et al., 1990; Lappe et al., 2013; Molholm et al., 2005). These generators are also distinguished in their proposed functions, with temporal generators largely reflecting the detection of feature-specific deviations (Giard et al., 1990; Hari et al., 1984; Molholm et al., 2005) and frontal generators potentially reflecting involuntary switching of attention in response to the changes in the acoustic environment (Giard et al., 1990; Molholm et al., 2005; Näatänen and Alho, 1995).

Studies on the underlying cortical mechanisms of the mismatch signal have found MMN-like activity in the core auditory field A1 (Fishman and Steinschneider, 2012; Javitt et al., 1994), suggesting that the detection of deviations from predictive regularities begin early on in cortical processing. However these signals may be more reflective of the effects of stimulus-specific adaptation (SSA) as opposed to a distinct and general deviance detection signal per se (Fishman, 2013; Fishman and Steinschneider, 2012). Since the MMN signal is distributed over a complex temporo-frontal cortical network (Garrido et al., 2009; Korzyukov et al., 2003; Näatänen et al., 2007; Schönwiesner et al., 2007; Szalárdy et al., 2013), it is possible that more general deviance detection signals that represent specifically the identification of violations from predictive regularities are located further downstream along the auditory pathway. Currently, the interpretation of the MMN as a representation of a top-down, predictive regularity representation versus an obligatory response relating to SSA effects is under debate (see Fishman, 2013 for details).

3. Attention and auditory scene analysis

The auditory system must not merely form these predictive regularity representations based upon the spectrotemporal regularities in an acoustic stimulus, but it must do so in the presence of competing acoustic information from other sound sources. This requires that the auditory system can identify and distinguish between the spectrotemporal regularities that are characteristic of distinct sound sources and parse the acoustic scene appropriately. Moreover, the auditory system must be able to track the evolution of these distinct regularities over time in order to maintain a Mable perceptual representation of a particular stream in the presence of these competing sounds. To accomplish this, the auditory system is capable of selectively attending to a particular stream and ignoring the competing sound sources.

Selective attention is the process by which the brain biases the processing of a particular object of interest at the expense of other objects in the environment. Recent electrophysiological work suggests that selective attention acts by enhancing the cortical representations of the attended sound streams, with population-level activity in the presence of competing sound streams reflecting primarily the activity in response to the attended stream in isolation compared to that of the ignored stream (Golumbic et al., 2013; Horton et al., 2013; Kerlin et al., 2010). These enhancements have been found primarily in low-frequency oscillatory phase (Golumbic et al., 2013; Kerlin et al., 2010) as well as high-frequency oscillatory power (Mesgarani and Chang, 2012a). Additionally, the auditory system seems to also suppress activity related to the competing stream (Horton et al., 2013), further enhancing the relative attentional gain on the attended stream. This attentional gain has also been shown to be frequency-specific (Da Costa et al., 2013; Lakatos et al., 2013), suggesting that selective attention may act as a feature-specific filter.

4. Representation of auditory objects

As noted above, auditory objects are reflections of the auditory system's transformation of an acoustic waveform from low-level sensory representations into perceptual representations. In the cortex, auditory information is hypothesized to be processed in two parallel pathways: a ventral pathway and a dorsal pathway (Griffiths, 2008; Kaas and Hackett, 1999; Rauschecker and Scott, 2009; Romanski et al., 1999). The ventral pathway is generally believed to mediate auditory perception by processing the content, identity, and meaning of a stimulus. The ventral auditory pathway begins in the core auditory fields A1 and R, which project to the anterolateral and middle-lateral belt regions of the auditory cortex (Kaas and Hackett, 2000; Rauschecker and Tian, 2000). These brain regions then project directly and indirectly to the ventrolateral prefrontal cortex (vlPFC) (Romanski et al., 1999). In contrast, the dorsal pathway mediates those computations underlying sound localization and the preparation of motor actions in response to those sounds, even in the absence of conscious perception or identification (Griffiths, 2008; Kaas and Hackett, 1999; Rauschecker and Scott, 2009; Romanski et al., 1999). In general, these computations fall under the heading of “audiomotor behaviors”. Although the dorsal pathway clearly contributes to auditory perception (Rauschecker, 2012), our focus will be on the contributions of the ventral pathway.

Whether the neural processing underlying auditory perception is strictly hierarchical or distributed is still an area of active research, and there is evidence to support both positions. For example, the results of several studies support the existence of specific areas that encode pitch (Bendor and Wang, 2005; Bizley et al., 2013; Patterson et al., 2002; Penagos et al., 2004; Warren and Griffiths, 2003). On the other hand, there is equally compelling evidence to suggest that pitch-processing is served by a number of cortical areas throughout the ventral pathway (Bizley et al., 2010; Bizley et al., 2013; Bizley et al., 2009; Garcia et al., 2010; Griffiths et al., 2010; Hall and Plack, 2009 ; Staeren et al., 2009).

Moreover, although the degree to which a brain area represents specific sound features is a matter of debate, it is clear that different areas of the cortex tend to be sensitive to different sound features. In the core auditory fields, neural activity is sensitive to a number of sound features, such as a stimulus' frequency, intensity and location, as well as more derived properties, such as timbre and stimulus novelty (Bendor and Wang, 2005; Bizley and Walker, 2009; Bizley et al., 2010; Bizley et al., 2013; Bizley et al., 2009; Javitt et al, 1994; Razak, 2011; Schebesch et al., 2010 ; Ulanovsky et al., 2004; Versnel and Shamma, 1998 ; Wang et al., 1995; Watkins and Barbour, 2011; Werner-Reiss and Groh, 2008; Zhou and Wang, 2010). Neurons in the anterolateral belt (ALB) prefer more complex stimuli, such as band-passed noise, frequency-modulated sweeps, and vocalizations (Rauschecker and Tian, 2000, 2004; Rauschecker et al., 1995; Tian and Rauschecker, 2004; Tian et al., 2001). Other areas show a greater degree of stimulus selectivity, such as neurons in the anterior portions of the temporal lobe, which are highly selective for individual vocalizations, and the voice of specific vocalizers. Finally, activity in the ventrolateral prefrontal cortex (vlPFC) seems to represent a processing stage beyond that of sensory processing since vlPFC activity reflects non-spatial auditory attention, auditory working memory, sound meaning, and multimodal sensory integration (Cohen et al., 2009; Gifford III et al, 2005; Lee et al., 2009a; Ng et al, 2013; Plakke et al, 2013a; Plakke et al., 2012; Plakke et al., 2013b; Poremba et al., 2004; Romanski et al., 2005; Romanski and Goldman-Rakic, 2002; Russ et al, 2008a; Russ et al., 2008b).

Whether neural activity reflects sensory components of a stimulus or the perceptual processing of that stimulus can be assessed directly by recording neural activity while animals are engaged in auditory tasks. For example, studies in the primary auditory cortex have found that neural activity correlates with behavioral reports of pitch (Bizley et al., 2010; Bizley et al., 2013; Bizley et al., 2009), amplitude modulation (Niwa et al., 2012), and tone contours (Selezneva et al., 2006). Although behaviorally correlated neural activity in the core does not necessarily mean the core is the locus of perceptual decisions (Gold and Shadlen, 2000, 2007), it does suggest that an important components of auditory-object processing occurs in the core.

Indeed, the results from several other studies suggest that neural correlates of perception are found in later portions of the ventral pathway. For example, MEG data suggest that the neural correlates of a listener hearing a sound, while engaged in an informational-masking paradigm, do not appear until the secondary (belt) auditory cortex (Gutschalk et al., 2008). Similarly, perceptual judgments of communication sounds (species-specific vocalizations and speech sounds) have also been found in belt region of the auditory cortex and higher auditory cortices (Binder et al., 2004; Chang et al., 2010; Christison-Lagay et al., 2011; Mesgarani and Chang, 2012b). Alternately, in studies of phonemic categorization, perceptual judgments are not found in the auditory cortex, but rather do not emerge until the level of the vlPFC (Lee et al., 2009a; Russ et al., 2008b; Tsunada et al., 2011).

It is unclear why some studies find choice-related activity as early as the core auditory cortex and others do not find it until much further downstream. However, it is clear that different parts of the auditory cortex have different stimulus preferences. Therefore, it is possible that the choice and complexity of stimuli and task may contribute to differences in where a stimulus is first perceived.

5. Representation of auditory streams

The core auditory cortex may play an important role in the neural encoding of streaming. In response to alternating ABAB tone sequences (see Fig. 1), A1 responses adapt to repeated presentations of a tone, as a functioning frequency and repetition rate. Specifically, a neuron increasingly adapts (i.e., fires less) as the repetition rate of the tones increases and as the spectral distance between the tone's frequency and the neuron's preferred frequency increases (Fishman et al., 2004; Fishman et al., 2001; Micheyl et al., 2005). In other words, A1 neurons respond more to their preferred frequency and are less suppressed by repeated presentations of this frequency than this “non-preferred” frequency.

Since A1 is tonotopically organized (Eggermont, 2001; Steinschneider et al., 1990), this pattern of firing rate suggests that the neural bases for stream segregation may be a place code. Under this hypothesis, the number of perceived streams is a function of the number of spatially separable active neural populations. In other words, this differential suppression acts to spatially separate the active neural populations such that only those neurons that are most sensitive to the tone frequency are active. The degree to which these populations can be discriminated is related directly to the level of differential suppression, as exemplified in Fig 2. For large frequency differences (or faster rates of repetition; Fig. 2a first panel), the degree of suppression is large (Fig. 2a second panel). Therefore, the populations responding to the different tones are minimally overlapping (Fig. 2a third panel) and could then be read out by downstream neurons as two distinct neural populations, giving rise to the percept of two streams. In contrast, for small frequency differences (or slower rates of repetition), differential suppression is minimal, leading to significant population overlap and a downstream readout of a single stream (Fig. 2b top three panels). In support of a neural-place code theory, a simple model of stream segregation based upon differential firing rates of neural populations in response to alternating tone sequences is consistent with the buildup of streaming in humans (Micheyl et al., 2005).

Figure 2.

Putative neural mechanisms mediating auditory streaming. The most extensive studying of the neural mechanisms of auditory streaming has focused on the processing of alternating tone sequences of varying frequency separations. (A) For alternating tone sequences with small frequency separations (first panel), neural populations that are most sensitive to one of the tones tend to be modulated by second tone as well (second panel). Given the tonotopic organization of A1 (Eggermont, 2001; Steinschneider et al., 1990), this would create essentially a single active population of neurons that could be read out downstream as representing a single stream (third panel). According to the temporal coherence model (Elhilali et al., 2009), the activity of the A and B populations exhibit high temporal coherence as well (fourth panel), which could also be read out downstream as a single stream. (B) For alternating tone sequences with large frequency separations (first panel), neural populations are only sensitive to the frequency of one of the tone bursts (second panel). Consequently, the A and B populations are effectively distinct populations (third panel) and can be read out downstream as two distinct streams. Consistent with this interpretation, the activity of the A and B populations is highly anti-coherent (fourth panel). (C) However, if the timing of the A and B tones is altered such that they are presented simultaneously (first panel), the story is slightly different with large frequency separations. Like with tone bursts with small frequency separations (see A), the neural populations are modulated by the frequencies of both tone bursts (second panel) but since the frequency separation is large, there are two distinct populations of activity (third panel) like in B. This suggests that the simultaneous tones are activating largely separate neural populations and should be perceived as two distinct streams. However, unlike B, the activity in these populations is highly coherent and is heard as a singSystream (fourth panel). Figure was adapted from Elhilali et al., 2009 and Micheyl et al., 2005.

While we explored in detail some of streaming work with respect to acoustic frequency and rates of repetition, this neural-place code is not limited to situations where streaming occurs on the basis of tone freauency. Indeed, recent work in the cat supports a neural-place code for stream segregation based upon location differences, with neurons preferentially responding to a particular spatial location aggregating in regions of primary auditory cortex (Middlebrooks and Bremen, 2013). It is also important to note that evidence for neural-place codes exists in the auditory pathway much earlier than the cortex (Pressnitzer et al., 2008). Thus, it is clear that the cortex is not necessarily responsible for the separation of neural populations under particular conditions. Indeed, subcortical processing may even extend to early specialization in the processing of vocalizations (Owren and Rendall, 2001; Portfors et al., 2009; Rendall, 2009). That a neural place code is maintained throughout the core auditory cortex suggests that it plays a role in the subsequent perception of auditory streams. However, although subcortical processing certainly contributes to auditory perception, there is no evidence to suggest that it is sufficient for auditory perception. Therefore, here, we will limit our discussion to the contribution of the cortex to perception.

In addition to a neural-place code, more recent studies suggest that the timing of neural activity also plays an important role auditory streaming. Elhilali et al. (2009) found that when the tone bursts are presented synchronously—instead of asynchronously as described above, listeners report hearing one stream. However, the neural responses in A1 to these synchronous tone sequences were similar to those elicited by the (asynchronous) alternating tone sequences. According to the neural-place code hypothesis, synchronous tone sequences with large frequency separations should be perceived similarly to alternating tone sequences (compare population profiles in Fig. 2a and 2c), which was not the case (Elhilali et al., 2009).

To reconcile this paradox, Elhilali et al. (2009) proposed a temporal-coherence model of stream segregation: streams are formed on the basis of the detection of neural populations with temporally coherent activity. Thus, for synchronous tone sequences or alternating sequences with small frequency separations, the active neural population(s) would respond in a temporally coherent manner, which could be read out downstream as evidence for a single stream. On the other hand, tone sequences with large frequency separations produce two neural populations responding in an anti-coherent manner and would be interpreted as a distinct auditory stream (Fig. 2a-c, bottom panels).

An argument for integrated temporal and place codes comes from work on environmental or background noise. While the response to environmental sounds at a population level is similar to the sum of activity in response to the different frequencies within the environmental sound, there are significant differences between the response to the environmental sound and individual stimuli at those frequencies (Rotman et al., 2001). This suggests that a simple place code is insufficient. Work from (Chandrasekaran et al., 2010) suggests that both spiking and LFP activity in response to environmental sounds are stimulus-locked, and that neural representation of environmental sounds is highly distributed across neural populations that use both place and temporal codes.

In the end, it is likely that both neural topography and temporal coherence play complementary roles in stream formation. While neural-place codes can explain the build-up of streaming, there is no clear explanation for the perceptual bi-stability of certain stimuli (Denham and Winkler, 2006). Alternatively, a strict interpretation of temporal coherence is also likely insufficient, as recent studies have found that temporally coherent sounds can, in fact, be streamed under certain conditions (Micheyl et al., 2013a; Micheyl et al., 2010; Micheyl et al., 2013b).

6. Potential mechanisms of temporal coherence

The temporal coherence model reflects an ever-increasing scientific interest in testing the relationship between auditory processing and neural-oscillatory activity (Giraud and Poeppel, 2012; Schroeder and Lakatos, 2009). Oscillatory activity represents the large-scale coordinated activity of neural populations over relatively long timescales, such as that needed for the temporal-coherence model of streaming. To be a putative mechanism for mediating temporal coherence processing, oscillatory activity should (1) reflect spectrotemporal regularities of acoustic sequences mat are useful in mediating streaming and (2) be related to the processing of acoustic events in a manner consistent with known phenomena of streaming.

Indeed, neural oscillations reliably entrain to tone sequences, with the frequency of the entrained oscillation corresponding to the repetition rate of the sequence (Lakatos, 2005; Lakatos et al., 2013). Additionally, oscillatory entrainment has been exhibited for patterned spectral modulations (Henry and Obleser, 2012; Luo et al., 2006; Patel and Balaban, 2000). Particularly, the phase-alignment of the entrained neural oscillation is directly related to the spectral distance between a cortical site's preferred frequency and the frequency of a tone burst: oscillations align to a high excitability phase during the presentation of a tone burst at the site's preferred-frequency but align to a low-excitability phase when the frequency is not preferred (Lakatos et al., 2013). Thus, neural oscillations could act as a spectrotemporal filter by differentially modulating the amount of a site's activity. In this manner, only the activity from sites that are sufficiently sensitive to a particular stimulus would be outputted for further downstream processing.

Despite these promising findings, it is still possible that oscillatory activity is simply epiphenomenal and not directly functionally relevant for auditory streaming (Shadlen and Movshon, 1999; Shadlen and Newsome, 1998). For instance, it is known that single neurons can act as coincidence detectors (Yin and Chan 1990) and information integrators (Huk and Shadlen, 2005), suggesting that computations relating to temporal coherence could occur without requiring a role for oscillatory activity. Coincidence detectors in relation to the processing of temporal coherence in auditory streaming have yet to be discovered, however, and therefore the exact mechanisms for mediating temporal coherence are not fully understood.

7. Auditory categorization

It is clear that auditory objects and stream formation are critical aspects of auditory perception, providing valuable information regarding the putative sound sources in the environment. However, an additional level of grouping of auditory information is also important: categorization. While objects and streaming are grouped on the basis of acoustics, categorization can be thought of as a grouping of objects or streams. Categorization is an adaptive process that provides high-level, abstracted representations of sensory information, which allows us to mentally manipulate, reason about, and respond adaptively to objects in the environment. Thus, in order to fully understand auditory perception, we must understand the processes involved in auditory categorization.

Similar to auditory-object formation, there is evidence to suggest that areas as early as the core auditory cortex may be have categorization-related activity. Specifically, the spatial pattern activation of the core auditory fields changes over the course of category learning: after categories are learned, stimuli within the same category evoke one spatial pattern of activity, whereas those in a different category evoke a second pattern (Ohl et al., 2001). Further supporting the role of core auditory regions in categorization is work from Selezneva et al. (2006), who found neural correlates of categorization during a contour-discrimination task. This is further supported by the work of Ley et al. (Ley et al., 2012), who trained participants in task specifically designed to create new category boundaries in an auditory task. Using fMRI, Ley and colleagues found that changes associated in processing these categories were seen in primary auditory cortex (as well as adjacent regions). However, it is also possible that the effects seen in core auditory regions may be due to feedback from higher areas (Buffalo et al., 2010; Fritz et al., 2010).

Other studies suggest that categorization-related activity occurs beyond the core fields. For example, when asked to categorize the human words bad, dad, and morphs of these words, ALB neurons in rhesus monkeys responded in a categorical fashion that mirrored their behavioral responses (Lee et al., 2009b; Tsunada et al., 2011). However, it is not until the level of the vlPFC that neural activity becomes correlated with the monkeys’ choices during this categorization task (Lee et al., 2009a; Russ et al., 2008b). The encoding of conspecific vocalizations also supports a role for secondary and higher-order auditory cortex in categorization: voice-specific (categorizing across calls) and call-specific (categorizing across vocalizers) neurons are found in the anterior temporal lobe (Perrodin et al., 2011; Petkov et al., 2008). Evidence for special processing of speech-sound categories in both anterior and posterior auditory cortex also comes from human fMRI research (Chang et al., 2010; Chevillet et al., 2013; Leaver and Rauschecker, 2010; Obleser et al., 2006; Obleser and Eisner, 2009; Obleser et al., 2010). Furthermore, categorical activity in these areas may be selective for human speech (and music) over other categories, such as birdsong and animal vocalizations (Leaver and Rauschecker, 2010). vlPFC neurons have also been shown to elicit category-related neural activity (Gifford III et al., 2005) since they code the “meaning” of vocalizations as opposed to their acoustic structure.

It is also important to note, however, that there is evidence for distributed representations of both low-level stimulus features and abstracted features or categories of the stimulus through the auditory cortex (including the posterior auditory cortex) (Belin et al., 2002; Bizley and Walker, 2009; Giordano et al., 2012; Stoereff et al., 2009; Zatorre et al., 2002). Overall, these studies suggest that while certain aspects of auditory perception are evident early on in the cortical pathway, a complete view of auditory processing requires contributions from a distributed cortical network.

8. Conclusions

In spite of the progress that has been made in the study of the neural mechanisms involved in auditory streaming, many questions yet remain as to how this process is accomplished. For instance, all of the previous neurophysiology studies have employed either passive listening conditions (Fishman et al., 2004; Fishman et al., 2001), or active listening conditions in tasks unrelated to auditory streaming (Lakatos et al., 2013; Micheyl et al., 2005) per se. A full understanding of the mechanisms responsible for auditory streaming will necessarily require a direct comparison of neural activity and behavioral reports of percepts. Moreover, it will become increasingly important to record large populations of neurons simultaneously over multiple cortical fields in order to accurately assess the roles of temporal coherence and place-codes in mediating auditory streaming, as well as further elucidating the specific roles of primary, non-primary, and multisensory cortical areas in stream formation and maintenance. These future studies across cortical areas should also address the degree to which each of these areas reflects the conscious percept of streams or the pre-conscious processing. Multiple types of stimuli should be used to establish whether streaming is a uniform, general process, or whether the complexity of the stream affects the manner in which it is processed. Furthermore, because categorization can be thought of as another form of auditory grouping and segregation, further study will need to be done to explore if and how activity associated with categorization affects streaming. Specifically, it will be important to determine the degree to which differential processing along the ventral pathway is dependent upon the acoustics of the stimuli, experience, or task demands.

Highlights.

1. Auditory system transforms acoustic stimuli into perceptual units called sounds

2. The ventral auditory pathway plays an important role in auditory perception

3. Both spatial and temporal models of rate coding needed for auditory perception