Intensity-Invariant Coding in the Auditory System

Abstract

The auditory system faithfully represents sufficient details from sound sources such that downstream cognitive processes are capable of acting upon this information effectively even in the face of signal uncertainty, degradation or interference. This robust sound source representation leads to an invariance in perception vital for animals to interact effectively with their environment. Due to unique nonlinearities in the cochlea, sound representations early in the auditory system exhibit a large amount of variability as a function of stimulus intensity. In other words, changes in stimulus intensity, such as for sound sources at differing distances, create a unique challenge for the auditory system to encode sounds invariantly across the intensity dimension. This challenge and some strategies available to sensory systems to eliminate intensity as an encoding variable are discussed, with a special emphasis upon sound encoding.

1. Introduction

An important feature of biological sensory systems is their ability to extract meaningful environmental signals under a wide variety of conditions. This ability is necessary for successful prey acquisition, predator avoidance and mate localization, among other crucial behavioral tasks. These biological systems perform remarkably well even in the face of incomplete information, signal degradation or competing signals. Systems capable of extracting relevant information consistently under extremely variable environmental conditions are termed robust, and robust sensory pattern recognition is extremely useful for the survival of many species, including humans.

One stimulus dimension across which sounds are relatively consistently perceived is intensity or sound level. In other words, as the total power of a target sound is varied over many orders of magnitude, listeners are able either to correctly identify it or to process it correctly relative to other similar sounds (Buus and Florentine, 1991; Hanna, von Gierke et al., 1986; Viemeister and Bacon, 1988). This type of intensity invariance typically represents a straightforward achievement for artificial pattern recognizers because the overall stimulus pattern (e.g., the spectrotemporal distribution of sound power) remains relatively constant as overall power is added to the signal in a linear medium such as air. A simple signal normalization in the pattern recognizer can therefore enable robust identification of the signal relative to changes in its intensity. Intrinsic nonlinearities in biological sensory systems, on the other hand, often mean that even an operation as simple as adding power to a signal could alter the neural representation of that signal. These nonlinearities are particularly strong in the auditory periphery, leaving open the question of how the central auditory system is able to create a consistent perception of a given sound as it changes in intensity.

Under environmental conditions, differences in sound intensity are often associated with differences in sound source distances. In fact, overall sound source intensity is one of the key stimulus features used to estimate target distance in adult humans (Ashmead, LeRoy et al., 1990; Litovsky and Clifton, 1992; Mershon and Bowers, 1979; Strybel and Perrott, 1984; Zahorik and Kelly, 2007). Manipulation of sound intensity (and therefore perceived loudness) leads to systematic errors in distance judgment for virtual sound sources (Mershon, Desaulniers et al., 1981). Loudness perception itself has received considerable attention (Glasberg and Moore, 2006; Plack and Carlyon, 1995; Zhang and Zeng, 1997), while leaving relatively unexplored the mechanism of perceptual invariance across intensity in the auditory system. The goal of this review, therefore, will be to focus upon some of the potential strategies available to the nervous system for encoding sensory signals over a wide dynamic range while still preserving a representation of the signal that can be exploited for invariant or nearly invariant perception of the corresponding object. Strategies that appear to be used by the auditory system will be emphasized.

2. Dynamic Range Stitching

No individual coding element (e.g., receptor or neuron) in the sensory systems of higher animals is capable of encoding the entire intensity range to which the organism is sensitive. Perhaps the most obvious means of building an invariant representation across a wide intensity range using discrete elements of much narrower intensity ranges is to construct these elements such that their individual input-output functions combine to collectively span the total range of interest. In such a case, one would expect a range of neuronal thresholds such that combining or “stitching” together individual neuronal responses would allow the sensory system to represent the full range of intensity normally available to the organism. To some degree this strategy appears to be used by the auditory system and is depicted graphically in Figure 1. Threshold measurements made in auditory nerve support the notion that individual neuronal dynamic ranges are dispersed somewhat across the total intensity range of hearing (Evans, 1972; Liberman, 1978; Liberman and Kiang, 1978; Sachs and Abbas, 1974). Thresholds of auditory nerve fibers have classically been evaluated as absolute spiking rate measures evoked by stimuli versus spontaneous rates, but similar trends hold true when statistical properties of rate responses are taken into account (Geisler, Deng et al., 1985; Young and Barta, 1986) and are logically extended when temporal information in the spike trains is considered (Carney, 1994; Colburn, Carney et al., 2003).

Figure 1. Dynamic Range Stitching.

One strategy to cover a wide overall intensity with a sensory system is to stitch individual coding elements together such that their more limited individual dynamic ranges combine to cover the total range of interest. This particular collection of 5 sigmoidal input/output functions is capable of collectively and equivalently encoding intensities from near 0 dB up to 100 dB, although each individual function would only be able to encode a relatively narrow range of intensities.

The strategy of dynamic range stitching can be also seen in responses in primary auditory cortex (A1). Figure 2 depicts the distribution of relative tone thresholds measured from over 500 neurons located in awake marmoset monkey A1. The pattern that emerges reflects essentially the same pattern seen in the auditory nerve, whereby a subset of thresholds is indeed distributed across a wide intensity range, but the bulk of thresholds trend toward lower values. Because the average dynamic range of these neurons is around 15 dB, the overall machinery of auditory encoding appears to be biased largely toward intensities within 30 dB or so of hearing threshold (Watkins and Barbour, 2010b). This finding is problematic for two reasons. First, for the “stitching” process to be most effective, the neuronal dynamic ranges would be expected to span more or less uniformly the complete intensity range of hearing, which is not the case here. In fact, prima facie evidence for the auditory system’s ability to encode loud or moderately loud sounds is surprisingly lacking from these data. The issue of not having sufficient unsaturated coding elements to account for observed intensity discrimination capabilities for loud sounds has been explored previously (Florentine, Buus et al., 1987; Viemeister, 1988a; Viemeister, 1988b). Second, a Bayesian argument could be made for a nonuniform distribution of individual neuronal dynamic ranges, but in that case one would expect on efficient coding grounds that these ranges would match at least approximately the environmental stimulus statistics. While the distribution of environmental sound intensities to which these animals were exposed is unknown, it is extremely unlikely to be biased so strongly toward intensities near the animal’s hearing threshold. In fact, these laboratory animals are reared and housed in a room with acoustically reflective walls, implying that intensities to which these animals are regularly exposed are likely to be substantially greater than the intensities regularly encountered by marmosets in the wild or reared in quiet conditions (Liberman, 1978). Given this likelihood, the finding of so many low-threshold neurons in the laboratory animals is even more striking. Therefore, the actual distribution of sound levels that appears to be represented in A1 does not match even the most basic expectations of actual sound levels that the auditory system would need to encode.

Figure 2. Probability density function of thresholds measured from 544 neurons in awake marmoset primary auditory cortex.

These measures demonstrate some neuronal coverage over a wide intensity range but a heavy skew toward the lowest intensities. The relative thresholds plotted are computed by subtracting out absolute thresholds of hearing at each frequency as determined by the collective neuronal responses. The curve plotted is the best difference of single exponentials that fits the actual data. Details of this data set can be found in (Watkins and Barbour, 2010b).

One proposal to account for this discrepancy for the stitching hypothesis involves invoking a unique subpopulation of neurons with dynamic ranges more thoroughly spanning the full intensity range of hearing (Phillips, Semple et al., 1994; Sadagopan and Wang, 2008; Suga, 1992; Suga and Manabe, 1982). The neurons in question have an expanded overall dynamic range because at intensities greater than a preferred intensity, their responses systematically diminish, hence the categorization of their input/output functions as “nonmonotonic” or “intensity-tuned.” The result is an expanded overall dynamic range relative to monotonic or intensity-untuned neurons. In some cases, the dynamic ranges of nonmonotonic neurons have been reported to be more uniformly distributed across the full intensity range of hearing than intensity-untuned neurons (e.g., see Sadagopan and Wang, 2008). In data collected from marmoset monkey A1 in the author’s laboratory, however, this distribution of thresholds and dynamic ranges of nonmonotonic neurons is still heavily biased toward lower intensities, implying that sound level representation in even this subgroup of cortical neurons is concentrated toward lower intensities (Watkins and Barbour, 2010b). Given that these animals are likely exposed to more intense sounds on average than animals in the wild, the finding of so many low-threshold neurons in the laboratory animals is even more striking.

The precise distribution of auditory neuronal input/output functions notwithstanding, a particular challenge to inferring auditory system function from these data likely relates to the discrepancy between how the data are collected in the laboratory compared with the properties of natural acoustic stimuli. In traditional laboratory studies of input/output (i.e., rate-intensity or rate-level) functions, an experimental animal is placed into an acoustic isolation chamber that enables very low background sound levels during the experiment. Additional wideband sounds present in an acoustic environment have been shown in many contexts and throughout the auditory system to affect neuronal dynamic ranges measured by tones, usually by shifting dynamic ranges toward higher intensities and possibly compressing them (Aitkin, 1991; Costalupes, Young et al., 1984; Geisler and Sinex, 1980; Gibson, Young et al., 1985; Phillips, 1985; Phillips and Cynader, 1985; Phillips and Hall, 1986; Ramachandran, Davis et al., 2000; Rees and Palmer, 1988; Sadagopan and Wang, 2008; Young and Barta, 1986). Furthermore, each probe tone in these experiments is typically presented with sufficient silence between stimulus presentations in order to avoid the response of one stimulus influencing the response of another. It is widely understood that the reason for waiting between stimulus presentations and, furthermore, randomizing the order of stimulus presentations is that the auditory system being probed has memory associated with it. In other words, the recent past affects neuronal responses because auditory neurons adapt to their recent history, thereby prompting experimental data collection procedures designed to average out these effects. Adaptation is another technique that sensory neurons can use to encode a larger overall dynamic range using individual coding elements possessing smaller dynamic ranges.

3. Adaptation

While sensory systems are capable of encoding a wide range of intensities, over relatively short periods of time a much smaller range of intensities is typically present in the environment. Figure 3, for example, depicts the mean amplitude ranges present in time intervals of different lengths for a jungle and an urban recording. Naturally, longer intervals contain a wider range of amplitudes on average than shorter intervals, which would be true for the vast majority of sounds. Even at relatively long intervals of several seconds, however, no more than 1/4 of the total dynamic range of the recording apparatus was occupied by the stimulus. A logical approach to encoding stimuli over a wide intensity range, therefore, is to adapt the narrower dynamic ranges of individual coding elements in response to recent stimulus statistics at an appropriate time scale. The effect would be to shift the dynamic ranges of coding elements to align with the most common recent signal values encountered. Depending upon the statistics of the sensory stimulus and the rate of adaptation, the coding element dynamic ranges could be relatively narrow yet still effectively encode sensory stimuli over a much greater intensity range. An additional advantage of this strategy is that it utilizes fixed resources more effectively by shifting many of them away from sensitivity to events that have an extremely low probability of either occurring or being detected or both. Adaptive processes could also explain the distribution of thresholds in Figure 2: adaptive neurons in silent conditions would naturally adapt to have their lowest thresholds and highest gains.

Figure 3. Ranges of sound amplitude over varying interval durations.

The temporal dynamics of natural sounds yield short-term fluctuations in stimulus amplitude. In both of these two examples of 20-minute ambient recordings made in two different settings, the mean amplitude range of recordings as a fraction of the maximum possible range steadily increases as longer sliding interval durations are used to calculate the averages. At short intervals under 1 s, less than 10% of the recording apparatus’s total input range was used on average to make the recording. For intervals of 20 s, this value was closer to 25%.

Adaptative coding strategies are commonly used in biological sensory systems and take numerous forms. Modulating the amount of signal energy entering the sensory system transduction apparatus is an approach used by both visual and auditory systems. The familiar pupillary light reflex represents a negative feedback circuit that constricts the pupil of the eye in response to bright light (Loewy, 1990). Two acoustic reflexes similar to the pupillary reflex dampen middle ear vibrations in the face of loud sounds: the tensor tympani reflex and the stapedial reflex. These reflexes appear to be much more dynamic than the pupillary reflex, however, contracting rapidly after the onset of loud sounds and synchronously with the speaker’s own vocalizations (Sellari-Franceschini, Bruschini et al., 1986; Silman, 1984). Presumably the adaptive nature of these reflexes is well-suited to the temporal dynamics of natural acoustics, which might reasonably require faster reaction for acoustic stimulation than the pupillary reflex requires for light stimulation.

Sensory adaptation also commonly occurs in the receptors themselves. Mechanoreceptors of the somatosensory system, for example, adapt their responses to recent pressure magnitude, apparently by alterations in their membrane properties that affect depolarization (Nakajima and Onodera, 1969). Chemical receptors for taste and smell desensitize when a ligand activating them is present for extended periods of time (Borisy, Hwang et al., 1993; Song, Cygnar et al., 2008; Wei, Zhao et al., 1998). In the visual system, photoreceptors convert light energy into a molecular conformational change that signals the presence of a photon. The photoreceptors become less sensitive upon transducing at a high rate, which occurs at higher light intensities and can be regulated by additional chemical processes (Kefalov, Estevez et al., 2005; Pepperberg, 2003). Sound is converted in the mammalian ear into neuronal signals through a mechanotransduction mechanism whereby cochlear fluid vibrations are converted into transmembrane ion flows in hair cells of the inner ear through special ion channels sensitive to mechanical forces. The mechanical properties of this transduction mechanism, as well as the chemical properties of the hair cells themselves, appear to adapt to recent stimulus history in a way that can affect the gain of this mechanism (Eatock, 2000; Holt and Corey, 2000; Le Goff, Bozovic et al., 2005).

This approach of adjusting signal gain at the initial stages of processing in order to center the much lower dynamic ranges of coding elements upon the mean signal intensity is a logical one and also represents a standard approach deployed in engineered systems. The use of automatic gain control (AGC) circuits, of which there are many types, has automated numerous processes of engineered gain adaptation. For example, photographic film and modern imaging sensors have dynamic ranges far smaller than the intensity ranges found in nature, hence the need to use an aperture to modulate the total amount of light entering the camera lens and center the dynamic range of the scene onto the sensor. Photographic camera apertures previously needed adjustment by hand to avoid overexposing or underexposing film. Modern cameras virtually all have some kind of light meter coupled to an electronically controlled aperture through an AGC circuit. AGC circuits are also present in modern telephones, hearing aids, voice recorders, etc., and simplify the process of using such devices by insulating the user from the need to adjust gain by hand. One potential drawback of this approach, however, is that the memory inherent in AGCs introduces a nonlinearity that can distort some signals excessively.

Downstream adaptation to stimulus statistics can take multiple forms and can occur across different levels within a sensory system. Raw intensity is largely adapted out early in the visual pathway, and higher order adaptations such as to stimulus contrast become more prevalent at later processing stages (Baccus and Meister, 2002; Brown and Masland, 2001; Chander and Chichilnisky, 2001; Hosoya, Baccus et al., 2005; Kim and Rieke, 2001; Laughlin, 1989; Rieke and Rudd, 2009; Shapley and Victor, 1978; Smirnakis, Berry et al., 1997; Solomon, Peirce et al., 2004). This scenario is advantageous for the visual system because images are formed by contrasts between light and dark areas rather than overall intensity. A pattern recognizer for a face, for example, would ideally have similar performance in a dim room as on a bright day so long as sufficient contrast existed in the image. Early removal of absolute intensity is an evolutionarily adaptive and efficient solution for extending the dynamic range of individual photoreceptors to create the full range available to the visual system (Hosoya, Baccus et al., 2005). This intensity information is still acquired and transmitted to the brain by an alternate pathway, which enables perceptual judgments of brightness as well as proper function of the pupillary light reflex.

A similar argument can be made for somatosensation, where changes in absolute pressure might occur with some low frequency but not represent the most relevant sensory information. Sitting, wearing clothes or holding an item are all actions that will create substantial pressures on the mechanoreptors in the skin. While some information about absolute pressure detected by the skin would be important to represent (e.g., to be able to pick up an egg without crushing it), the most relevant ongoing information to represent is likely to be fluctuations in pressure at relatively short time scales (e.g., to determine if a recently touched surface is smooth or rough). This is equivalent to saying that the AC signal is likely to carry more useful information than the DC signal or that low-probability (novel) stimuli may have more behavioral relevance than signals that have been present for some time.

Adaptation in response to recent stimulus history also occurs in the auditory system and can be detected in auditory nerve fibers (Relkin and Doucet, 1991; Rhode and Smith, 1985; Wen, Wang et al., 2009) in addition to more centrally located neurons (Dean, Harper et al., 2005; Dean, Robinson et al., 2008; Kvale and Schreiner, 2004; Watkins and Barbour, 2008; Watkins and Barbour, 2010a). Like adaptation in other sensory systems, auditory adaptation appears to be present within the transduction mechanism itself (Goutman and Glowatzki, 2007), as well as downstream (Bartlett and Wang, 2005; Nelson, Smith et al., 2009; Ulanovsky, Las et al., 2004; Xu, Kotak et al., 2007). This adaptation results in neuronal input/output functions being modified to align their regions of maximum slope more closely with the most probable stimuli, thereby maximizing discriminability for the most probable stimuli as well as overall information transmission (Brenner, Bialek et al., 2000; Dean, Harper et al., 2005; Dean, Robinson et al., 2008; Kvale and Schreiner, 2004). This is equivalent to saying that the distribution shown in Figure 2 becomes less leftward skewed (i.e., the mean threshold increases) as the average sound intensity increases. Important exceptions to this rule have been reported for some response types, however, that preserve sensitivity to lower probability stimuli of presumably significant behavioral relevance (Watkins and Barbour, 2008; Watkins and Barbour, 2010a). This sensitivity preservation could be particularly useful under conditions of multiple concurrent dynamic sounds, particularly if each sound has a different mean intensity. Consider, for example, how the two stimuli depicted in Figure 3 would best be encoded by elements of narrow dynamic ranges if they were present simultaneously.

Unique psychophysical phenomena can be used to infer adaptive physiological mechanisms (Wolfson and Graham, 2007; Wolfson and Graham, 2009) and, conversely, physiological evidence for adaptive phenomena can be used to predict the existence of particular psychophysical phenomena (Zeng, Turner et al., 1991). In the latter case, neurophysiological experimental results indicated that high-spontaneous rate (low-threshold) auditory nerve fibers recover from adaptation induced by a loud preceding sound an order of magnitude faster than low-spontaneous rate (high-threshold) fibers (Relkin and Doucet, 1991). This kind of experimental protocol is termed forward-masking to signify that the output of interest is the threshold shift or “masking” induced forward in time by a preceding stimulus. Very shortly following a loud sound, then, psychophysical hearing thresholds are masked because all the neuronal thresholds have been shifted upward (Figure 4A,B). Waiting a bit longer, however, (up to 400 ms in these experiments) creates in intriguing situation where the most sensitive neurons have almost completely recovered their initial thresholds while the least sensitive neurons have not yet begun to recover (Figure 4C). The result is about a 20 dB “coding gap” where almost all of the auditory neurons are either saturated or nonresponsive. A straightforward prediction of this neurophysiological finding is that psychophysical thresholds would be expected to be elevated at this intermediate intensity range for the time scales in question. Thresholds for greater or lesser intensities than the gap location would be lower, thereby creating a nonmonotonic threshold curve as a function of intensity. When intensity discrimination thresholds were measured in human listeners shortly after an intense masking stimulus, this hypothesis was borne out robustly (Zeng, Turner et al., 1991). Given that intensity-tuned neurons all the way to auditory cortex appear to be the primary inheritors of high-spontaneous rate/low-threshold auditory nerve fiber activity (Watkins and Barbour, 2010b), these central neurons might be expected to have distinct adaptive properties, as well. This prediction has been shown to be correct, leading to the surprising finding that intensity-tuned neurons adapt in such a way not to maximize encoding of overall stimulus statistics, but to maximize encoding of the particular case of soft sounds preceded by loud sounds, much as in the case of the forward masking paradigm (Watkins and Barbour, 2008; Watkins and Barbour, 2010a). The complete role of these neurons in encoding dynamic acoustic stimuli remains the subject of further investigation.

Figure 4. Effects of a loud sound on the auditory system’s ability to encode subsequent tones.

Two classes of neurons contribute to the population of auditory nerve fibers. One class exhibits high spontaneous rates, low thresholds and rapid recovery from forward masking. The other class exhibits low spontaneous rates, high thresholds and slow recovery from forward masking. A. In silence the two groups span a wide intensity range with their collective dynamic ranges. B. After a loud preceding stimulus (100 ms, 900 – 1000 Hz noise at 90 dB SPL), both groups adapt toward higher intensities. C. Because the low-threshold neurons recover from the effects of the loud tone more quickly than the high-threshold neurons, for a short time the auditory system is relatively “deafened” to intermediate intensities, as probed by tones (25 ms, 1000 Hz) of different intensities presented within 400 ms following the offset of the masker. (Adapted from Zeng, Turner et al., 1991)

Peculiar adaptations such as these bear upon issues of efficient coding with limited resources and sensory discrimination across stimulus intensity but do not directly address the issue of invariance in perception across intensity. In fact, given the previous discussion that the neural code actually changes depending upon the temporal stimulus context and multiple strategies to implement optimal stimulus encoding, achieving an invariant neural representation across intensity likely represents a considerable challenge for downstream circuits to implement. This task is made considerably more difficult in the auditory system because of the intensity-dependence of neuronal receptive fields beginning at the auditory periphery.

4. Variation in Neuronal Responses Across Intensity

The nature of the mechanical vibrations in the cochlea leads to a map of acoustic frequency onto position along the cochlea. The nonlinear nature of cochlear mechanics leads to an alteration in vibration pattern as sound intensities increase such that cochlear auditory filters become less selective to frequency (i.e., filters have increasing bandwidth) as stimulus intensity increases (Glasberg and Moore, 1990; Kiang, Watanabe et al., 1965; Liberman and Kiang, 1978). Potentially compounding this situation is a finding in songbird auditory neurons that not only does the spectral filter change with intensity, but so can the temporal filter, most likely due to additional nonlinearities (Nagel and Doupe, 2006). While intensity represents a straightforward stimulus parameter to normalize out in engineered linear systems (or even in some engineered nonlinear systems such as those with automatic gain control circuits), the auditory system presumably must dedicate substantial neural resources toward inverting nonlinearities introduced in the auditory periphery by the sensory transduction apparatus.

All auditory nerve fibers exhibit an increasing bandwidth with increasing intensity, resulting in a filter shape resembling a “V” when response is plotted as a function of both intensity and frequency. Many central auditory neurons also share this response characteristic, even in auditory cortex. Another set of central auditory neuronal response classes, however, has been described by numerous authors as “level tolerant” because their bandwidths remain relatively constant with sound level or intensity (Ehret and Schreiner, 1997; Sadagopan and Wang, 2008; Suga and Manabe, 1982; Suga and Tsuzuki, 1985; Sutter, 2000). In some cases these neurons have a monotonic rate-intensity profile, while in others they are nonmonotonic or intensity-tuned. Taken together, these three classic response types have been classified as “Type V,” “Type I,” and “Type O” responses based upon the shape of their frequency response areas, with the latter two corresponding to level-tolerant neurons (Ramachandran, Davis et al., 1999). Model neurons of all three classes with the same characteristic frequency, threshold and bandwidth at 10 dB above threshold are depicted graphically in Figure 5.

Figure 5. Classic auditory response types as a function of sound frequency and intensity.

Pure tones varied in frequency and amplitude (intensity) are typically used to evaluate responsiveness of auditory neurons. Auditory nerve responses all exhibit some form of increasing frequency bandwidth as intensity increases, referred to as a Type V response. Central auditory neurons may have Type V responses or intensity-independent bandwidths. If the input/output function of one of these latter neurons uniformly increases or saturates with increasing intensity, it is referred to as a Type I neuron. If, on the other hand, neuronal output diminishes above a particular best intensity, it is referred to as a Type O neuron. (From Chen, Watkins et al., 2010)

Type I and type O neurons are not present in the auditory nerve and must therefore be created by neural circuitry in the auditory system. Sadagopan and Wang (2008) have suggested that the express purpose for the creation of these neurons is to counter the frequency-dependent nonlinearities in the cochlea that give rise to intensity-dependent bandwidths in the auditory nerve. This is a compelling hypothesis that is consistent with the additional hypothesis that auditory receptive field bandwidths gradually narrow along the ascending auditory pathway from the periphery to primary auditory cortex (Suga, 1995). Neural pathways in other sensory systems tend to produce increasing receptive field sizes more centrally through divergent projections, so the unique nature of the frequency-dependent nonlinearities in the auditory system may be the driving force behind the putative narrowing of auditory receptive fields. Questions remain regarding these hypotheses, however. Auditory neurons with nonmonotonic input/output functions are nonexistent in auditory nerve but become steadily more common at higher auditory centers until they represent a majority of the neurons in A1 (Watkins and Barbour, 2010b). Of considerable interest, however, is the sizable minority of neurons that still exhibit Type V responses similar to those observed in the auditory nerve. Why would this type of response be valuable for the auditory system to retain all the way to cortex if a major purpose of the intervening circuitry was to eliminate that type of response?

Furthermore, the type of decoder necessary to extract intensity-invariant natural stimuli from any of these three neuronal subpopulations is not entirely clear. Figure 6 depicts a plausible map of characteristic frequency, bandwidth and threshold of model neurons in a square grid representing A1 (Chen, Watkins et al., 2010). This version of the map assumes a uniform distribution of thresholds, as depicted in Figure 1. By combining the models of neuronal responses depicted in Figure 5 with the map in Figure 6, a graphic depiction can be obtained of the spatial activity of neurons driven by any particular stimulus. In fact, one could imagine performing a functional imaging experiment on a similarly arrayed collection of real neurons to discern the pattern of activation for different stimuli. A simulated version of such an experiment was done in this case for two steady-state vowels at two different intensities. Each model neuron was assumed to act as a simple linear integrator of stimulus energy falling within its frequency response area (i.e., the dark areas plotted in Figure 5). Population response patterns for each neuronal subtype were calculated separately, as was a mixture of all three types combined in equal proportion. The results can be seen in Figure 7.

Figure 6. Neuronal feature maps in a model of primary auditory cortex.

Self-organizing feature map algorithms were used to create plausible maps in A1 of neuronal characteristic frequency, receptive field bandwidth and response threshold (Watkins, Chen et al., 2009). A1 is depicted here as a square grid of neurons, and the three maps shown all exist superimposed upon the same grid. Neurons in the bottom right corner of the grid, for example, would respond to intermediate frequencies with relatively high bandwidths and relatively low thresholds. (From Chen, Watkins et al., 2010)

Figure 7. Simulated functional imaging results for two vowels at two different intensities.

Simulated functional imaging experiments for the A1 feature maps in Figure 6 applied to the neuronal subtypes depicted in Figure 5. In this case two steady-state vowels (spectra shown in insets) were presented to the model network at two different intensities. At low intensities, Type V neurons are actually the most selective and result in the least overall activation. They become the least selective at high intensities, however. Type I neurons might be expected to be visibly more selective than Type V neurons at high intensities, but do not appear to be so with these wideband sounds. Type O neurons unsurprisingly show no change in selectivity at different intensities. The most striking result is that in all neuronal subpopulations and their combination, activity patterns are most similar between two perceptually different stimuli at the same intensity.

The vowel-driven population response patterns for linear Type V and Type I neurons appear to be rather similar to one another for a given stimulus, especially at higher sound levels. This finding was somewhat surprising given the extremely wide bandwidths of Type V neurons at higher intensities, but could potentially be brought about by the wideband nature of the stimulus. Type V neurons do result in a sparser population response at lower intensities, as would be expected from their tapered frequency response areas at the lowest intensities. Type O neurons represent a sparser population at higher intensities. Because of the nature of the model, however, all three neuronal classes exhibit overcomplete representations of the stimuli. Noticeable is how similar the response patterns of all the subtypes and the pooled population are to perceptually different stimuli at the same intensity. Additionally, the response patterns to the same stimuli at different intensities exhibit little resemblance to one another. For any vowel, type O responses show a unique pattern of activation at different intensities. The similarities between Type O responses to different vowels at the same intensity, however, are just as striking as the differences between responses to the same vowels at different intensities, raising a logical question regarding what kind of advantage for downstream decoding the “place code” Type O neurons might provide relative to other response classes (Shamma, 2003). In fact, a level-invariant decoder based upon Type O responses would not have a fundamentally different structure than for Type I and possibly even Type V responses.

Known flaws exist in this computational model that remove it somewhat from physiological reality. Realistic silence-adapted threshold distributions were not taken into account, for example; had they been, though, the model would predict even more population activity for the vowel stimuli. Furthermore, adaptation phenomena were not taken into account in this model. Cortical neurons are also unlikely to be purely linear integrators of sound energy at different frequencies and intensities, especially for wideband sounds such as those used here. In fact, studies of auditory receptive fields using wideband stimuli at different intensities have revealed intensity invariance, even in the auditory nerve (Barbour and Wang, 2003; Calhoun, Miller et al., 1998; Valentine and Eggermont, 2004; Yu and Young, 2000). This phenomenon is apparently mediated by adaptive processes away from a neuron’s characteristic frequency (Blake and Merzenich, 2002). In like fashion, songbird vocalizations appear to be extracted reliably across intensity by some auditory neurons (Billimoria, Kraus et al., 2008). Frequency-dependent nonlinearities in the cochlea therefore result in intensity-dependent neuronal bandwidths, which appear to be most prominent when measured with pure tones in isolation and are not as prominent when other stimuli are used. The question remains whether the neurons that account for invariance in perception across intensity have already been identified in A1 and are simply in need of proper experimental probing to reveal this fact, or if more downstream circuits must be involved to decode the A1 responses. In any case, Figure 7 remains a graphic testament to the nature of the problem the auditory system must solve. Stated another way that can be used to formulate specific research questions, what nonlinearities can be added to this linear model to create an intensity-invariant neural code?

One interesting parallel can be made between the auditory system and another sensory system along these lines. Like the auditory system but unlike somatosensation or vision, receptors in olfaction also lose specificity at higher odorant concentrations, and this phenomenon has actually been visualized by real functional imaging studies like the one simulated in Figure 7 (Ng, Roorda et al., 2002). Like the auditory system, downstream neural circuitry appears to be performing a sparsification on the stimulus representation, resulting in more narrowly “tuned” neurons than receptors. One proposal for how this might occur is that some downstream circuits are specialized to evaluate ratios of odorant concentrations rather than absolute concentrations (Uchida and Mainen, 2007). This idea could be the olfactory equivalent of the observation that central auditory neuron receptive fields tend to maintain the same bandwidth when measured by wideband stimuli. Nevertheless, the auditory and olfactory systems are still quite capable of processing single tones and single odorants in isolation, so this hypothesis is unlikely to account for the full abilities of the respective sensory systems. Modern methods of dynamical systems analysis may begin to reveal solutions to invariant stimulus encoding across stimulus intensity not with individual neurons but with dynamic networks of continually interacting neurons (Galan, Weidert et al., 2006; Stopfer, Jayaraman et al., 2003).

5. Concluding Remarks

The process by which images are parsed for behaviorally relevant content may be referred to as “object recognition.” The process by which this task takes place over noisy variations in signal power is known as “robust object recognition.” While a physical object may persist unaltered in the environment, variations in lighting, distance, viewing angle, orientation and intervening objects can dramatically alter the initial visual representation of that object. Nevertheless, the higher visual systems of many animals implement astounding robust object recognition engines.

An equivalent ability to segregate individual sound sources from interfering acoustic stimulation is possessed by the auditory system. The rules by which invariant perception is achieved by these two systems are likely to differ, but the result is similar. The intensity domain is one arena likely to differ substantially between vision and audition. Retinotopic-specific nonlinearities do not dramatically distort early visual representations the same way that frequency-specific nonlinearities distort early auditory representations. Intensity therefore reflects an intriguing stimulus parameter to ponder from the perspective of robust sound recognition and a challenging one to probe effectively. It is likely that in the process of peeling away the neural mechanisms underlying perceptual invariance of sounds in the face of variable intensity, other rules of stimulus encoding affecting perception will also be elucidated. The result will be an improved understanding of the neural representation of complex sounds available to downstream cognitive processes to assess, interpret and ultimately act upon.