Neural Representations of Pitch in Auditory Cortex of Humans and Other Primates

Abstract

Pitch perception is crucial for vocal communication, music perception, and auditory object processing in a complex acoustic environment. The neural representation of pitch in the cerebral cortex has long been an outstanding question in auditory neuroscience. Several lines of evidence now point to a distinct non-primary region of auditory cortex in primates that contains a cortical representation of pitch.

Introduction

Our ability to hear pitch allows us to determine if an acoustic signal sounds “higher” or “lower” than another acoustic signal. We can track pitch changes over time to hear a musical melody or to recognize intonations in tonal languages like Chinese, and do so robustly across different musical instruments or speakers. Two musical instruments have dissimilar sounds because of how they spectrally shape the acoustic energy they produce (i.e., which frequencies are amplified or attenuated). Yet they can still play the same musical note, and thus the same pitch, if the fundamental frequencies of the acoustic waveforms match. For a string instrument such as a violin, the fundamental frequency is equal to the vibration frequency of the a plucked or bowed string. In a more general sense, this fundamental frequency is related to the temporal periodicity of the sound’s acoustic waveform [1].

How does the auditory system extract the fundamental frequency from the complex spectrum of a sound in order to generate a percept of pitch? Several recent findings indicate that there is a specialized region in the auditory cortex of primates that is involved in the representation of pitch [2-10]. Here we review several key studies in the identification of a pitch processing center in primate auditory cortex, and discuss issues concerning the neural substrate of pitch perception.

1. What is pitch?

Pitch is defined as “that attribute of auditory sensation in terms of which sounds may be ordered on a musical scale” (American Standards Association). The key distinction between frequency and pitch is that the former is a physical description of a sound, whereas the latter is a perceptual attribute of a sound. Although frequency and pitch are typically similar for sounds composed of a single spectral component (pure tones), this relationship becomes more complicated when sounds are composed of multiple spectral components (e.g., harmonic complex sounds). For such sounds, the pitch is related to their fundamental frequency. Spectrally, the fundamental frequency can be thought of as the highest frequency for which the spectral components of the sound are integer multiples. Thus a harmonic complex sound consisting of 100, 200 and 300 Hz tones has a pitch of 100 Hz (Figure 1A). Temporally, pitch is typically related to the repetition rate of periodic amplitude envelope changes in a complex acoustic signal. For example, a sequence of brief bursts of broadband noise repeated periodically at 100 times per second has a pitch of 100 Hz (Figure 1B).

Figure 1.

Example of the spectrum and acoustic waveform for three different acoustic signals that have the same pitch (100 Hz).

A. Harmonic complex tone: composed of the 1^st (fundamental), 2^nd, and 3^rd harmonic, each with randomized phase.

B. Repeated broadband noise burst (100 Hz repetition rate).

C. Missing fundamental harmonic complex tone: composed of the 2^nd, 3^rd, and 4^th harmonic, all in cosine phase.

Both speech and animal vocalizations contain spectrally complex, harmonically structured sounds. Pitch acts to group these harmonics together into a single percept that is related to the periodicity of the source generating a sound (the vibration of the vocal apparatus). Pitch perception, a necessary feature of our auditory system, allows us to hear two speech sounds as distinct from each other when both have similar spectrums and source locations but differ in their fundamental frequencies, such as the same word spoken by a male and a female speaker standing next to each other. One of the remarkable features of the pitch perception is that a spectral component does not need to be present at the fundamental frequency in order for a pitch equal to the fundamental frequency to be perceived [1]. In other words, although individual components of a harmonic complex sound consisting of 200, 300, and 400 Hz tones do not generate a pitch of 100 Hz when they are played one at a time, a listener would hear a pitch of 100 Hz when these components are played together, even though there is no energy at 100 Hz (Figure 1C). This is a well-known phenomenon called “missing fundamental pitch” and is a hallmark of pitch perception [1]. Our ability to perceive pitch from missing fundamental harmonic complex sounds explains why we are able to hear the pitch of someone’s voice over the telephone system that effectively cuts out lower frequencies where fundamental frequencies of human voices are normally present.

2. Potential neural representation schemes of pitch in the brain

Acoustic signals can be mathematically analyzed by the Fourier transform which decomposes a sound into its spectral components. The auditory system performs a similar transformation in the cochlea where low frequency sounds or components excite the apical end of the basilar membrane, while the basal end responds best to high frequency sounds or components [11]. This tonotopic representation is preserved at each level of the auditory system, up to and including auditory cortex, such that neurons typically respond to a restricted range of frequencies, and are physically ordered within a brain area by the frequency to which each neuron is most sensitive (the “tonotopicity” or “tonotopic map”). While tonotopicity provides an explicit representation of frequency, it does not provide an explicit representation of pitch for complex sounds. Previous MEG experiments in humans have suggested that a pitch map coexists with the tonotopic map in primary auditory cortex (AI) [12-13]. Pitch map topographies both parallel [12] and orthogonal [13] to AI’s tonotopic map have been proposed, but these topographies have not been confirmed using imaging techniques that directly measure spatial locations of neural activity with sufficient resolution. Another possibility is that pitch is processed in parallel with frequency but in separate brain regions such that frequency is represented topographically in AI and pitch is represented in a non-primary area. In the pursuit of neural mechanisms for pitch perception, human and animal studies have been complementary to each other. In order to bridge pitch-related research between animal and human studies, it is important to establish similarities in how pitch is perceived as well as the anatomical and physiological properties of the brain areas studied. In this regard, non-human primate models play an essential role to help reveal neural mechanisms of pitch perception.

3. The perception of pitch is not unique to humans

An advantage of using non-human primates as experimental models for studying pitch-related questions is that the frequency range of their hearing is similar to that of humans [14]. The ability of monkeys to hear the pitch of the missing fundamental was first demonstrated in a behavioral study by Tomlinson and Schwarz (1988) [15]. Rhesus monkeys were trained to push a button if the two complex tones presented sequentially had the same fundamental frequency. The monkeys were able to perform the task even when a missing fundamental complex tone was used, demonstrating their ability to hear the missing fundamental pitch. The ability to perceive this missing fundamental is not unique to primates, and has also been shown in several other animal species including birds [16] and cats [17]. In addition, monkeys have been shown to be capable of spectral pitch discrimination [18], melody recognition [19-20] and octave generalization [21], each of which requires the perception of pitch. One important difference between humans and some animal species is the size of their cochlea. For smaller cochleas, excitations on the basilar membrane caused by individual components of a harmonic complex tone are more closely spaced than those in larger cochleas. Different pitch processing mechanisms have been postulated for sounds with harmonics producing segregated regions of excitation on the basilar membrane (“resolved harmonics”) and for sounds with harmonics that are too close together and only produce a single region of excitation (“unresolved harmonics”) [1,11]. Thus animals with smaller cochleas may need to rely more on the pitch processing mechanism that utilizes unresolved harmonics than do humans.

4. Anatomical and physiological similarity of auditory cortex among primates

The anatomical study by Brodmann (1909) [22] suggested that the structure of the temporal lobe is largely preserved across primate species (New World monkeys, Old World monkeys, and humans). Studies in recent years have revealed distinctions between various auditory cortical areas using anatomical [23-29] and physiological criteria [23,24,30-39]. Humans and monkeys appear to share a similar organization of primary, primary-like, and secondary cortical areas based on accumulating evidence [26,27], suggesting a generalizable structure and function of auditory cortex among primates.

Primate auditory cortex is divided into a core region of primary and primary-like areas surrounded by a belt of multiple secondary areas. In humans and monkeys, the core areas of auditory cortex can be distinguished from belt areas by their cytoarchitecture, namely a more prominent granular layer [23,24,27,29]. In monkeys the core areas have been shown to receive thalamic inputs from the principal (ventral) nucleus of the medial geniculate body (MGB) [26]. In contrast, the belt areas receive more dominant thalamic inputs from the non-leminscal divisions (dorsal and medial) of the MGB [26]. In both monkeys and humans, neurons in core areas respond strongly to narrowband sounds such as tones, whereas neurons in belt areas respond better to more complex sounds (e.g., noise, frequency modulation, low-contrast spectrums, and vocalizations) [32,36-39]. Within the core areas, for both monkeys [23,24] and humans [33], two mirror symmetric tonotopic maps sharing a low-frequency border have been identified, corresponding to AI and the rostral field (R). In humans, the core areas are generally confined to Heschl’s gyrus [27], with AI located medially to R (Figure 2A). However, this is difficult to determine precisely by anatomical landmarks due to substantial intra-subject variability. Monkeys do not possess an anatomical landmark for the location of AI, and the location of AI must be determined physiologically or histologically. AI is typically buried within the lateral sulcus in monkeys, except for a few New World species (e.g. marmosets, owl monkeys) for which the lateral portion of AI is located on the surface of the superior temporal gyrus [7,23]. There is anatomical [23] and physiological evidence [7,23] of a third core area (RT) that lies rostral to R (Figure 2B). Kaas and Hackett [26] have postulated that each core area is connected to a medial and lateral neighboring belt area, with additional belt areas located on the rostral and caudal ends of the core region of auditory cortex (AI, R and RT). Three of these lateral belt areas (CL, ML, and AL) have been mapped electrophysiologically, and have been found to possess similar mirror symmetric tonotopic maps as their adjacent core areas [38,39]. Additional higher auditory areas such as parabelt [40-41] and the rostral pole [42] have been defined anatomically, but little is known about their physiological properties.

Figure 2.

Diagram of human and marmoset auditory cortex

A. Side view of a human brain (top), horizontal cross section of temporal lobe (middle), and zoomed in view of Heschl’s gyrus (bottom). Primary auditory cortex is presumed to occupy the medial portion of Heschl’s gyrus (with variability between subjects). The location of neighboring areas (R, pitch center, lateral belt) is an approximation based on Schneider et al. [6], Formisano et al. [33], and Patterson et al. [3].

B. Side view of a marmoset monkey’s brain (top) and a zoomed in view of the temporal lobe (bottom), indicating core, belt, parabelt, and the pitch center. The borders between each auditory area are estimated based on data from Bendor and Wang [7], and Pistorio et al. [60].

Legend:

HG-Heschl’s gyrus, STG-Superior temporal gyrus, ITG-Inferior temporal gyrus, aSTG-Anterior superior temporal gyrus, PT-Planum temporale, SI-Intermediate sulcus, HS-Heschl’s sulcus, CS-Circular sulcus, FTS-First transverse sulcus, LS-Lateral sulcus, STS-Superior temporal sulcus, AI-Primary auditory cortex, R-Area R (Rostral auditory cortex), RT-Area RT (Rostrotemporal auditory cortex)

5. A pitch-processing center in primate auditory cortex

Where is a sound’s pitch encoded in the brain? The information needed to extract the pitch of a complex sound is contained in both the discharge rates and temporal firing patterns of the population of auditory nerve fibers [43-45]. There is no clear evidence that subcortical stations of the ascending auditory pathway explicitly encode pitch. In cats, bilateral lesions of the entire auditory cortex impair the discrimination of changes in the pitch of the missing fundamental, but not changes in frequency alone, demonstrating the necessity of auditory cortex in pitch perception [46]. Humans with partial auditory cortex lesions have also been shown to have poorer pitch discrimination [47], with lesions of the right auditory cortex showing larger deficits [48-49]. Behavioral studies in patients with auditory cortex lesions have further suggested that lesions anterior to primary auditory cortex result in more pronounced deficits in pitch discrimination [49]. These studies point to a possible cortical source of pitch representation. Several recent studies have identified a specific region in primate auditory cortex that appears to be involved in representing a sound’s pitch [2-10]. A schematic showing the location of this pitch-processing center in relation to its neighboring cortical areas for both humans and monkeys is shown in Figure 2.

Evidence from human studies

In an fMRI study, Patterson and colleagues [3] identified a specific region of human auditory cortex (lateral Heschl’s gyrus), which was preferentially activated by temporally regular sounds with a pitch. The acoustic stimulus used in this study is iterated rippled noise (IRN), which is generated by iteratively adding delayed broadband noise [1,50]. By adjusting the delay and the number of iterations, the fundamental frequency and pitch salience of the resulting sound can be changed, respectively. For IRN sounds with low frequency pitches, auditory filters in the high frequency range cannot distinguish between noise and IRN stimuli, due to the spectral resolvability limitation, even though these two sounds have physically different spectrums. Yet, subjects hear a pitch from IRN sounds but not from noise because of the temporally regular acoustic structure of the IRN sound (extracted presumably by temporal pitch mechanisms). By subtracting the Blood Oxygenation Level Dependent (BOLD) signal originating from the IRN sound from that evoked by noise, Patterson and colleagues determined that only lateral Heschl’s gyrus, a non-primary auditory region anterolateral to primary auditory cortex, responded to the temporal regularity or pitch of the acoustic stimuli (this occurred bilaterally) [3].

In another imaging study by Penagos, Melcher, and Oxenham [4], the BOLD signal was compared between four harmonic complex sounds that had either a low or high pitch and occupied either a low or high spectral range. Of these four sounds, only the harmonic complex sound with the low pitch and high spectral components had unresolved harmonics. Sounds with unresolved harmonics have a weaker pitch salience than sounds containing resolved harmonics, for which spectral cues are available [1]. Thus, Penagos et al. [4] were able to compare BOLD signals between sounds evoking a strong or weak pitch salience, but matched in their fundamental frequency or spectral range. Bilaterally, a restricted region of non-primary auditory cortex, anterolateral to AI, was found more weakly responsive to the sound with low pitch salience than the other three sounds with high pitch salience [4]. This study therefore confirms the location of the pitch-processing center identified by Patterson et al. [3] using a different type of acoustic stimuli and extends the earlier finding to demonstrate the sensitivity for pitch salience within this pitch-processing center.

The significance of the lateral Heschl’s gyrus in pitch representation was further investigated by Schneider et al. [6] using a combination of psychophysics, anatomical MRI scans and MEG measurements. For harmonic complex tones with a few components, a person may hear the pitch increasing or decreasing when the frequencies of harmonics are decreased while the fundamental frequency is increased [1]. Different subjects showed a bias towards using the fundamental frequency or spectrum frequency when discriminating pitch changes. These biases were found to be highly correlated with hemispheric asymmetry in relative size between the right and left lateral Heschl’s gyrus (and not with other regions of auditory cortex) [6]. Subjects that relied more on fundamental frequency to discriminate pitch had a larger left lateral Heschl’s gyrus, while subjects using spectrum frequency had a larger right lateral Heschl’s gyrus. In addition, MEG responses recorded from a source estimated as lateral Heschl’s gyrus showed a similar asymmetry between hemispheres. Stronger responses were obtained from the left hemisphere for subjects with a bias towards using fundamental frequency in pitch discrimination. In contrast, subjects with a bias towards using spectrum frequency had larger MEG responses recorded from the right hemisphere.

Evidence from non-human primate studies

Given the evidence for a pitch-processing center in human auditory cortex and the anatomical and physiological similarities of auditory cortex between humans and monkeys, it is reasonable to expect that monkeys possess a pitch-processing center in their auditory cortex as well. Schwarz and Tomlinson [51] searched for single-unit responses to the fundamental frequency of missing fundamental harmonic complex sounds in AI of three awake macaque monkeys previously trained on a pitch discrimination task, but failed to find any neurons responsive to the fundamental frequency centered at neuron’s characteristics frequency (CF). Schwarz and Tomlinson concluded that pitch is either represented implicitly across a population of neurons in AI or an explicit representation exists outside of AI. However, Fishman et al. [52] were unable to find an implicit representation for the missing fundamental in AI based on population neuronal responses, using multi-unit recordings in awake macaque monkeys.

In a recent study in awake marmoset monkeys (a New World primate species) [7], Bendor and Wang searched for single-unit responses to the missing fundamental in AI and surrounding non-primary areas and identified a restricted region anterolateral to AI containing pitch-selective neurons. Neurons were identified as pitch-selective if they responded to missing fundamental sounds (with harmonics outside the neuron’s excitatory frequency response area) and pure tones with a similar pitch. A typical pitch-selective neuron responded to an array of spectrally dissimilar sounds (harmonic complex tones, click trains, iterated ripple noise) when the pitch was near the neuron’s preferred fundamental frequency, which was found similar to the neuron’s CF determined by pure tones. Relative to AI, this newly identified region containing pitch-selective neurons in marmoset monkeys is in a similar location to the pitch-processing center found in humans [2-6,8-10] (Figure 2). In addition, Bendor and Wang [7] found that pitch-selective neurons preferred temporally regular sounds and were sensitive to pitch salience changes due to harmonic frequency and order, in agreement with the imaging studies by Patterson et al. [3] and Penagos et al. [4]. The microelectrode recording study by Bendor and Wang [7] also showed that the pitch-processing region found in marmosets contained non-pitch-selective neurons (spanning a similar range of CFs) that responded to the spectral frequency of the sound, rather than the fundamental frequency. Potentially, these two classes of neurons co-localized within the pitch-processing center respectively encode spectral and missing fundamental pitch percepts. For ambiguous pitch changes, in which the fundamental frequency and spectrum shift in opposite directions, as in the study by Schneider et al. [6], these two types of neurons would provide conflicting information. Unequal weighting towards one of these neuron types within the pitch-processing center (and/or between hemispheres) may be the cause of a subject’s perceptual bias of hearing pitch changes based on the fundamental or spectral frequency. No topography has yet been identified within the primate’s pitch-processing center in above cited studies [2-10], possibly due to spatial resolution constraints.

Many marmoset AI neurons located outside the pitch-processing region are tuned to the modulation frequency (repetition rate) of an amplitude- or frequency-modulated sound. However, in sharp contrast to pitch-selective neurons, these responses require that the acoustic signal’s spectral components be within the neuron’s frequency response area [57]. In awake marmoset AI, neural responses to stimulus repetition rates below ~40 Hz (near the lower limit of pitch [61,62]) are represented temporally by stimulus-synchronized discharges, while a monotonically tuned rate code is used by another population of neurons to represent higher repetition rates [58]. Neurons with harmonically related multi-peaked frequency response areas have previously been observed in marmoset AI [59], although they differ from pitch-selective neurons in that their primary frequency tuning and harmonic responses are most commonly outside the frequency range of pitch perception, and that they respond to each component of a complex harmonic tone. Together, the findings from marmoset auditory cortex suggest that pitch-processing is a specialized function by a subpopulation of neurons in a restricted region of non-primary auditory cortex, whereas processing of temporally modulated or harmonically rich signals is a general function by other auditory neurons located across the tonotopic axis.

Differences between observations from studies in primate and non-primate species

Several previous studies investigated how missing fundamental sounds are represented in the auditory cortex of non-primates species. Microelectrode recordings in gerbils suggested that AI neurons could respond to the periodicity of amplitude-modulated tones with the spectral components located outside neuron’s excitatory frequency response area [53], in contrast to a putative non-primary auditory cortical area encoding pitch in primates [2-10]. Schulze et el. [54] has also found a semi-circularly shaped map of best fundamental frequency in gerbil auditory cortex using optical imaging techniques. Differences in the results from these studies in primate and non-primate species could be due to an evolutionary divergence of pitch-processing strategies within auditory cortex, or may result from methodological differences. Combination tones at the fundamental frequency are produced within the cochlea by missing fundamental sounds, creating a potential ambiguity regarding whether a missing fundamental or combination tone is the source of the neuron’s evoked response [1]. The use of appropriate sound levels (to ensure that combination tones are below a neuron’s pure tone response threshold) [7], as well as the use of noise maskers [4,7] or a cancellation tone [55] are among the necessary control conditions to confirm that a true missing fundamental response is observed.

6. Further questions on pitch processing in auditory cortex

The discovery of a pitch-processing center in human (Figure 2A) and monkey (Figure 2B) auditory cortex is only the first step in understanding the physiological mechanisms of pitch perception. Several important questions remain unanswered. First, what is the source of inputs to pitch-selective neurons (corticocortical or thalamocortical)? The location of the pitch center appears to be overlapping low-frequency portions of field R (primary-like) and lateral belt areas AL and ML. This suggests that the pitch-selective neurons may receive inputs from both ventral and dorsal divisions of the auditory thalamus (MGB) that respond to narrowband and wideband sounds, respectively [26]. This could be a possible source of the response to pure tones and broadband missing fundamental sounds. Alternatively, given the extensive connectivity between AI, R, and neighboring belt areas surrounding the pitch-processing center, the pitch-selective neurons may extract the fundamental frequency using inputs from neighboring cortical areas (including AI). Second, do pitch-selective neurons use a spectral and/or temporal mechanism to extract the fundamental frequency of complex sounds? This is an issue that has been at the center of debate among auditory researchers for the last over 60 years. Computational models and auditory nerve data support the possibility of both a purely temporal mechanism and a hybrid mechanism using both spectral and temporal information [1,43-45]. In monkeys, AI neurons with temporal and spectral response properties potentially useful for these pitch models have been observed [56-59], but whether they provide an input to pitch-selective neurons is unknown.