Pitch Perception in a Developing Auditory Brain

Introduction

A harpist plucks the string of her harp and uses the enchanting sound of a musical pitch to captivate her audience. But how does the auditory system turn the vibration of a harp’s string into a pitch that we perceive as music? And more importantly, why is this pitch so important for hearing?

Indeed, whether it is to understand the words spoken around us, for the enjoyment of music, or perhaps to hear the voice of a server in a noisy restaurant, we rely on pitch to navigate the complex acoustic environments around us every day. In music, melodies are composed of sequences of pitch changes. In speech, pitch in vowels contributes to their identification, whereas pitch in voices conveys information regarding emotion, attitude, and talker identity. In tonal languages such as Cantonese, pitch variations in words change their meaning. Pitch is also used to help segregate simultaneous sounds in noisy environments.

How pitch is encoded by the ear and the brain has been a topic of scientific debate for many decades (Yost, 2015). However, although pitch has been extensively studied in mature auditory systems, less is known about the development of pitch perception in humans. Indeed, if pitch is critical to navigating our noisy world, can infants perceive pitch once hearing begins? What do babies hear when the string of a harp is plucked?

Neurophysiological studies conducted in human and nonhuman primates show the involvement of the auditory cortex, one part of the auditory brain, in pitch processing (Bendor and Wang, 2005), Thus, infant pitch perception is particularly interesting because the auditory brain undergoes a protracted and extended period of development. Although infants show responses to sound in the third trimester of gestation (Birnholz and Benacerraf, 1983), significant immaturity in the auditory cortex is observed at birth and throughout the first year of life. As a consequence of this immaturity, it is hypothesized that early responses to sound are supported primarily by subcortical processing, with a transition to more adultlike cortical mechanisms after the first six months of life (Eggermont and Moore, 2012).

This article considers the studies of infant pitch perception in the context of what they reveal about auditory brain development and how sound is perceived with an immature brain. Previous articles in Acoustics Today have covered infant speech development (Vick, 2018) and other aspects of auditory brain development (Kanold, 2022), whereas this article focuses on the perception of pitch in sounds like speech and music. I first begin with a definition of pitch and the two primary ways that pitch is encoded by the ear, which form the basis of the place and temporal models of pitch perception.

What Is Pitch?

One commonly adopted definition of pitch is that it is an attribute of sound that can be ordered on a scale from low to high (American National Standards Institute, 2013). The pitch of a pure tone corresponds to the frequency of its single component in the spectral domain (Figure 1B) and to the period of the waveform in the time domain (Figure 1A). The musical pitch produced by a harp’s string is an example of a complex pitch (Figure 1C). Musical notes or vowels in speech are examples of a harmonic complex, which consists of multiple frequency components that are all integer multiples of the fundamental frequency (f₀). Although a complex sound has many separate frequency components, the pitch of the sound is a unitary percept that corresponds to its f₀ (Figure 1D). However, it should be noted that pitch is defined perceptually as opposed to by a physical property of sound.

Figure 1.

A: time waveform of a pure tone where the period corresponds to its pitch (1/f). B: spectrum of a pure tone where the single frequency component corresponds to its pitch. C: time waveform of a complex tone where the period (T) corresponds to its pitch. D: spectrum of a complex tone where the fundamental frequency (f₀) corresponds to its pitch.

Place Versus Temporal Coding

There are two primary ways that the pitch is encoded in the ear that correspond with the spectrum and waveform of a sound. A rate-place code, which forms the basis of place models of pitch perception, represents the spectrum. When a sound enters the inner ear, it travels along the basilar membrane. Different frequencies in the sound maximally stimulate different regions along the basilar membrane, in effect doing a spectral analysis. Thus, the base of the basilar membrane is displaced by high-frequency components while the apex is displaced by low-frequency components. The frequency of a sound is encoded by the place of excitation on the basilar membrane, and the intensity of a sound is encoded by the firing rate of auditory nerve fibers at each place.

A temporal code, which forms the basis of temporal models of pitch perception, represents the time waveform. The temporal code relies on the tendency for auditory nerve fibers to fire at the same time during each cycle of vibration of the time waveform (phase locking). For most naturally occurring sounds, we would have access to both peripheral codes, but how these two codes are used by the ear and the brain to encode pitch is still a topic of active debate.

Early Infant Pitch Sensitivity

Despite significant immaturities in the auditory cortex, infants appear to respond to pitch in both speech and music. From the time they are born, infants show preference for their mother’s voice over the voice of an unfamiliar female (DeCasper and Spence, 1986). Infants can discriminate a change in the frequency of pure tones (Wormith et al., 1975) and pitch contours in syllables (Karzon and Nicholas, 1989). They also show preference for high-pitched versus low-pitched singing (Trainor and Zacharias, 1998) as well as infant-directed speech, which is characterized by a high f₀ and exaggerated pitch contours (Cooper and Aslin, 1990).

One of the challenging aspects of studying pitch perception is that oftentimes when the pitch in a sound changes, frequency and spectral changes also occur. However, many of these studies of early infant pitch perception did not differentiate between responses to frequency or spectral changes as opposed to pitch, and thus it is difficult to determine whether infants were indeed responding to pitch alone.

Observer-Based Psychophysical Procedure

Although it is difficult to determine what a preverbal infant hears, one approach that has a long history in both auditory and visual studies of infant perception is the observer-based psychophysical procedure (Werner, 1995). Sound discrimination can be measured with this method in infants from about 2 to 18 months of age. Sounds are presented to the infant through an insert ear tip or a speaker while the infant was sitting on a caregiver’s lap inside a sound-attenuated booth (Figure 2). An assistant stays inside the booth to keep the infant attentive and facing the midline. No one hears what the infant is listening to; the caregiver is listening to music while the assistant is listening to the experimenter’s instructions. The experimenter outside the booth starts each trial when the infant is facing the midline and attentive.

Figure 2.

Observer-Based Psychophysical Procedure. A six-month-old infant is siting in a caregiver’s lap listening to sounds via an insert ear tip (left). Infants are conditioned to respond by pairing a large pitch change with a video that turns on or mechanical animals that move (right).

Infants are first conditioned to respond by pairing a large pitch change that they can perceive with reinforcers being shown, such as mechanical toys that move or a video that turns on. Common infant responses to these reinforcers include head turns or eye darts toward the mechanical animals, depending on the infant’s age. The experimenter judges whether a change trial occurred based only on the infant’s behavior.

Once an infant-experimenter pair has passed the conditioning phase of training, they move on to testing. During the experimental test phase, trials when a pitch change is played or no-change trials when sounds of the same pitch continue occur with equal probability. Different studies typically establish different criteria as evidence for discrimination. For example, infants may be required to have correct responses on four out of five consecutive change trials and four out of five consecutive no-change trials to demonstrate that they are responding to the sound changes.

Evidence of Central Pitch Extraction

Although infants demonstrate the ability to discriminate pure tones, given the immaturity of the auditory brain, it might be expected that they cannot perceive the pitch of a complex tone. Most models of pitch perception, regardless of whether it is a place or temporal representation, require a mechanism for pitch extraction. Thus, using the place code, the immature auditory system would need to resolve the harmonics of a complex tone, integrate the spectral information, and then extract the f₀. By contrast, using the temporal code, infant auditory nerve fibers would need to generate an accurate representation of the time waveform via phase locking and then extract the period.

There is evidence from past studies that pitch extraction requires processing in the brain to integrate information across the spectrum of a sound as opposed to encoding by the ear alone (Houtsma and Goldstein, 1972). Moreover, physiological studies suggest that an area outside the primary auditory cortex may be where pitch is extracted (Bendor and Wang, 2005). Given that until four months of age, only the most superficial layer of the auditory cortex contains mature axons (Moore and Guan, 2001), it would not be surprising if infants do not perceive complex pitch if a central mechanism for pitch extraction is required.

One method commonly used to control for responses to the f₀, or the pitch of a sound, is a classic phenomenon called pitch of the missing fundamental. The pitch produced by a harmonic complex tone is so strong that even if energy at the f₀ is missing, a pitch corresponding to the fundamental is still perceived. This phenomenon demonstrates that the auditory system must be able to extract information about the f₀ from the higher harmonics alone (see demonstration at tinyurl.com/ymrsfs2j). Requiring listeners to discriminate missing fundamental complexes also ensures that they are responding based on pitch as opposed to frequency or spectral changes because the f₀ is, in fact, not present in the tone.

Infants have demonstrated the ability to categorize missing fundamental complexes by pitch (Clarkson and Clifton, 1985). However, due to the nonlinear response of the cochlea, it is possible that discrimination of the missing fundamental pitch was based on combination tones produced by the cochlea (Pressnitzer and Patterson, 2001). One stimulus control that can limit the listeners’ ability to rely on combination tones to perceive the pitch of the complexes is to use a noise band to mask the combination tones in the range of the missing fundamental. Indeed, infants as young as three months of age can categorize missing fundamental complexes by pitch, even in the presence of masking noise in the range of the missing fundamental (Lau and Werner, 2012). The results of these experiments suggest that central pitch extraction mechanisms are functional by this early age.

Unresolved Harmonics and the Temporal Code

By 6 months of age, infants can discriminate a change in frequency of a 4,000-Hz pure tone as well as adults do (Olsho et al., 1987). Discrimination ability at lower frequencies, however, continues to improve through childhood and does not reach adultlike levels until adolescence (Maxon and Hochberg, 1982). One explanation for the difference between high and low frequencies is that the place code is used for high frequencies and develops rapidly, whereas the temporal code is used for low frequencies and develops more slowly.

In fact, many aspects of the development of temporal processing remain unknown (see Cabrera and Lau, 2022, for a review). Levi et al. (1995) recorded the frequency following response and the envelope following response to amplitude-modulated pure tones using electroencephalography (EEG), suggesting that temporal coding is functional in one month olds (Levi et al., 1995). Infants have also demonstrated the ability to perform discriminations that rely on temporal processing. For example, six month olds can discriminate speech contrasts when presented with processed syllables containing envelope cues but degraded temporal fine structure (Bertoncini et al., 2011).

However, when the temporal code is required due to limited spectral information for pitch, seven month olds demonstrate greater difficulty with pitch discrimination. The basilar membrane in the cochlea is often modeled as an array of band-pass filters known as auditory filters. Low-number harmonics fall in separate auditory filters and are separated by the cochlea and are thus called resolved harmonics. However, as the bandwidth of the auditory filters increases with increasing frequency, high-number harmonics fall within the same auditory filter and are referred to as unresolved harmonics because they cannot be separated by the cochlea. Unresolved harmonics thus rely on the temporal code for pitch because place-based information for pitch is not available.

Clarkson and Rogers (1995) found that more infants were able to discriminate complex tones composed of resolved harmonics compared with the number of infants that were able to discriminate unresolved harmonics. Butler et al. (2013) presented infants with high-pass filtered iterated ripple noise (IRN), a stimulus that also relies on the temporal code for pitch. To create IRN, a sample of noise is created, a delay is imposed on the noise, and then it is added back to the original noise iteratively. It also produces a weak pitch that increases in salience with the number of iterations. Butler et al. (2013) found that performance varied widely across infants and concluded that discriminating the pitch of IRN is difficult for infants. It is possible that the infants’ difficulty perceiving the pitch of unresolved harmonics and IRN in these two studies could be an indication of immature temporal pitch extraction. However, in a companion EEG study, Butler and Trainor (2013) did record a mismatch negativity to IRN pitch changes in infants despite the variability in behavioral performance they observed.

Furthermore, Lau and Werner (2014) found that both three- and seven-month-old infants were able to discriminate the pitch of unresolved harmonic complexes. This finding is consistent with past results showing that a temporal representation of pitch is available in the auditory nerve (Cariani and Delgutte, 1996). Moreover, this finding suggests that this temporal representation is functional in human infants by three months of age. Nevertheless, the variability in performance observed across studies suggests that temporal pitch is less salient for infants as it is for adult listeners.

High-Fidelity Pitch Discrimination

A hallmark of pitch perception is that adults can discriminate pitch with fine precision, an ability thought to be important for speech and music perception, as well as listening in complex acoustic environments. Many adult listeners can discriminate a less than 1% change in the f₀, and for those who cannot, rapid improvements in pitch discrimination can be observed even after brief periods of training (Micheyl et al., 2006).

Lau et al. (2021) compared discrimination of different degrees of pitch change in three and seven month olds as well as in musician and nonmusician adults to investigate the influence of cortical maturation on the acuity of pitch perception. They conducted a missing fundamental pitch categorization task to test the infants’ and adults’ ability to detect a change in the f₀ within a sequence of complex tones, each containing a random selection of consecutive harmonics, leading to random changes in timbre from tone to tone. Timbre is the perceptual attribute of sound that differentiates instruments (e.g., violin and guitar) or voices that are producing the same pitch and loudness. Surprisingly, both three and seven month olds performed as well as musician adults on this task, discriminating smaller changes in f₀ than the nonmusician adults.

This finding may be because the random variations in timbre presented in the tones used in this study interfered with adult pitch discrimination but not that in infants. Stilp et al. (2010) have shown that adults are able to take advantage of statistical regularity in stimulus attributes to improve task performance and show rapid perceptual learning after a relatively brief exposure to covariations in sound features. As pitch and timbre often covary in natural sounds (Whalen and Levitt, 1995; Kitahara et al., 2005), this perceptual interference observed between pitch and timbre may be an efficient coding strategy. Thus, one explanation for the findings of Lau et al. (2021) is that infants have not learned the statistical covariation between pitch and timbre.

Nevertheless, the primary result of this study shows that accurate pitch and timbre discrimination can be achieved by infants as young as three months of age. Importantly, these findings imply that the fully mature auditory cortex is not required for accurate pitch discrimination, suggesting either that subcortical processing is sufficient for the f₀ and spectral coding or that the f₀ and spectral-peak discrimination is possible with an immature auditory cortex.

Melody Discrimination

One important aspect of pitch is that it that can be used to produce musical melodies. Although infants can discriminate complexes based on the f₀, they may not perceive melodic pitch in the same way as adult listeners. However, when presented with melodies composed of pure tones or two-component complexes, infants can detect changes to the melodies. Infants can detect a change in any position of a six-note melody (Trehub et al., 1985). The change in the melody can be in a different key or in the same key as the original melody (Trehub et al., 1984). Infants can even discriminate melodies composed of missing fundamental complexes (Lau et al., 2017). Interestingly, infants can detect changes in melodies belonging to scales from native Western scales as well as those from nonnative cultures (Lynch et al., 1990). Lynch et al. presented melodies based on native Western scales as well as nonnative Javanese pelog scales to American infants and adults and found that the infants were able to perceive the native and nonnative scales equally well, whereas the adults perceived the native scales better. These findings suggest that although infants are able to discriminate scales from different cultures, music perception is influenced by culturally specific listening experience by adulthood.

Musical Pitch Structure

Musical pitch structure is organized by several fundamental principles including consonance and dissonance, transpositional invariance, and tonal hierarchy. These principles appear to be perceived by infants from an early age and form the basis for other higher level pitch structures such as harmonic syntax, which does not appear until the childhood years (Trainor and Unrau, 2012).

Consonance and dissonance are considered fundamental organizing principles in musical pitch. Tones with f₀s in simple integer ratios such as the octave (2:1) or the perfect fifth (3:2) sound consonant, whereas tones with f₀s in complex integer ratios such as major seventh (15:8) tend to sound dissonant. Adult listeners from Western societies prefer consonance over dissonance, but there are divergent perspectives on the origin of this preference. One viewpoint is that consonance and dissonance arise from biological factors, whereas the opposing viewpoint is that it results from experiential factors (Weiss et al., 2020). For example, supporting the importance of experience, one study found that individuals from an Amazonian society that had minimal exposure to Western culture did not show a preference for consonance and dissonance (McDermott et al., 2016).

However, if infants perceive the distinction from a young age, that would be evidence to support that it is biological in origin. Indeed, Trainor and Heinmiller (1998) have shown that newborns and infants in the first few months of life look longer to consonant intervals than dissonant intervals, suggesting that they can discriminate between the two and that they prefer consonance over dissonance. Schellenberg and Trainor (1996) presented infants and adults with a sequence of consonant intervals and asked them to judge whether a test interval belonged to the sequence. Both infants and adults performed better on the task when the test interval was dissonant as opposed to consonant, suggesting that consonance has an influence on interval discrimination from a young age.

Consonance gives rise to the perception of tone chroma, also referred to as octave equivalence, the dimension of pitch that makes tones an octave apart sound similar. Demany and Armand (1984) found that by three months, infants show a perceptual equivalence for two pure tones forming an octave. They presented two melodies composed of pure tones to infants. The second melody consisted of tones shifted by an octave or another interval such as a seventh. Demany and Armand found that infants reacted less to transformed melodies consisting of pitch shifts by an octave than to shifts by larger or smaller intervals, suggesting that three month olds perceive tone chroma.

Finally, infants demonstrate the ability to perceive pitch contours in music. Relative pitch processing is critical to the perception of music because pitch relationships define melodies. Infants 5 to 10 months of age can recognize a familiar tune at any pitch after hearing it as little as 3 times. When tones are reordered or changed, infants perceive the melody as different, suggesting that infants can perceive relative pitch in melodies (Trehub et al., 1984).

Future Directions: What Is Happening in the Infant Brain?

The studies reviewed in this article suggest that for pitch, a fundamental aspect of sound, infants show accurate discrimination by about three months of age, despite significant immaturities in the auditory cortex at that age. Furthermore, infants’ use of pitch for music perception seems to parallel that in adults in many ways.

The results of these studies also identify the many aspects of infant pitch perception that remain unclear. Future studies should further investigate infants’ ability to use pitch while listening under noisy real-world conditions, and infants’ ability to learn the statistical covariation between pitch and timbre as well as the development of pitch perception in infants with impaired pitch perception, such as those who use cochlear implants. Finally, neurophysiological studies of pitch perception can further our understanding of how pitch and sound in general is processed in the developing brain.

Magnetoencephalography (MEG) is one neuroimaging approach that allows for the recording of robust, temporally precise neural signals with high signal-to-noise ratios in infants as young as two months of age (Figure 3). Source localization techniques in MEG allow for the differentiation of subcortical and cortical sources and, perhaps most important for studying brain development, advanced techniques for tracking and correcting head movements that allow MEG to be recorded from infants who are awake and listening. Figure 4 shows an example of infant neural responses to a pitch change, recorded with MEG; these signals can be studied in time and spectral domains and localized to their neural sources. With the rapid advances in neuroimaging technologies that can be used with infants, the hope is that we can further our understanding of this fundamental question in human auditory development: How, despite the protracted and extended developmental period of the auditory brain, infants demonstrate sophisticated sound-processing abilities from the time they are born.

Figure 3.

Infant magnetoencephalography (MEG). Eight-month-old participant is being prepared for MEG recording including the placement of head position indicator (HPI) coils on a cloth cap to record head movement, electrocardiogram electrodes to record heartbeat, and digitization of HPI coils, cardinal landmarks, and 200 additional points on the head (left). A foam head bumper is placed on the infant to limit movement in the MEG dewar (center). Infant is awake and listening to sounds during MEG recording with a parent sitting in the chair next to them and an assistant in the booth showing them toys to keep them still and attentive (right). Photo courtesy of the University of Washington Institute for Learning & Brain Sciences (I-LABS), Seattle.

Figure 4.

Grand averaged (N_ave) event-related fields (magnetometers) from 13 six-month-old infants with topographic maps shown at the peak of onset (0.208 s) and acoustic change response (1.206 s) to a pitch change from 160 to 200 Hz. Bottom black line, global field power (GFP). Color bar units are femtotesla.

Biography

Bonnie K. Lau is a research assistant professor at the University of Washington (Seattle). Her research investigates auditory brain and perceptual development and how that relates to language acquisition. She combines neurophysiological measures including electroencephalography and magnetoencephalography, psychophysics, and clinical assessment methods in her research.