Experience-dependent plasticity in pitch encoding: from brainstem to auditory cortex

Abstract

Linguistic and musical pitch provide an analytic window to evaluate how neural representations of important pitch attributes of a sound undergo transformation from early sensory to later cognitive stages of processing in the human brain, and how pitch-relevant experience shapes these representations. These pitch attributes are shaped differentially depending upon their functional relevance to a listener. Neural encoding of pitch-relevant information is shaped by the perceptual salience of domain-specific features at subcortical (auditory brainstem) as well as cortical stages of processing. The emergence of a functional ear asymmetry in the neural encoding of pitch-relevant information at a lower sensory processing level supports the view that local as well as feedforward and feedback mechanisms are involved in pitch-relevant processing. A theoretical framework for a neural network is proposed involving coordination between local, feedforward, and feedback components that can account for experience-induced enhancement of pitch representations at multiple levels of the auditory pathway.

Introduction

Hierarchical nature of pitch processing

Pitch is an important information-bearing perceptual component of language and music. As such, it provides an excellent window for studying experience-dependent effects on both cortical and brainstem components of a well-coordinated, hierarchical processing network. It is our view that a complete understanding of the neural organization of language and music can only be achieved by viewing these processes as a set of hierarchical computations. These computations are applied to representations at different stages of processing and are driven by experience-dependent sensitivity to linguistically/musically relevant features or dimensions. There is growing empirical evidence to support the notion that the neural representation of pitch relevant information at both brainstem and cortical levels of processing is influenced by one’s experience with language (or music) [1,2]. The neural mechanisms governing such experience-dependent plasticity are likely mediated by a coordinated interplay between ascending and descending neural pathways [3]. Physiological evidence from animal studies shows that signal representation in subcortical structures can be modulated by the efferent corticofugal system [4]. Evidence from evoked potentials suggests that enhanced subcortical activity in humans may also be mediated by the corticofugal system in those individuals with long-term experience in the language [5] or music domain [6], and even those with short-term language training [7]. A correlation between brainstem and cortical responses in musicians suggest that brainstem neural representations of pitch and cortical response timing are shaped in a coordinated manner through corticofugal modulation of subcortical afferent circuitry [8].

However, little is known about how language experience or musical training shapes pitch processing at each stage of this processing hierarchy. This review focuses primarily on electrophysiological studies of sensory stages of pitch processing in the brainstem and auditory cortex. Experience-dependent effects that are related to listeners’ domain of pitch expertise are argued to be compatible with a theoretical model that encapsulates local, feedback, and feedforward components in a neural network. Based on empirical evidence, we propose a theoretical framework to account for experience-dependent enhancement of pitch encoding in the brainstem and in the auditory cortex.

Features of pitch perception in tone languages

In tonal languages, voice fundamental frequency (f0) features (e.g., height, direction, slope) provide the dominant cue for high intelligibility of lexical tones [9]. We argue that f0 features, as opposed to categories, are essential for a better understanding of subcortical and early cortical stages of tone perception. In contrast to information relevant to morphology, syntax, semantics, and pragmatics, we assume that the shaping of pitch-related information relevant to tone perception begins well before the auditory signal reaches the cerebral cortex. Using multidimensional scaling analysis of paired dissimilarity ratings of pitch stimuli [10], we find that such features underlie a common perceptual space, and that they are differentially weighted depending upon the configuration of pitch patterns and their phonological relationship to one another in a listener’s native tone space [cf. 11]. Having established that behavioral weighting of pitch cues varies as a function of language experience, it seems reasonable to expect corresponding effects of such features at multiple stages of processing along the auditory pathway.

Response characteristics of brainstem evoked potentials related to pitch

The scalp recorded FFR reflects sustained phase locked neural activity in a population of neural elements within the rostral brainstem [12] that can be easily recorded between scalp electrodes placed at high forehead and the seventh cervical vertebra (C7). FFRs faithfully follow the temporal and spectral characteristics of the stimulus that can be extracted by frequency domain spectral and time domain autocorrelation analysis, respectively [5,13]. FFRs preserve spectrotemporal information related to the pitch of steady-state [14] and dynamic complex sounds in speech [5,13] and nonspeech stimuli [15–17]. Importantly, the encoding of pitch information preserved in the FFR is strongly correlated with perceptual measures [18,19]. These findings suggest that acoustic features relevant to pitch perception are well represented in the neural representations at the level of the brainstem.

Evidence from brainstem evoked potentials

We argue that the emergence of acoustic-phonetic features relevant to tone perception begins no later than 5–8 ms from the time the auditory signal enters the ear. FFRs preserve pitch information of lexical tones in both native and nonnative listeners. However, they are more robust in the former because of long-term exposure to their native pitch contours [5]. We therefore conclude that early sensory encoding of pitch information below the cerebral cortex is sensitive to language experience. No matter speech or nonspeech stimuli, pitch encoding is enhanced in the brainstem of Chinese listeners as compared to English [5,17]. Moreover, we find that Chinese exhibit more robust pitch representation precisely in those short portions of Mandarin tones that contain rapid changes in pitch movement [16].

With respect to pitch features, brainstem encoding of rising contours is found to be most important in discriminating Chinese from listeners of non-tone languages [20]. Pitch features important to tone perception are also found to be more resistant to degraded listening conditions in Mandarin listeners [21]. These advantages in encoding, however, diminish as the f0 rate of change within a pitch contour moves beyond what occurs in natural speech [22]. We conclude that experience-dependent neuroplasticity is primarily restricted to stimuli that are of functional relevance to the listener. Indeed, a strong correlation between neural and behavioral measures corroborates the view that pitch encoding at a subcortical, sensory level of processing plays an important role in shaping tone perception [19].

Language-dependent encoding mechanisms in the brainstem are especially sensitive to the curvilinear shape of pitch contours that occur in natural speech. Language-dependent effects do not occur no matter how close a rising or falling linear pitch pattern approximates a Mandarin tone [15,23]. A nonnative, curvilinear pitch pattern similarly fails to elicit a language-dependent effect [15]. Thus, experience-dependent effects in the brainstem are highly sensitive to specific features of pitch patterns in one’s native language.

In the music domain, the brainstem is sensitive to individuals’ long-term exposure to specific timbres. For example, pianists’ responses reflect the timbral characteristics of piano sounds with greater fidelity than those of non-pianists [24]. In the language domain, pitch encoding may similarly be influenced by the linguistic status of timbre. Indeed, f0 magnitudes were larger when co-occurring with a native as compared to nonnative vowel quality [25]. Such experience-dependent effects suggest that subcortical sensory encoding of pitch interacts with timbre in the human brainstem.

Experience-dependent pitch encoding is shaped by specific acoustic features within a particular domain

Long-term music training acts to enhance the magnitude and precision with which the brainstem responds to musical pitch [26,27]. These brain indices are correlated with an individual’s degree of training/experience [6] and, assuming the stimuli and task are behaviorally relevant to the listener, their perceptual performance [28,29]. Such findings illustrate the interaction between one’s pitch experience, brain physiology, and perception.

Our comparisons between the language and music domains reveal overall enhancements in brainstem FFRs elicited by either musical or linguistic pitch patterns in musicians and tone language speakers alike [26,28]. Thus, long-term pitch experience seems to improve the brain’s ability to encode pitch regardless of the domain of expertise. Yet, these sensory representations are subject to further “tuning” according to specific acoustic features encountered in a listener’s experience. Musicians, for example, show enhanced responses when pitch patterns intersect discrete notes along the musical scale; tone language speakers, on the other hand, during rapidly changing portions of tonal contours [26,30]. Such cue weighting is consistent with the relative importance of these perceptual dimensions in their respective domains. These findings collectively lead us to infer that both language and musical experience provide some mutual benefit to the neural extraction of pitch, but that specific features of the acoustic signal are highlighted in subcortical responses depending on their perceptual salience and function within a listener’s domain of expertise.

Language-dependent ear (a)symmetries related to perceptual features of pitch emerge in the brainstem

Hemispheric laterality for cortical speech processing is predictable based on low-level, spectral and temporal features, but at the same time may be modulated by high-level, abstract linguistic functions [31]. By virtue of fixed, structural asymmetries in the auditory pathway, monaural stimulation of either the left or right ear primarily drives the inferior colliculus and auditory cortex on the contralateral side. These structural asymmetries make it possible to ask whether ear asymmetries in pitch encoding at the level of the brainstem can similarly be modulated by functional properties of pitch.

Ear asymmetry at the level of the cochlea mimics hemispheric specialization for nonspeech signals [32]. Similarly, a functional ear asymmetry has been observed in brainstem responses depending upon the linguistic status of pitch in response to left and right monaural stimulation [33]. Within a portion of a Mandarin tone that exhibits a rapid change in pitch, a larger degree of FFR rightward ear asymmetry in pitch encoding was evoked as compared to its nonnative, mirror image counterpart. An asymmetry favoring left ear stimulation was evoked by the nonnative pitch contour only. No ear asymmetry was detected in response to the lexical tone. Its symmetrical response between ears may be a precursor of recruitment of higher processing structures between the left and right cerebral hemispheres. That processing of linguistic pitch involves both hemispheres is consistent with neuroimaging studies showing hemodynamic responses to pitch information varies between left and right perisylvian areas as a function of language experience [31]. This early shaping of the auditory signal at a preattentive, sensory level of processing is compatible with the idea that nascent representations of acoustic-phonetic features may emerge early along the auditory pathway.

Evidence from cortical evoked potentials

Early, automatic, preattentive pitch encoding is shaped specific acoustic features within a particular domain

The mismatch negativity (MMN), an automatic, preattentive cortical response, is an event-related potential with a frontal-central distribution about 150–300 ms after stimulus onset. It is well known that language experience influences the MMN response to speech sounds [34]. Of special interest to us is pitch, a multidimensional perceptual attribute that relies on several acoustic cues or features. The MMN may serve as an index of the relative saliency of underlying features of pitch that are differentially weighted by language or music experience.

A multidimensional scaling analysis of MMN responses to Mandarin tones revealed that native speakers of Mandarin Chinese, as compared to English, are more sensitive to pitch contour than pitch height [35,36]. But these effects are not necessarily language-specific. English-speaking musicians as well as native Chinese exhibited larger MMN responses relative to English non-musicians when presented with nonspeech homologues of Mandarin tones [37]. Thus, experience-dependent effects need not be driven solely by native linguistic categories. As with FFRs, these MMN data lead us to infer that some mutual benefit is gained in the neural extraction of pitch regardless of a listener’s domain of expertise.

Theoretical framework for experience-dependent neural plasticity

We propose an empirically-driven, hierarchical, theoretical framework to account for experience-dependent enhancement of pitch representation in the brainstem and in the early sensory level processing in the auditory cortex (Fig. 1). This framework includes (i) local pitch mechanisms in the inferior colliculus (IC) and auditory cortex (AC); (ii) a feedforward component (colliculo-thalamo-cortical projections); (iii) a cortico-collicular feedback component; and (iv) a cortico-thalamic feedback component [38].

Figure 1.

Anatomic model of the auditory pathway illustrating functional asymmetry in IC responses favoring RE stimulation, shown by the relatively thicker red lines compared to LE stimulation (blue lines). Local pitch mechanisms in the IC and AC (gray fill; (i)) are subject to experience-dependent plasticity. For linguistically-relevant sounds, this functional asymmetry is initially influenced by a cortico-collicularfeedback (iii) to promote reorganization in the IC for perceptual learning. A feedforward colliculo-thalamo-cortical component completes a core neural circuit (ii) that is known to be related to auditory learning and experience [38]. A cortico-thalamic feedback component (iv) further transforms output from the AC to the MGB. LE, left ear; RE, right ear; IC, inferior colliculus (brainstem); subscripts _L and _R, left and right, respectively; subscript _C, contralateral; MGB, medial geniculate body (thalamus); AC, (primary) auditory cortex; CF, corticofugal. At the level of the IC and AC, the solid black double arrow indicates that pitch information may be transmitted contralaterally in either direction. (cf. Fig. 7 [2], adapted with permission of Elsevier)

1. Pitch mechanisms in the IC and AC are subject to experience-dependent plasticity as a result of local reorganization via corticofugal modulation initial learning/training. Reorganization later permits pitch information to be extracted in a robust manner without on-line corticofugal influence. Feedforward, feedback, and local neuromodulatory inputs further transform the pitch representation at the cortical level. Both temporal- and rate-based local pitch encoding mechanisms are implicated in the auditory system.

2. Ipsilateral, dominant feedforward (colliculo-thalamo-cortical) projection from the IC to the AC is driven primarily by the contralateral ear, and transmits enhanced pitch-relevant information to the AC.

3. Ipsilateral, dominant feedback (cortico-collicular) projection (part of the corticofugal system) from the cerebral cortex to the IC facilitates experience-dependent reorganization in the brainstem.

4. Ipsilateral, dominant feedback (cortico-thalamic) projection transforms output from the AC to the medial geniculate body (MGB).

Behaviorally-relevant stimuli are thought to activate the local, feedforward and feedback loops in a coordinated manner at each stage of processing. Neuromodulatory inputs presumably induce large-scale, feature-specific plasticity to enhance sensory encoding. We propose that long-term experience sharpens specific pitch mechanisms in the IC and in the AC with particular sensitivity to linguistically-relevant dynamic segments. Once local mechanisms are reorganized via feedback/feedforward influence, local excitatory and inhibitory interactions that are known to play an important role in signal selection at each level of processing can presumably maintain the experience-dependent effect [39].

Conclusion

Long-term experience sharpens pitch mechanisms that are especially sensitive to linguistically-relevant dynamic segments in the rostral brainstem and the auditory cortex. Across domains, language experience and musical training provide some mutual benefit to the neural extraction of pitch. But it is also clear that neural responses are evoked differentially by specific pitch features depending on their perceptual saliency and function within a listener’s domain of expertise. This early shaping of the auditory signal at a preattentive, sensory level of processing is compatible with the idea that nascent representations of acoustic-phonetic features and hemispheric functional lateralization may emerge early along the auditory pathway. The observation of strong correlation between neural and behavioral measures supports the view that pitch encoding at early, sensory levels of processing may play an important role in the development of pitch percepts, i.e., higher-order representations that emerge at later stages in the processing hierarchy. An empirically based, hierarchical processing network involving coordination between local, feedforward, and feedback components can account for experience-induced enhancement of pitch representations at the level of the brainstem and auditory cortex. Future experiments should aim to evaluate the nature of the interaction and functional connectivity between stages of processing in the hierarchy by a comparative evaluation of neural representations at multiple stages concurrently.