Neuronal oscillations as a mechanistic substrate of auditory temporal prediction

Abstract

Neuronal oscillations are comprised of rhythmic fluctuations of excitability that are synchronized in ensembles of neurons and thus function as temporal filters that dynamically organize sensory processing. When perception relies on anticipatory mechanisms, ongoing oscillations also provide a neurophysiological substrate for temporal prediction. In this article we review evidence for this account with a focus on auditory perception. We argue that such “oscillatory temporal predictions” can selectively amplify neuronal sensitivity to inputs that occur in a predicted, task-relevant rhythm and optimize temporal selection. We elaborate this argument for a prototypic example, speech processing, where information is present at multiple time scales, with delta, theta, and low-gamma oscillations being specifically and simultaneously engaged, enabling multiplexing. We then consider the origin of temporal predictions, specifically the idea that the motor system is involved in the generation of such prior information. Finally, we place temporal predictions in the general context of internal models, discussing how they interact with feature-based or spatial predictions. We propose that complementary predictions interact synergistically according to a dominance hierarchy, shaping perception in the form of a multidimensional filter mechanism.

Neuronal oscillations underpin temporal predictions

Seminal theories in neuroscience consider sensory processing as an inference problem: given the noisy sensory and neural signals, the brain estimates which stimuli caused these observations.¹ Accordingly, neuronal representations start to be viewed as continuous top-down prediction signals, that is, internal models of causal dynamics in the world shaped by one’s perceptual, behavioral, and emotional experiences.² Such theories have two far-reaching corollaries: (1) predictions are reflected in the brain’s intrinsic activity;³ and (2) sensory-evoked neuronal activity corresponds to the perturbation of ongoing circuit dynamics by input signals, initiating the generation of an updated representation of the world.^4–7

However, it remains unclear how predictions are represented in the brain. One problem resides in the fact that an event can be anticipated with regard to its features (content), location (space), and/or moment (time) of occurrence.^8–10 Focusing on this latter dimension, dynamic attending theory suggests that during perception attention is modulated dynamically to optimize sensory processing at expected points in time.^11–13 Critically, this framework capitalizes on the fact that many stimuli and actions are rhythmically organized (e.g., speech, music, walking). By extracting such temporal regularities, the brain is able to predict the occurrence of subsequent events of interest and to optimize their processing.

Behavioral studies have shown that temporal expectations indeed optimize perception by dynamically modulating the gain of sensory information.^14–16 The signal-to-noise ratio of the predicted sensory signal is increased, which leads to improved detection performance. This effect is associated with a reduction of the large-scale sensory evoked neuronal response, as observed in both visual^17,18 and auditory modalities.^19–21 At the neurophysiological level, low-frequency neuronal oscillations are the hypothesized substrate of temporal predictions.⁵ Neuroelectric oscillations correspond to temporal fluctuations of the excitability state of neuronal ensembles²² and make it possible to selectively amplify neuronal input responses occurring at predicted moments.^23,24 This mechanism results in a selective neuronal tracking of attended sensory streams, as exemplified by the cocktail party effect, where listeners have to “tune in” to one conversation in a noisy scene.²⁵

Speech processing at multiple time scales

Speech, together with music, is arguably the most interesting stimulus to study the role of neuronal oscillations in perception. Speech perception arises from the dynamic sampling of acoustic information at multiple time scales simultaneously, referred to as multiplexing. Indeed, although syllabic modulations (1–7 Hz) are critical to speech comprehension,²⁶ acoustic modulations above 12 Hz also contribute to speech identification, as they are associated with some phonetic feature dimensions.^27,28 In recent years neuronal entrainment of cortical oscillatory dynamics to the stimulus acoustic envelope has been described, suggesting a prominent role of auditory delta (1–4 Hz), theta (4–8 Hz), and low-gamma (25–45 Hz) frequencies in speech parsing.²⁹ In addition to syllabic-based sampling at theta rate, fine-grained sampling (~ 30 Hz) preserves the detailed temporal structure of sounds necessary for phonemic contrasts extraction and articulatory routines execution;³⁰ slower sampling (<4 Hz) is suggested to underline more steady cues such as formants at the basis of vowel recognition but also vocal analysis (e.g., speaker identification, emotional prosody).

Most studies of speech perception emphasize that auditory responses phase-lock to speech syllabic modulations in the theta range.^31,32 However, in left auditory regions neuronal entrainment is also observed at low-gamma rate,^33,34 which indicates that auditory information is sampled at multiple time scales simultaneously.³⁵ Such neuronal tracking is dynamic, resulting from the close interplay between stimulus-driven responses and the top-down influence of internal predictions. This latter assumption is exemplified by two complementary sets of evidence. First, neuronal entrainment strengthens with learning and comprehension.^34,36–38 Second, in auditory cortex the modulation spectrum of spontaneous oscillatory activity predicts the interhemispheric differences observed during speech processing, where low-gamma activity dominates in the left hemisphere.^33,39,40 This strongly highlights the role of internal models as crucial in shaping the sensory neuronal response.⁴¹

Finally, in addition to theta and low-gamma oscillations, speech processing is also associated with neuronal entrainment in the delta range. Delta activity is likely to be involved in multiple facets of auditory processing. On one hand a right-lateralized neuronal entrainment is observed at delta rate during speech processing, co-occurring with theta entrainment but carrying complementary information.^34,42 This result is compatible with a role of delta activity in paralinguistic processing, capturing prosodic information occurring at a phrasal rate.⁴³ On the other hand, some reports indicate a left-lateralized involvement of delta oscillations during auditory processing, hypothesized to reflect motor-driven top-down modulations.^40,44

Motor origin of temporal predictions

The motor system sits at the top of the cortical hierarchy involved in the perception–action cycle.⁴⁵ Capitalizing on predictive coding or Bayesian inference frameworks, recent theoretical attempts suggest that the motor system may be conceptualized as a purely predictive system, generating top-down predictions that shape perception.^5,46–48 In particular, these models focus on the temporal dimension, emphasizing the temporal essence of the motor system. More than a century ago, rhythm and movement were hypothesized to be intrinsically linked.⁴⁹ Since then the motor system has been shown to be a key structure in the precise estimation of short durations,^50,51 and neuroimaging studies on beat and rhythm processing invariably report the involvement of the motor system.^46,52–54 Thus, the motor system might be involved in the generation of putative beats and engaged in the analysis of temporal sequences, neuronal motor activity corresponding to a temporal prediction signal that sharpens sensory representations.

Interestingly, neuroimaging studies of temporal attention have shown that orienting attention in time is specifically associated with activation of left parietal and premotor cortex.⁵⁵ Moreover, it was recently observed that delta oscillations in primary motor cortex covary with the occurrence of temporally predictable task-relevant visual cues.⁵⁶ A similar result was also observed during an auditory task, the accurate estimation of short duration depending on delta oscillatory fluctuations.⁴⁴ This pattern of findings suggests that the (left lateralized) motor system may drive temporal expectations through slow oscillatory dynamics.

Active sensing in the auditory domain

The involvement of sensorimotor loops in perception is often referred to as active sensing.^47,57 This framework proposes that perception is shaped by the motor system in two distinct ways. First, the motor system directs sensing organs (e.g., finger squeezes, ocular saccades, olfactory sniffs) toward relevant stimuli, thereby structuring the content of bottom-up sensory information inflow. Second, it modulates the processing of sensory information via top-down internal corollary discharge signals, that is, copies of movement commands sent to associated sensory structures.⁵⁸ This latter mechanism corresponds to an active sampling of sensory information by marking the timing (and location) of relevant sensory events according to current motor plans. This predictively modulates sensory processing according to the temporal (and spatial) patterns of motor activity patterns, thus providing “when” (and “where”) predictions at a minimum.⁵⁹

Active sensing was never described in the auditory domain because of its relative bottom-up disconnection from the motor system. Indeed, whereas bottom-up and top-down motor influences are contingent in most sensory modalities, audition is the exception because we cannot selectively move our ears. However, sensitivity to temporal information is at its best in the auditory modality, and music exemplifies the strong relationship existing between rhythm and auditory and motor systems.^53,60 This suggests that (1) the motor system could exert a specific top-down predictive influence on auditory processing, and (2) top-down motor influences, often conflated with bottom-up ones (the directing of sensing organs), could play a distinct and fundamental role.

Motor contributions to the temporal precision of auditory attention

We recently developed a mechanistic behavioral account of auditory active sensing.⁶¹ This work was based on four theoretically driven parameters. First, we focused on the auditory modality because of its relative bottom-up disconnection from the motor system. Second, we measured the ability of human participants to extract relevant auditory information interleaved with distracting information, in a cocktail party–like fashion, and embedded in rhythmic streams. Thus, performance critically depended on the implementation of temporal expectation. Third, we used a perceptual decision-making task to study the dynamics of evidence accumulation and its modulation by temporal expectation. This led us to the following experiment. Participants were asked to categorize sequences of eight pure tones as higher or lower pitched, on average, than a reference frequency. In order to drive rhythmic fluctuations in attention, the tones (targets) were delivered in phase with a reference beat, presented at the beginning of each trial, and in antiphase with irrelevant yet physically indistinguishable tones (distractors). Fourth, participants were instructed either to use their motor system overtly, by pressing in rhythm with the reference beat, or to keep the rhythm covertly, in both cases as a way to optimize the allocation of temporal attention (predictions) in a purely top-down fashion.

Our findings show that producing a rhythmic movement engages a top-down modulation that sharpens the temporal selection of auditory information. It improves the segregation between relevant and distracting information, facilitating perception of relevant items and suppressing perception of irrelevant ones. We additionally observed that the impact of overt motor tracking on perception parametrically scales with the temporal predictability of the auditory sequence. Finally, motor contributions depend on the temporal alignment between motor and attention fluctuations. This indicates that this rhythmic top-down process requires an exquisite temporal alignment among motor, attention, and auditory systems during the sampling of sensory information. Altogether, our results suggest that motor-driven temporal predictions optimize perception, and a most likely substrate for this effect is a slow oscillation timed by rhythmic motor activity.

Previous behavioral experiments on the top-down role of the motor system in auditory perception showed that self-initiated head movements can reset the perceptual organization of an auditory scene⁶² or influence the way an auditory rhythmic pattern is perceived.⁶³ Here we went several important steps further by firmly isolating top-down motor influences on auditory stimulus processing and by showing that the engagement of the motor system improves (1) the precision of temporal attention and, thus, (2) the quality of sensory selection.⁶¹ Our work confirms the operation of active sensing in the auditory domain, which emphasizes the prominent role of motor activity in sensory processing through its close interaction with attention.

Hierarchical organization of predictive filters in auditory perception

A first necessary step toward a Cartesian approach⁶⁴ is to study temporal expectations independently of other dimensions (such as location or feature). However, the drawback in most paradigms is that these attributes of noninterest are kept constant across trials and so are highly predictable. These strong (e.g., spatial or feature-based) expectations might in turn impact perception and bias results or limit interpretations.⁶⁵ Indeed, in primary auditory cortex the combination of temporal and spectral expectations results in both the amplification and sharpening of neuronal responses, in essence acting as a spectrotemporal filter mechanism.²³ Such a combination is not restricted to what and when dimensions in audition but can also integrate spatial⁶⁶ and intensity⁶⁷ information at higher levels. These data suggest that predictions constantly fuse into a dynamically evolving model of upcoming events, which enhances the representation of predicted stimuli.

However, predictions might not combine linearly. For example, although temporal expectations alone influence behavior during a visual task,⁶⁸ their impact is increased by concurrent spatial expectations and is erased if spatial expectations are invalid.^68,69 This indicates that the impact of temporal expectations on perception is optimized by the co-occurrence of spatial expectations. Because the visual cortex is retinotopically (i.e., spatially) organized, temporal predictions might not be able to efficiently modulate the first cortical stages of visual processing unless they are part of a spatiotemporal predictive filter that taps onto retinotopically specific regions. This suggests that the influence expectations exert on perception might rely on a hierarchy of predictive filters that trails the structure of the concerned sensory system. Spatial predictions would be necessary in the retinotopic visual system, whereas spectral predictions would be key in the tonotopic auditory system.

Overall, current research suggests that perception is a rhythmic process⁷⁰ and features neuronal oscillations and the motor system as key operators in sensory processing.⁴⁷ Recent evidence in developmental neurosciences accordingly pinpoints temporal sampling deficits as a potential cause of developmental dyslexia.^71,72 It moreover suggests that music, which is exquisitely rhythmic and linked to motor activity,⁵³ might be used as a rehabilitation strategy: by boosting temporal prediction mechanisms, musical rhythmic training could ultimately benefit language processing and language acquisition.⁷³