Separating Uncertainty from Surprise in Auditory Processing with Neurocomputational Models: Implications for Music Perception

Advances in neuroscience have led to the emergence of two complementary theories of neural information processing. First, the “Bayesian brain hypothesis” proposes that the brain actively predicts and represents incoming sensory information as probabilities that are updated in a near-optimal Bayesian manner (Friston, 2010; Aitchison and Lengyel, 2017). Second, “predictive coding” accounts posit that these predictions are generated from higher-level cortical regions and passed down toward early sensory regions. The difference between predicted and actual stimulus information results in a prediction error signal that is propagated bottom-up along the cortical hierarchy to update future predictions (Spratling, 2017). Importantly, predictive coding assumes that top-down predictions relate not only to the content of the sensory input, but also to the precision of such predictions (Heilbron and Chait, 2018). Predictions about precision modulate the gain of prediction errors so that surprises are attenuated in high-uncertainty environments.

Predictive processing in the auditory domain has traditionally been investigated using oddball paradigms. Typically, subjects are presented with a continuous stream of repeated sounds (“standards”). With a given probability, some sounds (“deviants”) are randomly modified by changing the pitch, timbre, timing, or volume. In neural electrophysiological activity recorded using electroencephalography (EEG) or magnetoencephalography (MEG), deviants evoke an increased negativity at ∼100–250 ms after stimulus onset compared with standards (Heilbron and Chait, 2018). This is known as a mismatch negativity (MMN) and is thought to largely reflect precision-weighted prediction errors (Lieder et al., 2013), which are driven by neural adaptation in lower auditory regions and short-term synaptic changes between frontal and temporal cortices (Garrido et al., 2009).

Because the MMN is a function of both prediction error and precision, separating the neural mechanisms of precision estimation from prediction errors remains challenging. Lecaignard et al. (2022) sought to address this via computational modeling of neural responses evoked by auditory stimulation, recorded using simultaneous EEG-MEG. To identify mechanisms underlying precision, the predictability of deviants was manipulated in a passive auditory oddball paradigm. In the low-uncertainty condition, a block consisted of two standards followed by a deviant tone, then three standards followed by a deviant tone, and so forth until eight standards were followed by a deviant. In the high-uncertainty condition, the number of standards preceding a deviant was randomly shuffled in each block. This manipulation ensured that the deviant probability remained constant, despite the two conditions affording different levels of predictability.

On the computational level, the authors used Bayesian model comparison to test the relative extent to which three model classes reconstructed the observed neural responses. Baseline models assume every tone elicits the same neural response. Change detector models compute the difference between incoming and preceding tones. Learning models predict deviant probability, recursively update predictions in a Bayesian manner, and introduce a learning constant, τ, to govern the weight previous trials have on current predictions. Lecaignard et al. (2022) found that learning models were best able to reconstruct evoked responses at the MMN time window. In particular, although the MMN did not vary for the two conditions given the same values of τ, the estimated value of τ was significantly larger when uncertainty was low.

On the neurophysiological level, dynamic causal modeling (DCM) was used to examine effective connectivity during the perception of standards and deviants in the two conditions. The following two types of connections are distinguished in DCM: extrinsic connections between cortical regions; and intrinsic connections within a cortical region. Extrinsic connections originate from excitatory pyramidal cells in infragranular layers of a source region and drive activity in a target region, while intrinsic connections modulate synaptic gain of infragranular pyramidal cells via local interneurons connecting infragranular, granular, and supragranular layers (Kiebel et al., 2008).

DCM allows one to infer connectivity dynamics by selecting the network architecture that most plausibly generates the observed neural activity and examining the coupling strength of extrinsic and intrinsic connections. In line with previous work (Auksztulewicz et al., 2017; Quiroga-Martinez et al., 2021), the most plausible network found by Lecaignard et al. (2022) comprised bidirectional connections between Heschl's gyrus (primary auditory cortex) and the planum polare, the planum polare and the inferior frontal gyrus, and the inferior frontal gyrus and the superior frontal area. Notably, they found significantly enhanced self-inhibition in all nodes of the network in the high-uncertainty condition. Furthermore, echoing previous findings, deviants elicited increased top-down and bottom-up connectivity, as well as reduced self-inhibition, compared with standards.

Results in the study by Lecaignard et al. (2022) demonstrate the influence of uncertainty on predictive mechanisms during auditory perception at the cognitive and neural levels. Cognitively, that learning models best explain neural activity in the MMN time window confirms previous work (Ostwald et al., 2012; Lieder et al., 2013) and supports the MMN as a neural signature reflecting the ability of the brain to infer and learn statistically. Importantly, their key contribution is showing that the learning constant τ in learning models is larger when uncertainty is low. This demonstrates that information from previous stimuli—regardless of whether they were standards or deviants—is upweighted in a more predictable context. This implies that the brain flexibly modulates its temporal integration window—depending on the uncertainty of the context—to most efficiently exploit information in its environment to generate future predictions. However, because τ is estimated from the data and not explained by the model, the observed difference indicates only a consequence of and not evidence for predictions of precision, as hypothesized in predictive coding. The precise mechanisms describing the computation of precision remains open.

Nevertheless, Lecaignard et al. (2022) show that, at the neural level, precision weighting is manifested as changes in local synaptic gain. In particular, self-inhibition is increased when the context is not conducive to making reliable predictions. Integrating this result with previous findings on the MMN (for review, see Garrido et al., 2009; Heilbron and Chait, 2018), a plausible mechanism could be as follows: upon hearing an auditory deviant, infragranular pyramidal cells along the cortical hierarchy between Heschl's gyrus and frontal regions are disinhibited by local interneurons, thereby enhancing the bottom-up propagation of predictions errors and top-down predictions. However, the extent of disinhibition is modulated by the uncertainty of the context, with increased disinhibition in more predictable contexts. This could facilitate statistical learning via changes in frontal–temporal coupling strength, which could result in an altered temporal integration time window as estimated by the learning parameter τ.

Separating mechanisms behind the generation of surprise from the uncertainty of the context is not only relevant to low-level auditory perception, but also to understanding how complex stimuli such as music can evoke pleasure in the listener (Koelsch et al., 2019). Recent work has shown that chords, melodies, and rhythms are most pleasing when they strike an optimal balance between the uncertainty of a prediction and the surprise from what is actually heard (Cheung et al., 2019; Gold et al., 2019; Matthews et al., 2020). Furthermore, current evidence reveals that the interaction between uncertainty and surprise in music engages not only the auditory cortex, but also regions in the mesolimbic reward network including the nucleus accumbens, amygdala, and hippocampus (Cheung et al., 2019; Matthews et al., 2020). Although the dynamic coupling between these brain regions during music listening have already been established via correlational approaches (Salimpoor et al., 2013; Shany et al., 2019), future studies could use directed approaches such as DCM to examine the flow of information within each region and within the network. Following results from the study by Lecaignard et al. (2022), we would expect auditory and reward-related regions to show increased intrinsic connectivity because of enhanced local inhibition during uncertain musical contexts.

There are nevertheless two key distinctions between music and auditory deviants. First, music structure is thought to be syntactically organized, with hierarchical relations spanning multiple timescales and beyond the local context (Rohrmeier and Pearce, 2018). Second, these relations are assumed to be established not only on hearing the stimulus, but also in long-term memory after extended exposure to a musical style (Pearce, 2018). Consequently, surprises in musical syntax elicit an early right anterior negativity, which is thought to be distinct from the MMN (Koelsch, 2009; Koelsch et al., 2019). However, the extent to which predictive processes integrating short-term acoustic information (e.g., timbre and pitch) and long-term abstract relationships (e.g., musical syntax) overlap remains unclear (Koelsch, 2009). In particular, neurocomputational mechanisms distinguishing the processing of uncertainty and surprise in musical syntax remains largely unexplored. A model-comparison approach similar to that of Lecaignard et al. (2022) may prove helpful in this investigation.

In summary, Lecaignard et al. (2022) provide a holistic account of how uncertainty modulates surprises in auditory perception at the neural and cognitive levels. Using neurophysiological and computational modeling of the MMN, they show that uncertainty influences local inhibitory synaptic dynamics and integration of past information when forming probabilistic predictions. These findings not only further support Bayesian inference and predictive coding as key neural processing mechanisms, but also provide helpful insights toward understanding the role of predictive processing in our appreciation of complex auditory stimuli that we call “music.”