Predictive coding: a more cognitive process than we thought?

Abstract

In predictive coding (PC), higher-order brain areas generate predictions that are sent to lower-order sensory areas. Top-down predictions are compared to bottom-up sensory data, and mismatches evoke prediction errors. In PC, the prediction errors are encoded in layer 2/3 pyramidal neurons of sensory cortex that feed-forward. The PC model has been tested with multiple recording modalities using the global-local oddball paradigm. Consistent with PC, neuroimaging studies reported prediction error responses in sensory and higher-order areas. But recent studies of neuronal spiking suggest that genuine prediction errors emerge in prefrontal cortex. This implies that predictive processing is a more cognitive than sensory-based mechanism– an observation that challenges PC and better aligns with a framework we call predictive routing.

Predictive coding models of brain function

Brains have evolved to create mental models that explain the regularities in the environment [1], [2]. According to predictive coding (PC; see. Glossary), these mental models issue internal predictions to drive sensation, thought, and action to reduce energetic costs while maintaining an efficient neuronal code [3], [4], [5]. This is achieved via implementing an approximation of Bayesian inference which involves combining predictions (priors) with sensory inputs (likelihood) to create a posterior probability (posterior). The posterior is an optimal combination of the prior beliefs with current sensory evidence [6], [7]. It serves as the brain’s best guess of the state of the world and body [8]. When sensory inputs arrive that do not accord with this internal model, a prediction error is generated [5]. Prediction errors are modulated according to their gain [4], [9], [10].

“Classical” predictive coding models [1], [4], [5], [11] propose that predictions are generated in higher-order areas of the brain and feed back down the hierarchy to lower-order areas, where they are compared with sensory inputs. Prediction errors travel in the opposite direction. They feed forward up the hierarchy to update internal models to make better predictions. Feedback predictions are thought to be subtractive. Predictable stimuli are uninformative, so they should drive less overall neuronal activity to save energy. In contrast, surprising/unpredictable stimuli enhance neuronal activity [12], [13], [14].

This classical predictive coding model has now been tested using both neurophysiological studies in animals and non-invasive studies as subjects were presented with the local-global oddball paradigm (Fig. 1A), which is designed to evoke prediction errors at two distinct stages of hierarchical processing. Here, we will focus mostly on studies in non-human primates and humans using this paradigm, given that primates have a deeper and more distinct cortical hierarchy compared to rodents [15], and that the concept of hierarchy plays a central role in predictive coding.

Figure 1: Local and global oddballs and recent evidence of their encoding in neuronal spiking activity.

A) Schematic of the local-global oddball paradigm. Neuronal responses to local oddballs can either be caused by true prediction error signals or by a release from sensory adaptation. Global oddballs are caused by habituating to a stimulus sequence. For example, in the sequence AAAB, AAAB, AAAB, the fourth stimulus is predicted to change. The infrequent stimulus (AAAA) is a global oddball, because the fourth tone is unpredicted despite being a repetition. The time scale of a trial is typically one to a few seconds. The time scale of a sequence is typically tens of seconds. For our review of the literature, we consider studies that have used stimulus repetition-based predictions as local oddballs vs. studies that have used more complex designs where prediction and repetition are dissociated as global oddballs. We note that local oddballs that are also global oddballs (AAAB, AAAB, AAAA) also dissociate prediction from adaptation, but only if there is sufficient time between the oddballs to rule out adaptation, see [64]. B) area legend on a macaque cortex flat map C) A recent study showing the lack of global oddball responses in population spiking (awake Tpt and FEF in Ci and Ciii respectively, and anesthetized Tpt and FEF in Cii and Civ respectively [32]. D) Spiking activity in V1 and V4 [49] is identical to a stimulus when it occurs in the expected position (green traces) as well as when it is unexpectedly early (purple traces) or late (yellow traces) in the sequence.

Studies recording data using electroencephalography (EEG), magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), and intracranial local field potentials (LFP) largely supported classic predictive coding by showing widespread cortical activation during stimuli that should evoke prediction errors. However, recent evidence from studies measuring neuronal spiking in non-human primates has shown sparser and higher-order origins for prediction errors, inconsistent with the proposal that feedforward processing represents prediction error. These recent results challenge the classic predictive coding models. Our aim here is to review these studies and suggest an alternative framework for how predictions are implemented, predictive routing (PR).

The Local-Global Oddball Paradigm

Initial evidence for prediction error coding came from studies using the Mismatch Negativity response (MMN) [16], [17], [18]. These paradigms rely on stimulus repetition to create predictions, and rare deviant stimuli elicit increased neural responses, consistent with a prediction error response. However, the neuronal activity caused by release from adaptation to an unrepeated stimulus vs. genuine prediction error is conflated in this experimental design. Neuronal adaptation, a low-level mechanism whereby stimulus repetition causes decreased responses due to neuronal fatigue [19], can explain these responses without invoking hierarchical, Bayesian brain processes [20], [21], [22]. To dissociate expectation violation from adaptation, paradigms such as deviance detection [23], [24], [25] and the local-global oddball paradigm (Fig. 1A, [26]) were developed.

The local-global oddball paradigm uses local oddballs to violate a locally repeated context (e.g., in sequence AAAB, “B” is the oddball and violates the prediction that A will repeat). Global oddballs violate a pre-habituated pattern [16], [27], [28], [29], [30]. Global oddballs involve changes in a stimulus within a sequence (in the sequence, AAAB, AAAB, AAAA, the final “A” is the global oddball). This dissociates a violation of expectation from a release of adaptation.

Predictive coding posits that brains construct complex models that are compared to sensory inputs during inference, with mismatches driving prediction error computations. This implies neurons should actively respond to stimuli, not merely passively adapt. Local oddballs and repetition-based oddballs, which persist under anesthesia [31], [32] when top-down connections are functionally inactive (Fig. 1C, [32]), provide limited evidence for predictive coding [33]. For predictive coding to explain cortical responses ubiquitously, more complex types of prediction errors (e.g., global oddballs) should evoke responses [1]. Indeed, neuronal model implementations of predictive coding hypothesize that a rare repetition in an environment of frequent alternations (i.e. global oddballs) will elicit prediction error signals at each layer of the network architecture [34].

While both local and global oddballs trigger prediction errors, they may do so at distinct hierarchical levels. Basic errors (local oddballs) may be more strongly encoded at earlier levels of the hierarchy, while complex errors (global oddballs) might be more prominent at later levels. However, both types of oddballs should modulate activity in sensory cortical areas for the claim that feedback responses instigate predictive models and that feedforward responses primarily signal prediction error to hold true. If sensory cortex primarily shows passive adaptation (e.g., responds only to local oddballs where prediction error and release from adaptation are conflated), this would weaken the argument for predictive coding as an active, canonical, cortex-wide mechanism.

fMRI, M/EEG, and LFP studies indicate widespread local and global oddball coding

fMRI [35], [36], M/EEG [16], [17], [18], [37], [38], and LFP studies demonstrated widespread representations of local [13], [17], [26], [39], [40] (Fig. 2A, C, E, Key Figure) and global oddballs (Fig. 2B, D, F), with some studies indicating that even primary and secondary sensory areas (highlighted in green and purple outlines in Fig. 2) encode these oddball types [26], [31], [39], [41], [42], [43], [44]. These studies aligned with the classic predictive coding proposal that sensory cortex compares top-down predictions with bottom-up sensory inputs and issues prediction errors due to their mismatch. They suggested a canonical computation for prediction error [11] and inspired more mechanistic studies of neuronal spiking to investigate the neuronal code associated with prediction error computations.

Figure 2: Current state of the local and global oddball literature.

The numbers related in the figure refer to Broadman Areas (BAs). The shaded (red vs. blue) regions represent activation (or lack of) to local and global. Negative (blue shading) results are only shown for spiking studies. Blue/green borders represented higher vs. lower order auditory/visual cortices. Flat maps are from macaques [106], [107] and human results are depicted in the homologous areas. A) Local oddball fMRI results [26], [41], [42]. B) Global oddball fMRI results [26], [41], [42], [108]. C) Local oddball M/EEG results [26], [44], [109], [110]. D) Global oddball M/EEG results [26], [44], [109], [110]. E) Local oddball LFP (note that we consider here also studies employing electrocorticography) results [12], [26], [31], [32], [39], [42], [111]. F) Global oddball LFP results [26], [31], [32], [39], [42]. G) Local oddball spiking results [12], [32], [33], [45], [112], [113], [114]. H) Global oddball spiking results. [13], [14], [48], [48], [49], [53], [115], [116]. For further details, see Supplemental Materials, Table S1.

Do sensory neurons feed forward prediction errors?

Spiking responses to local oddballs are observed throughout cortex (Fig. 1C,D, Fig. 2G), from primary sensory areas to higher-order cortex [12], [32], [33], [45], [46], [47], [48]. However, studies of neuronal spiking to global oddballs tell a different story (Fig. 2H). We recently reported on spike and LFP responses in mid-level auditory cortex area Tpt and from higher-order prefrontal cortex (PFC) (the frontal eye fields, or FEF, part of PFC) during the auditory local-global oddball task in macaque monkeys (Fig. 1C). Local oddballs were robustly signaled in neuronal spiking both in auditory cortex and FEF, but global oddballs did not register in the population response (Fig. 1Ci and 1Ciii). Spiking activity to global oddballs was also examined in macaque visual cortex (areas V1 and V4) by varying the list order in which a stimulus would fall relative to a predicted order (Fig. 1D). Neurons did not encode prediction errors [49]. Instead, firing rates of neurons in V1 and V4 could be described by their classic bottom-up properties, including their orientation preference and stimulus repetition, along with enhanced spiking to unrepeated stimuli.

These observations argue against classical predictive coding because they do not show that predictions suppress population activity spiking in sensory areas to save energy during processing of predictable stimuli. Neurons spiked equally to unpredictable and predictable stimuli, after controlling for adaptation. So, if predictions in these studies failed to emerge in spiking studies at the level of V1 and V4 in visual processing (Fig. 1D) and in mid-level auditory processing (Fig. 1C), is it possible that genuine predictive codes emerge later in the hierarchy than previously thought? A recent study examined this possibility using a visual local-global oddball paradigm and recordings in monkey PFC [48]. Prefrontal neurons spontaneously formed internal models of the task structure, including both local and global oddballs. While global oddballs were decodable from the population response, they did not constitute the primary neuronal representation, i.e., the overall neuronal response in PFC was not enhanced during global oddballs [48]. Only a small fraction of PFC (~2% in [48], 9% in [32]) neurons had a significant response at the individual (spiking) channel level to global oddballs, challenging the idea that prediction errors drive widespread and robust representations.

To determine how widespread population decoding of global oddballs was and whether it was a feature of the conscious state [26], [31], [50], we examined spike decoding of the stimulus, local oddball, and global oddball status in auditory cortex (area Tpt) and FEF (a part of the prefrontal cortex) as we manipulated consciousness with the anesthetic propofol. Notably, global oddball decoding was absent in all states in auditory cortex (Fig. 1Cv). In FEF, global oddballs were decodable in the awake state, but decoding fell to chance levels in the unconscious state (Fig. 1Cvi, note that spike rate reductions in FEF may have contributed to the lack of decoding). These studies of spiking neurons during local/global predictions (Fig. 1C,D, Fig. 2H) challenge classic predictive coding models. Genuine predictive codes emerged late in processing, not early as hypothesized. In PFC, neurons flexibly created internal representations of all the sensory and latent task elements, supporting the idea of multi-dimensional, mixed selectivity in PFC [51], [52]. However, these predictions did not result in suppression of overall firing rates, as they were only present in a sparse sub-space of neuronal coding. These predictive codes remained undetectable from spiking activity in early to mid-levels of the auditory and visual sensory hierarchies (although genuine predictive codes have been reported in inferotemporal cortex, a late stage of visual sensory processing [13], [14], [53]).

To summarize, these studies of neuronal spiking diverged in their conclusions from previous fMRI, M/EEG, and LFP data (Fig. 2) by demonstrating that feedforward processing in early to mid-levels of sensory cortex does not represent prediction errors, and that predictions did not exert an overall suppressive influence on the population response, as hypothesized in predictive coding. Below we consider, and argue against, the notion that the observed failures in detecting genuine predictive coding in sensory cortex can be attributed to factors such as recording methodology, lack of cell-type specificity, and lack of explicit task engagement. We also consider to what extent the local oddball effects, which are present in spike rate studies in sensory cortex, can be considered a genuine form of predictive coding. We argue that none of these factors can rescue classical predictive coding and that a new perspective, predictive routing (see Fig. 3), can more parsimoniously explain the current data.

Figure 3: Predictive coding and predictive routing.

A) Panel A shows the classical predictive coding model as realized in a laminar cortical circuit with two levels of hierarchy. Prediction error is computed within a cortical column and then fed forward via superficial layer (Layer 2/3) cells to higher order cortex. Prediction is fed back via deep layer (Layer 5/6) cells from higher order area. The proposed computation is given as the shaded equation, where ξ is the precision weighted error, ε is the error term, and Π is precision (i denotes level of cortical hierarchy). ε is calculated with a subtraction between state μ and top-down prediction f, implying a local inhibitory mechanism commonly thought to be realized via superficial layer inhibitory cells (in red). B) The predictive routing framework integrates the function of neural oscillations into the predictive processing mechanism in the cortex. Prediction error is carried by gamma, fed forward in superficial layers, and prediction is carried by alpha/beta, fed back in deep layers. Predictive routing provides the following novel elements: i) preparatory rhythmic activity, shown as alpha/beta oscillations in the figure (but note that other mechanisms for rhythmic preparation are also possible, e.g., [117]), that ii) suppresses specific sensory channels (left subpanel: top-down predictions for stimulus A are sent to A selective neurons, to suppress processing of A). This removes the need for explicit prediction error neurons, as prediction “errors” are the result of sensory inputs to unprepared cortex (right subpanel), and iii) sparser predictive suppression in sensory cortex (only relevant representations are suppressed). Previous studies are consistent with the idea that prediction error computations emerge from the interaction between cortical rhythms and layers [12], [32], [39], [58], [77], [118], [119].

Recording methodology?

The discrepancy between the fMRI, M/EEG, and LFP vs. spikes may be explained by the different nature of the signals recorded across studies. Although fMRI has ~mm spatial resolution, it has poor temporal resolution, which means that early (first ~150ms of response, reflecting feedforward processing, [54]) vs. late (after ~150ms from the onset of a visual stimulus, can reflect both feedforward and feedback influences) response elements are conflated. As a result, fMRI maps may reflect both processing occurring in each area as well as the top-down inputs to an area. M/EEG and LFP reflect transmembrane currents which can be the result of both inputs to an area as well as its local computations [55], [56]. Only neuronal spiking activity can unambiguously resolve which computations occur where and when in the brain (see Box 1 for more in-depth discussion).

Box 1: Recording modalities and their relationships: fMRI, EEG/MEG, LFP, and spiking.

Extracellular recording of spiking activity represents the action potential output of neurons surrounding the recording contact. In contrast, LFP signal represents the superposition of ionic cellular currents in the brain at the location of the recording [56], with >95% of contributors to the (high-frequency portion) of the LFP from within ~250 μm of the recording electrode [120]. However, fMRI represents the intravascular magnetic susceptibility due to hemodynamic changes and blood oxygenation [55]. Investigations of neurophysiological correlates of the BOLD signal have found that BOLD correlates strongly with local field potential signal in the gamma frequency range but very weakly with the spike rate response [55]. This suggests that the BOLD fMRI signal primarily reflects the input of an area instead of the output and local cortical computation, which is better represented in spiking. BOLD signals have also been found to correlate with gamma power particularly in superficial layers of cortex [121]. These are the layers of cortex that receive the primary anatomical feedback connections (via layer 1, [122]) and contain pyramidal neurons that project feedforward outputs (primarily via layers 2&3, [123]). This suggests that BOLD signals and gamma (and other frequencies) LFP represent an integration of top-down inputs with local processing, and perhaps an integration of these signals to compute prediction error [124]. In sum, divergence between LFP/fMRI and spiking results can be explained by their respective underlying neural processes, with EEG/LFP/fMRI reflecting composite signals that better represent input to a given area, and spiking reflecting the neuronal output and local computation within an area. We note that a similar divergence between EEG/LFP/fMRI results showing extensive top-down modulation in sensory areas, but spiking studies showing weak or no top-down modulation has also been encountered for studies of working memory [125] and attention [126].

These considerations suggest that to understand where and how predictive codes emerge, it is necessary to complement fMRI, M/EEG, and LFP studies with better coverage of neuronal spiking. Traditional single unit recordings were limited to just one or two areas and a handful of neurons at a time, limiting this method’s field of view. This makes it easy to potentially miss a sparse signal such as global oddball encoding which may engage a small proportion of neurons. To mitigate this, we and others have recently worked to expand the scope of brain areas and neurons that can be measured using Multi-Area, high-Density, Laminar Neurophysiology (MaDeLaNe, [57], [58], [59], see Box 2). The emerging evidence from studies employing MaDeLaNe is largely consistent with Figs. 1&2, and shows that genuine predictive codes emerged at much later cortical processing stages than hypothesized by classical predictive coding [59].

Box 2: The importance of multi-area, high-density, laminar neurophysiology (MaDeLaNe) recording techniques.

Classical predictive coding models have been difficult to experimentally test because the model makes use of multiple stages of cortical processing, with distinct cell types, cortical layers, areas, and directions of feedforward/feedback processing contributing to the hypothesized computation. Rigorously testing predictive coding therefore requires methodologies that can capture each of these dimensions. A method is needed that delivers simultaneously good spatial (neuronal specific) resolution, temporal resolution (to determine when the computation happens), together with sufficient coverage to track the evolution of the cortical responses across the hierarchy (a comparison of currently used methods is shown in Fig. I). We and others have employed multiple high-density electrodes (see Fig. I), combined with novel analytical tools [58] to gain layer information during predictive processing in multiple cortical areas [47], [59], [127], [128]. Methods such as calcium imaging, optogenetics and high-density recording can be used in conjunction to distinguish neuronal cell types during predictive processing [59], [63], [64], [68]. Taken together, such methods are necessary to make further progress in understanding predictive coding and routing, to understand the contributions of distinct layers, cortical cell types, and processing stages [59]. We propose to further refine and improve MaDeLaNe methodology to further gain spatial coverage along with cellular specificity, and to utilize this method in diverse species (including at a minimum both rodents and primates, e.g., [59]) and where deemed clinically appropriate, in the human brain (e.g., [129], [130]) to uncover the circuitry underlying distinctly human forms of prediction [119].

Figure I, Box 2: MaDeLaNe sampling compared to fMRI, EEG/MEG, LFP.

Comparison of commonly used neuro-imaging and neuronal recording methods in neuroscientific research and their respective spatial coverage, spatial resolution, and temporal resolution. Traditional methods that offer brain-wide sampling (M/EEG and fMRI) suffer from either poor spatial or temporal resolution. Electrocorticography (ECoG, which we have grouped together with studies measuring LFPs from intracortical electrodes) retains high spatiotemporal resolution with large-scale coverage but does not sufficiently resolve layers and neurons in the spatial resolution axis [69]. Traditional invasive recordings using single-unit microelectrodes have poor spatial coverage (typically, a few neurons at a time in one or two areas). MaDeLaNe methods retain high spatiotemporal resolution (for detecting single-neurons) while also sampling densely across layers and in multiple areas [47], [59], [128]. Note that the goal of future methodological development is greater spatial coverage, and finer spatial and temporal resolution to obtain neuronal and cell-type specificity (illustrated here as towards the reader in the projected 3D diagram).

Cell-type specificity?

Although difficult to study in primates with current methods (but see [60], [61], [62]), cell-specific circuits during local oddballs have been investigated in mice. These studies often utilized passive tasks and repetition to establish a predictable stimulus [45], [63]. For example, deviance detection (a prediction error signal using repetition to establish prediction) is abolished in mouse V1 when top-down inputs from PFC to V1 are suppressed [45]. Two specific cell types may be involved in this type of predictive processing. Pharmacogenetic blockade of Somatostatin+ (SOM+) cells in mouse V1 eliminated deviance detection. Also in mouse V1, a study found that vasointestinal peptide-positive interneurons (VIP+) increased their activity to repeated (predictable) stimuli, suggesting VIP interneurons play a role in signaling predictions [63]. In addition, chemogenetic blockade of these neurons disrupted deviance detection [63] and VIP+ interneurons signaled unexpected omissions [64]. It is therefore possible that genuine predictive coding does emerge within specific neuronal populations. These signals, contrary to the hypotheses of classical predictive coding, appear insufficient to drive robust prediction error responses that could evoke feedforward processing. They failed to elicit a significant population response to global oddballs in the sensory areas [59], [64], largely consistent with the studies in monkeys (Fig. 1–2).

Explicit task engagement?

Finally, it is important to consider task and behavioral context. In predictive coding, prediction errors are modulated by their gain ([4], [9], but see [5], where prediction errors are assumed to be an automatic feature of neuronal activity). Gain can be parameterized as a form of attention [9]. The primate studies that reported failures in detecting global oddballs in sensory areas [32], [33] used paradigms where oddballs were presented passively, and therefore the gain on prediction errors may have been low. However, attentional modulation is unlikely to account for the lack of global oddballs in these studies. First, even without an explicit task, PFC neurons spontaneously formed inner models of the task structure [32], [48], including global context/deviance, which was eliminated during unconsciousness [32]. This implies some level of awareness/attention was present in the passive local-global oddball tasks. Second, selective attention in V1, V2, and V4 cortex modulates the neuronal response by ~5–23% [65]. It is unlikely that this level of modulation would create a robust global oddball response despite its absence in the reported data. Nevertheless, future studies should carefully control the state of selective attention during oddball processing to explicitly study the neuronal mechanisms of gain modulation of prediction errors [10], [66].

Sensory cortical neurons encoded local oddballs, so they may be described as encoding a local prediction that the current features of the sensory environment will persist (and repeat). One might be tempted to call this a predictive code. Indeed, this locally predictive code based on adaptation at different time scales is sufficient to compute a probability distribution [67]. However, although local oddballs might be modulated by top-down processes, it is worth remembering that these repetition-based local predictions persist in deep anesthesia (Fig. 1Ci) during which frontal cortex becomes inactive (Fig. 1Civ) [31], [32]. Local oddball responses therefore do not require the active feedback machinery envisioned by predictive coding to pass high-level predictions down the hierarchy.

What functions can top-down predictive inputs serve in sensory cortex if they don’t drive spiking?

Based on the current evidence, we believe that genuine prediction errors are computed in higher order cortex and do feedback to sensory areas but do not drive a population spiking response (e.g., no prediction errors detectable in the mean spike rates) in those areas. What might be the function of this feedback if it does not drive a supra-threshold response? We speculate that this top-down feedback may be sparse and selective, such that a population response is not observed in lower-order cortex but that some neurons do receive preparatory feedback that matters for a small population, perhaps especially involving sub-populations of inhibitory interneurons [64], [68], but see [59]. A second possibility is that top-down feedback exists via extra-classical mechanisms that do not drive a population spike response, but still modulates sensory areas via mechanisms such as sub-threshold oscillatory coupling [12], [32], [69], [70], [71], ephaptic coupling [72], [73], or dendritic mechanisms [74]. A third possibility is that top-down feedback is used to drive the motor system for behaviorally relevant predictions [68], which many of the current paradigms do not address. Table 1 summarizes the neuronal activity observed during local and global oddballs.

Table 1: What activity do local and global oddballs elicit?

A chart summarizing activity elicited from local and global oddballs as demonstrated in the current literature.

	Local Oddballs	Global Oddballs
Hierarchical emergence	widespread (Fig. 2A, C, E, G)	higher order areas in spiking (Fig. 2H)
Cortical layer	superficial layers dominant [12], [45], [63], [92]	Agranular layers (activation outside L4) [59]
Frequency of neuronal response	increased gamma; decreased alpha/beta [12], [32], [39], [42], [118]	decreased alpha/beta [42], [118]
Attentional dependence (gain)	not dependent [44]	dependent [26], [44]
Cell type	all cell types that are released from adaptation [59], [63]	specific inhibitory cell types [29], [64], [68] but see [59]
Consciousness-Dependent	yes (in higher-order cortex) no (for lower-order cortex) [31], [32], [50]	yes [31], [32], [50], [108]

How to reconcile these observations? Predictive routing

The findings reviewed here on global oddball responses suggest that genuine predictions emerge only at high levels of the hierarchy, inconsistent with classical predictive coding (Fig. 3A). We argue that the existing evidence can be well-accommodated within an updated model, predictive routing (PR, Fig. 3B) [12], see also [75]. The PR framework builds upon several properties of neuronal oscillations: Gamma frequency (40–90 Hz) increases in power with sensory stimuli and is positively correlated with spiking activity [76], [77]. Gamma/spiking is anti-correlated with alpha/beta frequency (8–30 Hz) oscillations which strengthen in PFC (and its coherence with other cortical areas) during top-down tasks, such as selective attention [78], [79] and when stimuli are predictable [12], [39] (for reviews of the top-down effects of beta during cognition, see [77], [80], [81]). This suggests that alpha/beta may act as an executive control mechanism by turning up or down the amount of gamma/spiking needed for a task [77].

PR involves learning-driven formation of dynamic ensembles [82], [83] in higher-order areas. This involves mixed selectivity neurons, which have been frequently observed in PFC [48], [51], [52]. Neurons with this property can form complex predictions in real-time based on any combination of inputs, rather than respond based on static receptive field characteristics. Once a predictive ensemble is formed amongst these mixed selectivity neurons, feedback connections can transmit these signals from higher-order to lower-order cortical regions utilizing alpha/beta rhythms [12], [39], [69], [84], [85], [86], [87] and prepare sensory areas for stimulus processing. Alpha/beta rhythms can have an inhibitory effect on neuronal spiking and gamma [12], [88], [89], [90], [91] at specific phases [88], [90], leading to a state of relative inhibition or “preparation” in predicted pathways [12]. Unpredicted stimuli arriving in unprepared cortical areas elicit enhanced spiking and gamma-frequency (40–90 Hz) oscillations, engaging enhanced feedforward communication (Fig. 3B) [12], [13], [31], [39], [42], [69], [85], [92].

The key difference between predictive coding (PC) and predictive routing (PR) is that in PR, there are no explicit prediction error neurons [12]. In contrast, in PC there is dedicated canonical circuitry in cortical layers 2&3 for prediction error computation (Fig. 3A). In PR, higher-order cortex issues selective preparatory signals that suppress sensory processing. A prediction “error” is a result of inputs arriving at an unprepared cortex (Fig. 3B). Higher-order cortex can issue these predictions in a sparse and selective manner, such that predictive suppression targets only relevant representations. This is compatible with the idea of redundant coding in sensory cortex, when neurons in early to mid-levels of the hierarchy fire away to even the most highly predictable stimuli and are not modulated by global oddballs (Fig. 1C&D). Higher-order predictions (e.g., global oddballs) depend on temporal integration across longer time scales at the level of the full sequence (Fig. 1A, typically, several seconds). Neurons in early sensory cortex have fast time scales of temporal integration [93] and would be ill-equipped to receive predictive suppression for sequences with long time scales. PR proposes that complex predictions are formed in PFC and selectively suppress sensory areas with longer time constants, such as high-level visual areas [94]. In PR, predictions are a higher-order, more selective, and sparser signal than in PC, and are implemented via spectrolaminar mechanisms rather than dedicated error circuits.

We recently tested PR using propofol-mediated unconsciousness during the auditory local-global oddball paradigm [32]. Propofol essentially inactivated PFC (Fig. 1Civ) while sparing bottom-up sensory drive to auditory cortex (Fig. 1Cii). Alpha/beta band power modulation was also eliminated with propofol. Under these conditions of no top-down input from PFC, we presented local oddballs and recorded neuronal activity in sensory cortex. Paradoxically, we found that during unconsciousness, local oddball-related gamma increased [32] relative to the awake state. We interpret this as evidence for PR: without beta-band activity (which normally increases during processing of predicted stimuli) and without top-down inputs from PFC, sensory cortex became disinhibited and generated more oddball-related gamma (along with temporally exaggerated spiking to oddballs, Fig. 1Cii) compared to the conscious state.

Implications for clinical disorders

The proposed PR model (with its emphasis on selective suppression of specific sensory areas) offers a valuable framework for understanding the neural mechanisms underlying sensory processing and predictive coding, in particular for schizophrenia (ScZ) and autism spectrum disorder (ASD). A key neural deficit in ScZ is reductions in mismatch negativity signals and reductions in sensory-induced gamma oscillations [95], [96], [97], which implies that there may be profound deficits in bottom-up sensory processing and predictive processing [98]. Reduced bottom-up sensory processing in ScZ may lead to weakened top-down predictive models and disrupted top-down beta [95], [99]. Without accurate input from the sensory environment, the brain is left to generate its own internal model of the world, which can lead to false predictions due to the lack of reliable sensory input (i.e., hallucinations and delusions). This feedforward sensory processing via dampened gamma may be the result of decreased functionality of parvalbumin interneurons [100], [101], which contribute to gamma oscillations [102] and are decreased in ScZ in specific areas and layers [101], [103].

In contrast, ASD symptomology presents as abnormally increased prediction errors [98]. A recent study recording electrocorticography from non-human primates with an induced form of ASD showed abnormally high sensory responses to local oddballs in the local-global oddball task [104]. Additionally, individuals with ASD often struggle with shifting attention between different stimuli or tasks [105]. This impairment could affect their ability to process global oddballs, as it requires flexible attentional allocation and the ability to update mental models.

Given the evidence, we propose that sensory cortex primarily employs a redundant code, ensuring robust representation of sensory input. In contrast, higher-order cortical areas may rely more on a predictive code, utilizing top-down predictions to efficiently process information. Imbalances in this process may underlie the distinct clinical presentations of ASD and ScZ. We propose that the local-global oddball task and its variants be systematically applied to studies of ScZ, ASD (and other clinical disorders) as functional assays for sensory and higher-order cognition. The use of these tasks in animal models of these clinical disorders is another exciting avenue which could generate biomarkers for states of disorder and provide objective markers to drive therapeutics targeted to specific types of processing (e.g., local/global oddballs).

Concluding remarks

The literature on local-global oddball processing presents a challenge to predictive coding models: M/EEG, fMRI, LFP data indicated that both local and global oddballs modulate activity in both sensory and higher-order cortex. However, studies of neuronal spiking in primates have failed to find robust global oddballs in lower and mid-level sensory areas. Predictive codes emerged in PFC but only in a sparse sub-space of neuronal encoding. Therefore, predictions may not be as broadly suppressive or as canonical as hypothesized. From the perspective of PR, global oddballs are an emergent feature of higher-order cortex neurons displaying flexible, mixed selectivity. Predictions are sent to the appropriate level of processing. Early to mid-sensory processing may be largely immune from the effects of prediction. More sophisticated task designs and higher-density neuronal recordings will provide further insights into the cortical circuitry for prediction. This may lead to a better understanding of clinical disorders that depend on intact predictions (see Outstanding questions).

Outstanding questions.

• Do active tasks (and attention) enhance the gain of prediction error responses to global oddballs in sensory cortex?

• Is there a generic canonical microcircuit for prediction error that operates with similar computations for local, global, omission, and other types of violation?

• How can new high-density, cell-type specific methods be used to learn which areas and specific cell types are involved in predictions in the primate brain?

• How can the mechanisms of predictive routing model be causally tested? Does rhythmic alpha/beta activity exert specific suppression via distinct interneuron subtypes?

• Do distinct species represent predictions at different stages of cortical processing and with distinct mechanisms?

• How can predictive coding/routing derived experimental paradigms be used to address bottom-up vs. top-down theories of clinical disorders such as schizophrenia? Does feedforward sensory processing via dampened gamma result from decreased functionality of parvalbumin interneurons?

Highlights.

• Predictive coding models propose that predictions are generated in higher-order areas and feed back to lower-order areas, where they are compared with sensory inputs. Mismatches generate prediction errors.

• The local-global oddball paradigm is used to study predictive coding. Local oddballs are formed from repetition-based predictions and do not necessarily imply a predictive code. Global oddballs dissociate stimulus repetition from predictability.

• Neuroimaging and intracortical spiking data have been used to investigate the local-global paradigm. Signatures of global oddball processing in sensory cortex are found in neuroimaging but not spiking data.

• This provides evidence against predictive coding. We provide a conceptual framework to guide future work, which we call predictive routing.

• We apply predictive routing to autism spectrum disorder (ASD) and schizophrenia (ScZ).

Glossary

Autism spectrum disorder (ASD)

a developmental condition characterized by challenges with social communication, limited interests, and repetitive behaviors [98]

Bayesian inference

a model that uses known information, termed “priors,” and incoming stimuli to form a prediction about its sensory environment. Priors are used to give probabilities to incoming stimuli, while the incoming stimuli are used to update these probabilities to better represent the current environment

Gain

the brain’s ability to determine the importance of prediction errors based on prior knowledge. Prediction errors with low gain will not be signaled to higher-order cortex

Local field potential (LFP)

an overall signal of local network activity from a large number of neurons surrounding the recording site, with a spatial resolution of 120–250 mm

Local/global oddball paradigm

an auditory oddball paradigm with 2 levels of regularity used to measure cognitive and attentional capabilities of the brain

Mismatch negativity (MMN)

an auditory event-related potential that occurs when a sequence of repetitive auditory stimuli is interrupted by an occasional “oddball” sound that differs in frequency or duration

Neuronal spiking

electrical impulses generated from single neurons reflecting action potentials, that can be recorded using invasive microelectrodes in experimental settings

Predictive routing (PR)

a theory that hypothesizes a flexible system for predictions that is dependent on the state of learning, context, and conscious state, and is implemented via specific cortical layers and neuronal rhythms

Prediction error

when environmental sensory inputs arrive that do not accord with the brain’s predicted internal model

Predictive coding (PC)

a theory of brain function which proposes that the brain is constantly generating and updating a model of its sensory environment. Internal predictions feedback to inform sensory processing, which feeds forward prediction errors

Schizophrenia (ScZ)

A severe mental illness involving symptoms such as delusions, hallucinations, disorganized speech, trouble thinking, and lack of motivation [98]