What we think about when we think about predictive processing

Abstract

The predictive processing framework (PPF) attempts to tackle deep philosophical problems, including how the brain generates consciousness, how our bodies influence cognition, and how cognition alters perception. As such, it provides a zeitgeist that incorporates concepts from physics, computer science, mathematics, artificial intelligence, economics, psychology and neuroscience, leveraging and, in turn, influencing recent advances in reinforcement learning and deep-learning that underpin the artificial intelligence in many of the applications with which we interact daily.

PPF purports to provide no less than a grand unifying theory of mind and brain function, underwriting an account of perception, cognition and action and their dynamic relationships. While mindful of legitimate criticisms of the framework, to which we return below, an important test of PFF is its utility in accounting for individual differences such as psychopathology. These, then, are the central concern of this special issue of the Journal of Abnormal Psychology: what is the state of the art with regards to applying the PPF to the symptoms of mental illness? How might we leverage its insights to elevate and systematize our explanations, and ideally treatments, of those symptoms? And, conversely, can we refine and refute aspects of the PPF by considering the particular challenges that our patients experience as departures from the parametric estimates of the PPF?

Before we explore those lofty goals, we think it pertinent to inspect what the PPF is. Since many of its terms have everyday meanings, we feel it important to define some of them and sketch some of the assumptions and commitments of the PPF, since we adopt them when we develop PPF accounts in psychopathology.

Surprisingly Familiar: Revolutionary Old Ideas

The kernel of PPF accounts of the mind and brain is that prediction is central to perception, cognition and comportment. That is, perception is not just the passive receipt of sensory stimulation, rather the brain harbors predictions about likely inputs that color what is perceived. This insight traces back to the 4th century Buddhist scholar Vasubandhu, 9th century Islamic scholar Ibn Al Haythem, and the German Polymath Herman Von Helmholtz. For Helmholtz perception involved inference (though unconscious and non-deliberative) based on the association of ideas and previous experiences (Warren, 1921). Prediction therefore relates to associationism, which has its origins in western thought in Plato (Plato, 350 B.C./1999) and Aristotle (Aristotle, 350 B.C./1930), and was elaborated by Pavlov’s empirical work in the 20th century (Pavlov, 1927). These conditioning paradigms highlighted that mere contiguity was insufficient for learning and that the association of ideas was sensitive to surprise (McLaren & Dickinson, 1990), and indeed the processes of association and prediction engender a mismatch between what was expected and what was experienced. Building on this notion, the engineers Bernard Widrow and Marcian Hoff created a simple connectionist neural network of nodes, representing inputs and outputs as links between nodes (Widrow, 1960). Those links were strengthened by reducing a prediction error signal, the mismatch between the desired output from a given input and the output that actually occurred. A similar algorithm was proposed for animal conditioning by Rescorla and Wagner (Rescorla, 1972); environmental stimuli induce expectations about subsequent states of the world, exciting representations of those states. Any mismatch between the expectancies and actual experience is a prediction error, which is central to this framework.

The notion of an interplay between expectancy and experience resonates with Bayesian formalisms that are now central pillars for understanding basic psychological processes. Thomas Bayes was a British clergyman and mathematician whose doctrine of probabilities (published posthumously in 1873) (Bayes, 1958) embodies a formal approach to reasoning about data using hypotheses and captures the probabilistic nature of many of the tasks faced by organisms; to predict their environment and respond appropriately by minimizing its uncertainty about subsequent inputs. From the sense organs onward, the neural representations of stimuli are sculpted through hierarchical processing in the brain. Top-down expectations are communicated from areas with more abstract representations downwards through the hierarchical neural circuit (Mesulam, 2008). This casts association in statistical terms, where the prediction (or prior) is the weighted mean of some random variable. Prediction error then refers to the discrepancy between the predicted value of that variable and what is observed. Depending on the relative precision of priors and prediction errors, the error can be ignored or used to update subsequent expectations with new learning (Friston, 2005, 2009). The neuroanatomy and neurochemistry of backward and forward connections across cortical layers are exquisitely suited to this approach (Friston, 2005). Pavlov believed that Helmholtz’ unconscious perceptual inferences were aligned with his conditioned responses (Pavlov, 1927). Bayesian formalisms have been used to explain visual perception (Itti & Baldi, 2009; Rao & Ballard, 1999), perceptual learning (Fiser, Berkes, Orban, & Lengyel, 2010) and a phenomenon in which learning does not occur due to blocking (Courville, Daw, & Touretzky, 2006). Indeed, color-after effects in vision (McCollough, 1965), which are considered obligate and low-level appear to be subject to selective learning effects like blocking (Brand, Holding, & Jones, 1987; Sloane, Ost, Etheriedge, & Henderlite, 1989; Westbrook & Harrison, 1984). Taken together, these findings point towards a unified model of perception, action and belief driven by predictions and prediction errors.

One feature of these models that is now receiving attention is precision-weighting. The contributions that priors and prediction-errors make to inference and learning depend on their reliability (or inverse variance): more reliable prediction errors demand belief change, more reliable priors are robust to deviations. This feature too has precedence in the associative learning literature, wherein the associability of elements (cues, outcomes, causes, effects), their proclivity to enter into associative relations, or more simply the learning rate, is proportional to the degree of surprise the last time those elements were encountered. Often equated with attention, this model feature augurs a sensitivity to volatility, and, more broadly, underwrites beliefs that are robust to noise but malleable and adaptive to change.

Modelling What Matters to Gray Matter.

If much of what is attractive about PPF with regards to psychology is not particularly novel, what does PPF add? One valuable feature is its potential to be neurally realized, not just in the midbrain and basal ganglia, but across cortex. The suggestion, supported by neuroanatomical observations, is that the whole brain deals in predictions and prediction errors as part of a generative model of the causes of our ongoing sensorium. That model, and the cortex itself, is hierarchical such that activity in each layer tries to predict the activity in the layer projecting to it (Friston, 2005). For example, hierarchical predictive coding models of vision reflect features of visual receptive fields, like end-stopping - that some cells respond more vigorously to short than long stimuli. Rajesh Rao and Dana Ballard (Rao & Ballard, 1999) showed that a hierarchical (three-layer) model tracking predictions and prediction errors about natural image inputs evinced end-stopping, carried by ‘cells’ (nodes in the model) that signaled prediction errors.

There are several more components that provide more traction to PPF. Entropy refers to the uncertainty associated with model predictions. If our model of the world has low entropy, the data sampled from it are predictable. Agents should strive to occupy predictable states. Related to entropy is surprisal, the amount of information yielded by being in a particular state. Free energy, which refers to the probability of observing some data given a model of how those data were generated, serves as a proxy for surprisal. The free energy principle states that any self- organizing system (not just brains and bodies, but any living thing) acts to minimize mismatches between their predictions about the world and the way the world is. By minimizing free energy – brains minimize surprisal indirectly. Since free energy only depends on the sensory data and the model of the causes of those data – the claim is that free energy minimizing agents avoid surprises and live longer, in a manner that is computationally tractable – i.e. that could be realized in a brain. Another way of minimizing prediction error is active inference. Actions change the data that are sampled from the world (Braitenburg, 1986; W.T. Powers, 1973; W. T. Powers, 1978). Through active inference, prediction error, free energy and surprisal can all be minimized by gathering more predictable data – a confirmation bias of sorts.

Given the large number of variables (internal and external) with many possible values, inference and learning within PPF may become intractable, irrespective of ex-post-facto explanatory models. It is therefore reassuring that PPF computations can be tried and tested in silico. Predictive coding is an encoding strategy in signal processing whereby the expected features of an input signal are suppressed and only unexpected features are conveyed. This strategy is employed in the MP3 format, the vocoded speech popular in electronic music, and in the popular 1980s Speak and Spell toy. In each case, the compression process is reversed and the source signal re- synthesized from the prediction error. However, simply because there are models whose dimensions are tractable is insufficient, it remains to be confirmed empirically whether the models required to explain actual neural signal and behavior embody those more tractable features. This represents a key test of the PPF and attendant theories: do the assumptions they require hold? If they do not, we will have to refine and reject aspects of the PPF.

Predictive Processing in the Wild: Predictions as Motivated Perceptions

Although it may have triggered hysterics or even violence in people, no Speak and Spell -- a toy perfectly capable of predictive coding -- ever developed a serious mental illness to our knowledge (though studies of simulated patients with aberrant prediction errors have been reported (Hoffman et al., 2011; Yamashita & Tani, 2012)). The kinds of psychopathology that interest us have an inherently emotional quality that requires explaining. Disrupted emotion processing may underlie disorders ranging from internalizing disorders such as anxiety and mood disorders to externalizing disorders such as antisocial personality disorder and addictive disorders to thought disorders such as schizophrenia. Emotional disruptions may also constitute a broad liability related to comorbidity within and across these disorders (Kret & Ploeger, 2015). This view is bolstered with substantial evidence from cognitive neuroscience research exhibiting perceptual and attentional biases towards hedonic stimuli across these disorders (Cisler & Koster, 2010; Kret & Ploeger, 2015; Sabharwal et al., 2016). Despite the importance of emotion and motivation in clinical and cognitive accounts of mental disorders, the role of affect is often under-explored in PPF accounts of mental disorders.

This lack of emphasis on affect in PPF accounts may be due to a biased view that perception of emotional stimuli is driven by “bottom-up” processing due to physical characteristics or evolutionary significance of these stimuli (Bannerman, Milders, de Gelder, & Sahraie, 2009; Ohman, Flykt, & Esteves, 2001b). Consistent with this view, neuroscience research has focused on examining the neural pathways that promote ‘automatic’ perception of emotional stimuli (Fox, 2002; Vuilleumier & Pourtois, 2007). Hence, most research in psychopathology has focused on relatively “automatic” perception of emotion stimuli and how this disrupts cognition in mental disorders (Mathews & MacLeod, 1994; Ohman, et al., 2001b). For example, empirical studies examining perception of threatening stimuli in anxious individuals often utilize tasks that exogenously drive perception through the use of unanticipated or task-irrelevant stimuli where emotional stimuli ‘pop out’ amongst non-emotional stimuli (Fox et al., 2000; Ohman, Flykt, & Esteves, 2001a) Williams, Mathews, & MacLeod, 1996), are peripheral to fixation (Mogg & Bradley, 1999), or appear rapidly in a stream of images (Arend & Botella, 2002). PPF suggests a shift in focus from emotion perception that is driven automatically to neural representations that are shaped via hierarchical processing in the brain (Sussman et al., 2017; Mohanty & Sussman, 2013). Similar to non-emotional stimuli, top-down predictions or priors regarding valenced stimuli can be communicated hierarchically from higher order areas and the resulting prediction errors. In the present issue, Lyndon & Corlett describe how strongly consolidated trauma-related memories could lead to inaccurate but overly precise prior beliefs, triggered by trauma-relevant stimuli, resulting in the on-going symptoms of PTSD. While this represents an important step for incorporating emotional stimuli within PPF, overall, the role of PPF in explaining perception of external emotional stimuli (exteroception) is underdeveloped, including how stimuli acquire valence in the first place and how these processes can result in symptoms.

In contrast to exteroception, the role of PPF in interoception, or the perception of internal or somatic body signals for the generation of emotion and its disruption in mental disorders is well elaborated (Seth & Friston, 2016; Barrett, Quigley, & Hamilton, 2016). In the PPF framework, emotions have been recast as inferences regarding the best explanation for our interoceptive signals (heart rate, blood pressure etc). Hence, the discrepancy between predictions and interoceptive signals is minimized by updating predictions about the causes of the interceptive signals or by changing autonomic states such that they are more predictable (active inference) (Seth & Friston, 2016). Emotional states thus have been hypothesized to reflect changes in the uncertainty about the somatic consequences of action, such that increased precision of predictions about the (controllable) future result in positively valenced brain states while a loss of prior precision and uncertainty about the consequences of action are associated with negatively valenced states (Seth & Friston, 2016; Clark, Watson, & Friston, 2017; Joffily & Coricelli, 2013). Here, Mollick & Kober (this issue) discuss models of drug addiction characterized by more precise beliefs about reward-related physiological states in addicted individuals, who then ignore sensory evidence to the contrary (e.g., inaccurate priors). More recent models have extended the idea of precise priors to a further level of the hierarchy and proposed that depressed or anxious mood acts as a hyperprior such that the brain is certain that it will encounter uncertain uncontrollable environments (Clark, Watson, & Friston, 2017). While these explanations of depression and anxiety through the PPF lens are compelling, empirical data are not yet forthcoming.

There is also considerable evidence indicating that psychotic symptoms, traditionally considered disorders of thought without much regard to emotion, are in fact intricately intertwined with emotions (Freeman, Garety, & Kuipers, 2001; Freeman, 2007) and that cognitive deficits in psychoses are influenced by emotion and motivation (Barch & Dowd, 2010; Sabharwal et al., 2016). Indeed, Wellstein and colleagues (this issue) show that paranoia is explicable in terms of perturbed belief updating about social reputations. We have very little reliable data that inform our inferences about other individuals’ beliefs and intentions. Therefore, the central challenge of social inferences stems from an over-reliance on our generative models. If our mechanisms of inference are compromised, social inferences, because they are so challenging, will be among the first to be impaired.

The current studies were also selected to determine how PPF reflects on transdiagnostic symptoms over-and-above diagnostic categories. Donaldson and Mohanty (this issue) report that prediction errors in a popular perceptual task correspond to positive but not negative symptoms across people with schizophrenia and affective psychoses. If prediction errors are robust across diagnoses, they may also shed light on the transition to psychosis. Indeed, in a sample that includes both first episode psychosis and those at clinical risk for psychosis, Haarsma and colleagues (this issue) highlight how the trajectory of reliance upon perceptual priors changes with the magnitude of psychotic symptom expression. Furthermore, as reported by Fryer and colleagues (this issue), smaller prediction errors in a sample of clinical high-risk participants predict the likelihood of developing a more serious psychotic disorder.

This interlocking body of work builds a PPF account of prediction errors in various sensory and integrative tasks related to the development and expression of some symptoms and not others. In this regard, an important hypothesis to test was whether autistic symptoms might also be explicable in terms of an underweighting of prior expectations on sensory inference. This idea is examined, and ultimately rejected, by Ward and colleagues (this issue), lending further specificity to the PPF.

Another growth area for PPF is embodiment. There are of course PPF accounts of motor control and agency, which have been applied to explain the perturbations of bodily ownership and intentionality that characterize delusions of passivity and hallucinations. However, depression and anxiety are also associated with disruptions of bodily ownership and discomfort as well as immune and metabolic disturbances. We predict that the exquisite and powerful links between the brain, mind and periphery (underpinning sense of bodily ownership, immune recognition, and placebo responses) will be key growth areas for PPF in future. PPF certainly has detractors, whose concerns – including (but not limited to) its particular assumptions, falsifiability, tractability and originality – we take very seriously. Indeed, there are no doubt those among our readership who are not only skeptical of unifying theories but question their utility at all. Again, predictive coding is considered as one particular instantiation of the Free Energy Principle and part of the broad family of entities that comprises the PPF. This exercise reminds us that there are analytic levels (Marr, 1977) to consider with regards to the PPF: different authors tend to agree that the computational goal of the brain is to minimize prediction errors through a generative model. They depart with regards to the specific features of that model (the algorithmic level, what is represented and how those representations are manipulated in service of the computational goal) and how that model may be instantiated in the brain (the implementational level, how cells and systems perform those computations). These levels are worth considering as readers evaluate the papers in this special issue. Are they clearly expressed? Are the claims and commitments similar to other PPF papers and models encountered elsewhere? Do the authors describe whether and how neurons can even represent the quantities being computed and manipulated?

Ultimately, the value of any approach should be measured by what we learn from adopting it, how we turn that learning into better treatments for those who suffer and perhaps a deeper understanding of ourselves as living agents who both comprise and interact with our environment. We have not, thus far, had a theory with such broad and potentially integrative reach. This can be daunting, but we hope too that it is enlightening.