Figure 5.

Belief propagation as neuronal message passing. The equations on the left show the form of belief propagation (Equations 2 and 3) when expressed in a neuronally plausible form. These equations are written in terms of the sufficient statistics of the probability distributions and auxiliary variables representing prediction errors (${\mathbf{\epsilon }}_{\tau }$) and membrane potentials (${\mathbf{v}}_{\tau }$). The softmax ($\sigma$) functions act as neuronal transfer functions, converting presynaptic potentials to firing rates (${\mathbf{s}}_{\tau }$), which represent the sufficient statistics of the posterior beliefs. Forwards and backwards messages across the transition factors ($\mathbf{B}$) are written as ${\overrightarrow{{\mathit{\mu }}_{\tau }}}^{\mathbf{B}},{\overleftarrow{{\mathit{\mu }}_{\tau }}}^{\mathbf{B}}$ respectively. Red indicates an excitatory connection, while blue is inhibitory. The starred connection represents the subtraction of the ascending message from that passed on to other neuronal populations. This plays the role of an ascending ‘loop’, as in circular inference accounts of neuronal computation^4,37. The analogous descending loops are the inhibitory connections between the neurons representing messages in opposite directions. This formulation assumes that the neurons representing the messages have much shorter time constants than those representing marginal beliefs, allowing the former to be ‘enslaved’ by the latter^15,101. Although omitted here (and in later figures) for simplicity, the normalisation induced by the softmax functions could be mediated via recurrent inhibitory connections within a layer of neurons.