Figure 1.

This figure illustrates the dependencies among hidden states and control states of a generative model of choice behaviour. Left panel: these equations specify the generative model in terms of the joint probability over observations, the states causing these observations and the precision of prior beliefs about those states. The form of these equations rests upon Markovian or independence assumptions about controlled state transitions. Right panel: The dependencies among these variables are depicted on the right. This Bayesian graph illustrates the dependencies among successive hidden states and how they depend upon past and future control states. Note that future control states (policies) depend upon the current state, because policies depend upon relative entropy or divergence between distributions over the final state that are, and are not, conditioned on the current state. The resulting choices depend upon the precision of beliefs about control states, which in turn depend upon the parameters of the model. Observed outcomes depend on, and only on, the hidden states at any given time. Given this generative model, an agent can make inferences about observed outcomes using standard (variational Bayes) schemes for approximate Bayesian inference (see next figure). See (Friston et al., 2013) for further details.