Table 1.
Applications of active inference for Markov decision processes
| Application | Comment | References |
|---|---|---|
| Decision making under uncertainty | Initial formulation of active inference for Markov decision processes and sequential policy optimisation | Friston et al. (2012c) |
| Optimal control (the mountain car problem) | Illustration of risk sensitive or KL control in an engineering benchmark | Friston et al. (2012a) |
| Evidence accumulation: Urns task | Demonstration of how beliefs states are absorbed into a generative model | FitzGerald et al. (2015b, c) |
| Addiction | Application to psychopathology | Schwartenbeck et al. (2015c) |
| Dopaminergic responses | Associating dopamine with the encoding of (expected) precision provides a plausible account of dopaminergic discharges | FitzGerald et al. (2015a), Friston et al. (2014) |
| Computational fMRI | Using Bayes optimal precision to predict activity in dopaminergic areas | Schwartenbeck et al. (2015a) |
| Choice preferences and epistemics | Empirical testing of the hypothesis that people prefer to keep options open | Schwartenbeck et al. (2015b) |
| Behavioural economics and trust games | Examining the effects of prior beliefs about self and others | Moutoussis et al. (2014), Prosser et al. (2018) |
| Foraging and two-step mazes; navigation in deep mazes | Formulation of epistemic and pragmatic value in terms of expected free energy | Friston et al. (2015) |
| Habit learning, reversal learning and devaluation | Learning as minimising variational free energy with respect to model parameters—and action selection as Bayesian model averaging | FitzGerald et al. (2014), Friston et al. (2016) |
| Saccadic searches and scene construction | Mean-field approximation for multifactorial hidden states, enabling high-dimensional beliefs and outcomes, c.f., functional segregation | Friston and Buzsaki (2016), Mirza et al. (2016) |
| Electrophysiological responses: place-cell activity, omission-related responses, mismatch negativity, P300, phase precession, theta–gamma coupling | Simulating neuronal processing with a gradient descent on variational free energy, c.f., dynamic Bayesian belief propagation based on marginal free energy | Friston et al. (2017a) |
| Structure learning, sleep and insight | Inclusion of parameters into expected free energy to enable structure learning via Bayesian model reduction | Friston et al. (2017b) |
| Narrative construction and reading | Hierarchical generalisation of generative model with deep temporal structure | Friston et al. (2017d), Parr and Friston (2017c) |
| Computational neuropsychology | Simulation of visual neglect, hallucinations and prefrontal syndromes under alternative pathological priors | Benrimoh et al. (2018), Parr and Friston (2017a), Parr et al. (2018a, b, 2019) |
| Neuromodulation | Use of precision parameters to manipulate exploration during saccadic searches; associating uncertainty with cholinergic and noradrenergic systems | Parr and Friston (2017b, 2019), Sales et al. (2018), Vincent et al. (2019) |
| Decisions to movements | Hybrid continuous and discrete generative models to implement decisions through movement | Friston et al. (2017c), Parr and Friston (2018) |
| Planning, navigation and niche construction | Agent-induced changes in environment (generative process); decomposition of goals into subgoals | Bruineberg et al. (2018), Kaplan and Friston (2018) |