Figure 2.

Entropy is the average surprisal and represents the uncertainty over states at a given moment in time. The free energy is a quantity that will always bound the surprisal from above. In the inference step (i.e., perception), free energy is minimized, i.e., the upper bound on surprisal is “pushed” down by changing beliefs or predictions concerning sensory outcomes. This minimization enforces a tight upper bound on the entropy. In the active (i.e., action) step sensations are sampled from the external states that reduce surprisal and thus ensure an upper bound on entropy.