Figure 2.

Illustration of six time steps of the Bayesian model in action with no change-points. Starting from t = 1 from top to bottom: the model begins with a uniform prior distribution over dimension d and feature f, p(d, f | 𝒟_1:t) (rows in 3 × 3 plot denote dimensions, with three features each; shading denotes probability; see scale on right). Next the three stimuli, {s}_t+1 = {s_t+1(1), s_t+1(2), s_t+1(3)}, are observed and their values, p(r_t+1 | 𝒟_1:t, s_t+1(i)), computed. Given these values the choice probability CF(s_t+1(i)) is computed for each option (here with β = 10) and a choice c_t+1 is made. After the choice the model receives feedback about the reward, r_t+1, and uses this to compute the likelihood for each dimension and feature pair, p(r_t+1 | d, f, c_t+1), which is equal to either ρ_h or ρ_l. This likelihood is then multiplied by the prior distribution p(d, f | 𝒟_1:t) to obtain (after normalization) the posterior distribution over dimensions and features, which is the new prior distribution at the next time step. As is evident from the figure, on every trial the model gains new information about all the dimensions and features, thus allowing it to rapidly converge on the correct inference n(in this case, that the second texture feature, dots, were the most rewarding).