Figure 1.

(a) The four possible environmental situations at any given time. In the fast-switching case (α_f), the environmental state changes between high (good, E_g) and low (bad, E_b) reward probabilities frequently (thick solid arrow). With a low switching probability (α_s), the environmental state tends to remain good or bad for a long time (thin solid arrow). With a very small probability γ, the switching rate can change between fast and slow (dashed arrows). (b) Approximate form of the optimal strategy, to aid intuition. The individual’s belief about the current environmental situation is three-dimensional, which is difficult to visualize. Here, we represent belief state in two-dimensions, as though we had summed across the joint probabilities to obtain the most common decision with respect to each combination of P(E_g) and P(α_f). The circles at each corner show the appropriate behaviour if knowledge were perfect (i.e. each probability is either 0 or 1). The individual should try (green shading) if the probability of good conditions (i.e. a high reward probability) is large and rest (red shading) if it is small. Due to the need to learn about current conditions, there is a range of belief space where the individual should try even if the probability of a reward is low (purple shading). The boundary between trying and resting (dashed line) is influenced by the probability that the switching rate is fast, because this alters the value of information