Figure 7. Model inference task with natural stimuli.

(A) We model a simple task of inferring the variance of local curvature in a region of an image. The system encodes randomly drawn image patches that model saccadic fixations. Individual image patches are encoded in sparse population activity via V1-like receptive fields (see Figure 7—figure supplement 1). Image patches are then decoded from the population activity, contrast-normalized, and projected onto V2-like curvature filters. The observer computes the variance of these filter outputs. (B) After a gaze shift from an area of low curvature (bottom square, ${\displaystyle \theta ={\theta }_1}$) to an area of high curvature (top square, ${\displaystyle \theta ={\theta }_2}$), the observer must update its estimate of local curvature. (C) Image patches that are surprising given the observer’s estimate (red) have larger variance in curvature, while expected patches (white) have low variance in curvature. Frames of highly overlapping patches were slightly shifted for display purposes. (D) Individual image patches have a large impact on the observer’s estimate when the observer is uncertain and when image patches have high centered surprise, analogous to the behavior observed in simple model environments (see Figure 1B). Shown for ${\displaystyle \lambda =0.1}$. Impact spans the interval [0, 34.12]. (E) The observer can exploit its uncertainty to adapt the sparsity of the sensory encoding (heatmap; blue trace). When the observer is certain (white marker), population activity can be significantly reduced without changing the inference error. Increases in uncertainty (red marker) result in bursts of activity (red bar). An encoder optimized for constant reconstruction error produces activity that remains constant over time (green trace). Inference error spans the interval [0, 2.22]. (F) The observer can similarly exploit the predicted surprise of incoming stimuli to reduce population activity when stimuli are expected. Inference error spans the interval [0, 1.57].

Figure 7—figure supplement 1. Sparse coding model of natural image patches.

(A) Each reconstructed image patch ${\displaystyle {\hat{x}}_t}$ is represented as a linear combination of basis functions ${\varphi }_i$ multiplied by sparse coefficients $\left\{y_{i,t}\right\}$. (B) Small values of the sparsity constraint $\lambda$ (upper row) result in a more accurate reconstruction of the stimulus, but require a stronger population response to encode. Large values of $\lambda$ (bottom row) result in a weaker population response at the cost of decreased quality of the stimulus reconstruction.