Figure 7. The performance of cognitive algorithms depends on the sampling regime.

A. We consider a random walk generator on a “ring of cliques” state-space⁵⁰ and compare three different sampling regimes (diffusion, superdiffusion, and minimally autocorrelated sampling) with respect to three distinct objectives which quantify the degree to which the sequences generated are optimized for exploration (panel B), learning (panel C), and sampling-based estimation (panel D) respectively. B. Exploration efficiency is defined as the fraction of states explored in an environment relative to the cumulative distance traversed. A steeper slope indicates that more states are explored for less travel distance. With respect to clustered state-spaces (panel A), the characteristic heavy-tail of the superdiffusive propagator implies that occasionally the next state sampled will be drawn from an alternative clique of states unlike diffusion which tends to remain within a single clique (Fig. E7). Minimally autocorrelated sampling is relatively inefficient since it repeatedly samples states across cliques in order to avoid the likelihood of re-sampling nearby states and thus is penalized heavily for traversal distance. C. Consolidation accuracy is the matrix correlation between the learned and true successor representations (SRs). This is plotted as a function of the number of sequences generated. Diffusive replay facilitates the most accurate structure learning of the underlying state-space. D. Sampling coverage is the fraction of states sampled across multiple sequences as a function of the number of sampling iterations. Minimally autocorrelated sampling excels since it is far less likely to resample states within and across sequences. E. The true successor representation matrix for the environment. F-H. The learned successor representations for superdiffusive, diffusive, minimally autocorrelated sequence generation respectively. I-L. Two-dimensional spectral embeddings of the true (panel I) and learned (panels J-L) SRs. The diffusive regime (panel K) successively recapitulates the geometric structure of the true SR (panel I) being composed of five well-spaced cliques of states. In contrast, the SR learned from superdiffusive replay (panel J) does not separate the cliques as clearly due to Lévy jumps leading to the erroneous consolidation of illusory long-range transitions. Minimally autocorrelated sequence generation (panel L) corrupts the spatial structure of the state-space.