Figure 4. Richly trained RNNs replicating Overwrite representational geometry.
(A) Diagram of the inputs and outputs of the recurrent neural networks (RNNs). (B) Temporal decoding of RNNs under rich (small initial weight values – dark hues) and lazy training regimes (large initial weight values – light hues). (C) Two example decoding profiles of cross-temporal, cross-order, and cross-context decoding of RNNs representations under rich (upper) and lazy (lower) training regimes. (D) Euclidean similarity between empirical decoding results and the predictions of the Ordinal, Pre-allocate, and Overwrite models. (E, F) Representation dissimilarity matrices are calculated for neural data (E), and the RNNs (F). (G) The Pearson correlation r between representation dissimilarity matrices of empirical neural data and the RNNs. *: p < 0.05, **: p < 0.01, ***: p < 0.001, evaluated using a two-sided paired t-test to compare whether the best-fitting model exhibited significantly higher similarity than the second-best model. Colors indicate the best-predicting model (blue: Overwrite; red: Ordinal; green: Pre-allocate). Note that the Ordinal and Pre-allocate models share identical decoding profiles (Fig. 2B) for cross-temporal and cross-order decoding, and thus, both red and green colors are being used here. Abbreviations R1: Remember 1st; R2: Remember 2nd.