Fig 8. Untrained RNNs can be used to estimate value, read out beliefs, or decode hidden states, but do not resemble belief dynamics.

A. Time-varying activations of 20 example units in response to an odor input, in an untrained RNN with 50 units, initialized with a gain of 0.9 (see Materials and methods). B. Same as panel A, but for an initialization gain of 1.9. C. RPE MSE (see Fig 3D) as a function of initialization gain, after training each Value ESN’s value weights to estimate value during Starkweather Task 2. Circles depict the median across N = 12 Value ESNs initialized with the same gain. Dashed line indicates median across Task 2 Value RNNs with the same number of units. Same conventions for panels D-G. D. Belief R² (see Fig 4B) as a function of initialization gain. E. Cross-validated log-likelihood of state decoders (see Fig 4C) as a function of initialization gain. F-G. Number of time steps it took each Value ESN’s activity to return to its fixed point following an odor (panel F) or reward (panel G) observation, as a function of the initialization gain. H. Difference between each model’s odor memory and reward memory (see Fig 5E), for Value ESNs initialized with a gain of 1.9 (red) and Value RNNs (purple); same conventions as Fig 3D.