Table 1. Free parameters of the algorithm.

η MF1	learning rate in 1-move trials and second moves in 2-move trials
η MF2	learning rate in first moves in 2-move trials
${\eta }_r^{MF1}$	learning rate for replay in 1-move trials and second moves in 2-move trials
${\eta }_r^{MF2}$	learning rate for replay in first moves in 2-move trials
${\beta }_1^{MF1}$	inverse temperature in 1-move trials
${\beta }_2^{MF1}$	inverse temperature for second moves in 2-move trials
${\beta }_2^{MF2}$	inverse temperature for first moves in 2-move trials
θ	initialisation mean for QMF values
γ m	fixed bias for each action, subject to ∑m γa = 0
η MB	state-transition model learning rate
ρ	fraction of the state-transition model learning rate that is used for updating opposite transitions
ϕ MB	state-transition model forgetting
ϕ′MB	state-transition model forgetting upon spatial re-arrangement
ω	state-transition model re-arrangement success
ϕ MF	QMF values forgetting
ϕ′MF	QMF values forgetting upon spatial re-arrangement
ξ	gain threshold for initiating replay