Algorithm 4 Learning and forgetting.

Parameters: gain ${\displaystyle \gamma }$, threshold ${\displaystyle \theta }$, goal-learning rate ${\displaystyle \alpha }$, forgetting rate ${\displaystyle \delta }$ Input: adjacency matrix ${\displaystyle \mathbf{A}}$, resource signals ${\displaystyle \mathbf{F}}$   ${\displaystyle \mathbf{M}\leftarrow 0}$                                                ${\displaystyle ▹}$ initiate map synapses at 0   ${\displaystyle \mathbf{G}\leftarrow 0}$                                                 ${\displaystyle ▹}$ initiate goal synapses at 0   ${\displaystyle t\leftarrow 0}$                                                          ${\displaystyle ▹}$${\displaystyle t}$ counts the steps   ${\displaystyle s(t)\leftarrow x}$ ${\displaystyle ▹}$ start random walk at ${\displaystyle x}$   while learning do     ${\displaystyle t\leftarrow t+1}$     ${\displaystyle s(t)\leftarrow }$ a random neighbor of ${\displaystyle s(t-1)}$                  ${\displaystyle ▹}$ continue the random walk     ${\displaystyle u_i(t)\leftarrow {\delta }_{i,s(t)}}$ for every point cell ${\displaystyle i}$                               ${\displaystyle ▹}$ point cell output     ${\displaystyle \mathbf{v}(t)\leftarrow {(\frac{1}{\gamma }\mathbf{1}-\mathbf{M})}^{-1}\mathbf{u}(t)}$                                       ${\displaystyle ▹}$ map cell output     for all map cell pairs ${\displaystyle (i,j)}$ do       if ${\displaystyle {\displaystyle v_j(t-1)>\theta }}$ then                                       ${\displaystyle ▹}$ if pre-synaptic high         if ${\displaystyle {\displaystyle v_i(t)>\theta }}$ then                                    ${\displaystyle ▹}$ if post-synaptic also high           ${\displaystyle M_{ij},M_{ji}\leftarrow 1}$                                     ${\displaystyle ▹}$ potentiate the synapses         else                                                    ${\displaystyle ▹}$ if post-synaptic low           ${\displaystyle M_{ij}\leftarrow {\mathrm{e}}^{-\delta }{M}_{ij}}$                                      ${\displaystyle ▹}$ depress the synapses           ${\displaystyle M_{ji}\leftarrow {\mathrm{e}}^{-\delta }{M}_{ji}}$         end if       end if     end for     ${\displaystyle \mathbf{r}\leftarrow \mathbf{G}\mathbf{v}(t)}$                                                         ${\displaystyle ▹}$ goal signals     for every goal neuron ${\displaystyle k}$ do       ${\displaystyle D\leftarrow F_{k,s(t)}-r_k}$  ${\displaystyle ▹}$ difference between resource signal and prediction from the map       if ${\displaystyle {\displaystyle D>0}}$ then         ${\displaystyle ▹}$ if the resource signal exceeds the prediction from the map         for every map neuron ${\displaystyle j}$ do           ${\displaystyle G_{kj}\leftarrow G_{kj}+\alpha D{v}_j(t)}$                             ${\displaystyle ▹}$ potentiate goal synapses         end for       else                                     ${\displaystyle ▹}$ if resource signal less than prediction         for every map neuron ${\displaystyle j}$ do           ${\displaystyle G_{kj}\leftarrow {\mathrm{e}}^{-\delta v_j}{G}_{kj}}$                                    ${\displaystyle ▹}$ depress goal synapses         end for       end if     end for   end while