Algorithm 1. Generating the training data set

Input: domain $C=\left\{c_1,c_2,\dots ,c_n\right\}$,

set $\mathbb{A}=\left\{\uparrow ,\nearrow ,\to ,\searrow ,\downarrow ,\swarrow ,\leftarrow ,\nwarrow ,\odot \right\}$ of possible actions,

probability $p_{TA}$ of true alarms (Equation (3)),

rate $\alpha$ of false alarms and their probability $p_{FA}=\alpha p_{TA}$ (Equation (4)),

sensor sensitivity $\lambda$,

range $\left[{\xi }_1,{\xi }_2\right]$ of possible numbers $0<{\xi }_1<{\xi }_2\le n-1$ of targets,

length $L\in \left(0,\infty \right)$ of the agent’s trajectory,

number $N\in \left(0,\infty \right)$ of agent trajectories,

initial probability map $P\left(0\right)$ on the domain $C$.

Output: data set that is an $L\times N$ table of pairs $\left(c,P\right)$ of agent positions $c$ and corresponding probability maps $P$.

1. Create the $L\times N$ data table.

2. For each agent trajectory $j=1,\dots ,N$ do:

3. Choose a number $\xi \in \left[{\xi }_1,{\xi }_2\right]$ of targets according to a uniform distribution on the interval $\left[{\xi }_1,{\xi }_2\right]$.

4. Choose the target locations $c_1, c_2,\dots ,c_{\xi }\in C$ randomly according to the uniform distribution on the domain $C$.

5. Choose the initial agent position $c\left(0\right)\in C$ randomly according to the uniform distribution on the domain $C$.

6. For $l=0,\dots , L-1$ do:

7. Save the pair $\langle c\left(l\right),P\left(l\right)\rangle$ as the $j$th element of the data table.

8. Choose an action $\mathbb{a}\left(l\right)\in \mathbb{A}$ randomly according to the uniform distribution on the set $\mathbb{A}$.

9. Apply the chosen action and set the next position $c\left(l+1\right)=\mathbb{a}\left(c\left(l\right)\right)$ of the agent.

10. Calculate the next probability map $P\left(l+1\right)$ with Equations (20) and (21).

11. End for

12. End for

13. Return the data table.