Algorithm 1 MFMC algorithm to rebuilt a set of size p of T–length artificial trajectories from a sample of n one-step transitions.

Input: n, h(., .),x0, Δ(., .),T,p

Let denote the current set of not yet used one-step transitions in n; Initially,

← n;

for i = 1 to p (extract an artificial trajectory) do

t ← 0;

$x_t^i\leftarrow x_0$;

while t < T do

$u_t^i\leftarrow h(t,x_t^i)$;

$\mathcal{H}\leftarrow \underset{(x,u,r,y)\in \mathcal{G}}{\text{arg min}}\Delta ((x,u),(x_t^i,u_t^i))$;

$l_t^i$ ← lowest index in n of the transitions that belong to ;

t ← t + 1;

$x_t^i\leftarrow y^{l_t^i}$;

$\mathcal{G}\leftarrow \mathcal{G}\backslash \left\{\right(x^{l_t^i},u^{l_t^i},r^{l_t^i},y^{l_t^i}\left)\right\}$; \\ do not re-use transitions

end while

end for

Return the set of indices ${\{l_t^i\}}_{i=1,t=0}^{i=p,t=T-1}$.