Algorithm 1: Forward-Backward Sweep Method (FBSM)

//— Initial step —// $k\leftarrow 0$ $p_0^k\left(s\right)\leftarrow p_0\left(s\right)$ for $t=0$ to $T-dt$ do    Initialize $u_t^k\left(z\right)$    $p_{t+dt}^k\left(s\right)\leftarrow p_t^k\left(s\right)+{\mathcal{L}}_{u_t^k}^{†}{p}_t^k\left(s\right)dt$ end for while $J\left[u_{0:T-dt}^k\right]$ do not converge do    if k is even then      //— Backward step —//      $w_T^{k+1}\left(s\right)\leftarrow g\left(s\right)$      for $t=T-dt$ to 0 do         $u_t^{k+1}\left(z\right)\leftarrow arg{min}_u{\mathbb{E}}_{p_t^k\left(x\right|z)}\left[\mathcal{H}(t,s,u,w_{t+dt}^{k+1})\right]$         $w_t^{k+1}\left(s\right)\leftarrow w_{t+dt}^{k+1}\left(s\right)+\mathcal{H}(t,s,u_t^{k+1},w_{t+dt}^{k+1})dt$      end for    else      //— Forward step —//      $p_0^{k+1}\left(s\right)\leftarrow p_0\left(s\right)$      for $t=0$ to $T-dt$ do         $u_t^{k+1}\left(z\right)\leftarrow arg{min}_u{\mathbb{E}}_{p_t^{k+1}\left(x\right|z)}\left[\mathcal{H}(t,s,u,w_{t+dt}^k)\right]$         $p_{t+dt}^{k+1}\left(s\right)\leftarrow p_t^{k+1}\left(s\right)+{\mathcal{L}}_{u_t^{k+1}}^{†}{p}_t^{k+1}\left(s\right)dt$      end for    end if    $k\leftarrow k+1$ end while return $u_{0:T-dt}^k$