Algorithm 2 Generation algorithm for the safe decision controller

Input:$MDP=(S,A,T,R),{\tau }_r=<r,\phi ,{\phi }_r,risk,ris{k}_r>,done,update$ Output: Optimal safety policy $\pi$   1: Init ($\pi$)   2: $s_t\leftarrow s_0$   3: $a\leftarrow NOP$   4: while $s_t!=done\mathrm{and}episode<episodes$  do   5:    // Iterative selection of optimal value   6:     $A^{{}^{\prime }}=A$   7:    $a\leftarrow choose\left(A^{{}^{\prime }}\right)$ // Select action   8:    $env\leftarrow \mathrm{Environmental}\mathrm{sampling}$   9:    while $monitor(s_t,a,env)$ do // monitor 10:        $A^{{}^{\prime }}\leftarrow A^{{}^{\prime }}-a$ 11:        if $A^{{}^{\prime }}!=⌀$ then 12:           $a\leftarrow choose\left(A^{{}^{\prime }}\right)$ 13:        else 14:           $a\leftarrow choose(a_{\mathrm{conservative}}\in A)$ 15:           break 16:        end if 17:    end while 18:    $update(s_t,a,\pi )$ // Update parameters 19:    Go to the next state $s_{t+1}$ 20: end while