Algorithm 2 Pre-training method for control actor network.

Initialize$\mu ,K$ for$ep=0,1,\dots ,N$do     $t\leftarrow 0,x_0\leftarrow random\left(x\right)$     for $t\le t_{final}$ do         Choose ${\tau }_t={\mathcal{H}}_{\sigma }\left(x_t\right)$         Choose $u_t={\mathcal{U}}_{\lambda }(x_t,{\tau }_t)$         Apply $u_t$ after the sampling time unit ${\tau }_t$         Add ($x_t$,${\tau }_t$,$u_t$) to the data buffer $\mathcal{D}$         if $\mathrm{buffer}\mathrm{size}\ge \mathrm{mini-batch}\mathrm{size}$ then            for data in $\mathcal{D}$ do                Generated by the control strategy network ${\widehat{u}}_t={\mathcal{U}}_{\eta }\left(x_t,{\tau }_t\right)$                Compute $\mathcal{L}\left(\mu \right)=\frac{1}{n}{\sum }_{i=1}^n{\left({\widehat{u}}_{ti}-u_{ti}\right)}^2$                Using the adam optimizer to update the network parameters $\mu$            end for         end if     end for end for