Algorithm 1 Model-free self-triggered control based on HRL.

Initialize$W,\theta ,\beta$ for$ep=0,1,\dots ,N$do     $t\leftarrow 0,x_0\leftarrow random\left(x\right)$     for $t\le t_{final}$ do         Choose ${\overline{u}}_l$ and ${\tau }_t$ using actor network with noisy signal         Apply ${\overline{u}}_l$ during ${\tau }_t$ and read $x_t$,$r_t$ on each sampling time unit $dt$         Add ($x_t$,${\overline{u}}_l$,$n·dt$,$r_t$) to the experience replay buffer $\mathcal{D}$         if $\mathrm{buffer}\mathrm{size}\ge \mathrm{mini-batch}\mathrm{size}$ then            for data in buffer do                Compute $\mathcal{L}\left(W\right):=\frac{1}{n}{\sum }_{i=1}^n{\left(\mathrm{G}\left(x_{ti},{\overline{u}}_{li},{\tau }_{ti}\mid W\right)-{\sum }_{t=t_l}^{t_l+{\tau }_{ti}}\left(x_{t_i}^{T}Q{x}_{t_i}+{\overline{u}}_{l_i}^{T}R{\overline{u}}_{li}\right)\right)}^2$                Using the adam optimizer to update the critic network parameters W                Compute the performance evaluation metrics J and $\eta$                Update the actor network parameters $\theta$ and $\mu$ using the formula (24)            end for         end if     end for end for