Algorithm 2 The single-layer DDPG-based MRA training algorithm.

1:(Input) ${\lambda }_i,{\mu }_i,{\nu }_i,\forall i$, batch size $\eta$, actor learning rate ${\alpha }_a$, critic learning rate ${\alpha }_c$, decay rate d, discount factor $\zeta$, and soft update parameter $\tau$; 2:(Output) Learned actor/critic to decide $P_i,{\theta }_i,f_i,\forall i$, for (7); 3:Initialize actor $Q_a(s;{\omega }_a)$, critic $Q(s,a;{\omega }_c)$, action $a^{\left(0\right)}$, replay buffer D, and set initial decay rate $d^{\left(0\right)}=1$; 4:for episode = 1 to $\mathcal{M}$ do 5:     Initialize state $s^{\left(0\right)}$ and ${\rho }^{\left(0\right)}$; 6:     for time $t=1$ to $\mathcal{N}$ do 7:          Normalize state $s^{\left(t\right)}$ with (32); 8:          Execute action $a^{\left(t\right)}$ in (30), obtain reward ${\mathfrak{r}}^{\left(t\right)}=U^{\left(t\right)}$ with (23), and observe new state $s^{\prime }$; 9:          if replay buffer D is not full then 10:               Store transition $(s^{\left(t\right)},a^{\left(t\right)},{\mathfrak{r}}^{\left(t\right)},s^{\prime })$ in D; 11:          else 12:               Replace the oldest one in buffer D with $(s^{\left(t\right)},a^{\left(t\right)},{\mathfrak{r}}^{\left(t\right)},s^{\prime })$; 13:               Set $d^{\left(t\right)}=d^{(t-1)}·d$; 14:               Randomly choose $\eta$ stored transitions from D; 15:               Update the critic online network by minimizing the loss function in (36); 16:               Update the actor online network with the gradient obtained by (37); 17:               Soft update the target networks with their parameters updated by (29); 18:               $s^{\left(t\right)}=s^{\prime }$; 19:          end if 20:     end for 21:   end for