Algorithm 1 The single-layer DQN-based MRA training algorithm.

1:(Input) ${\lambda }_i,{\mu }_i,{\nu }_i,\forall i$, batch size $\eta$, learning rate $\alpha$, minimum exploration rate ${ϵ}_{min}$, discount factor $\zeta$, and exploration decay rate d; 2:(Output) Learned DQN to decide $P_i,{\theta }_i,f_i,\forall i$, for (7); 3:Initialize action $a^{\left(0\right)}$ and replay buffer $D=\varnothing$; 4:for episode = 1 to $\mathcal{M}$ do 5:    Initialize state $s^{\left(0\right)}$; 6:    for time $t=1$ to $\mathcal{N}$ do 7:        Observe current state $s^{\left(t\right)}$; 8:        $ϵ=max(ϵ·d,{ϵ}_{min})$; 9:        if random number $r<ϵ$ then 10:           Select $a^{\left(t\right)}\in \widehat{\mathbf{A}}$ at random; 11:        else 12:           Select $a^{\left(t\right)}=arg{max}_{a^{\prime }}{Q}^{*}(s^{\left(t\right)},a^{\prime },\omega )$; 13:        end if 14:        Observe next state $s^{\prime }$; 15:        Store transition $(s^{\left(t\right)},a^{\left(t\right)},{\mathfrak{r}}^{\left(t\right)},s^{\prime })$ in D, where ${\mathfrak{r}}^{\left(t\right)}$ is $U^{\left(t\right)}$ obtained with (23); 16:        Select randomly $\eta$ stored samples $(s^{\left(j\right)},a^{\left(j\right)},{\mathfrak{r}}^{\left(j\right)},s^{(j+1)})$ from D for experience; 17:        Obtain $\widehat{Q}(s^{\left(j\right)},a^{\left(j\right)},{\omega }^{\prime })$ for all j samples with (13); 18:        Perform SGD to minimize the loss in (14) for finding the optimal weight of DNN, ${\omega }^{*}$; 19:        Update $\omega ={\omega }^{*}$ in the DQN; 20:        $s^{\left(t\right)}=s^{\prime }$; 21:    end for 22:end for