Algorithm 4 The two-layer hybrid MRA training algorithm with the pricing strategy.

(Input)${\lambda }_i,{\mu }_i,\forall i$, ⋯; ⋯ for episode = 1 to $\mathcal{M}$ do     for time $t=1$ to $\mathcal{N}$ do       ⋯       Observe next state ${\mathbf{S}}^{\prime }$;       Obtain ${\nu }_{P_i^d},\forall i$, by using (50)       for each link i do          for iteration $k=1$ to $\mathcal{K}$ do            Update $P_i\left[k\right]$by using (47) with${\nu }_i={\nu }_{P_i^d},\forall i$;            Update ${\theta }_i\left[k\right]$by using (48) with${\nu }_i={\nu }_{P_i^d},\forall i$;            ⋯          end for       end for       ⋯     end for end for