Algorithm 2: The CRE bias $\beta$ and the number of SBSs activation $\left|S\right|$ are given to optimize the ABS ratio $\alpha$.

Require: $A_t=(\alpha ,\beta ,\left|S\right|)$.

Ensure: Optimized ABS ratio $\alpha$.

1: Initialize $Q_{\alpha }(s,a)$, state s and n = 0;

2: Setting learning rate ${\lambda }_1$, greedy probability ${\epsilon }_1$, discount factor ${\gamma }_1$ and $threshol{d}_1$;

3: while n <= $threshol{d}_1$ do

4:  In state s, select the optimal action a with greedy probability ${\epsilon }_1$;

5:  Observe r;

6:  randomly transfer from s to $s^{{}^{\prime }}$;

7:  Update $Q_{\alpha }(s,a)$ according to Formula (18);

8:  $s\leftarrow s^{{}^{\prime }}$;

9:  $n=n+1$;

10: end while

11: Output: $\alpha =a$;