Algorithm 3: The ABS ratio $\alpha$ and the number of SBSs activation $\left|S\right|$ are given to optimize the CRE bias $\beta$.

Require: $a_t=(\alpha ,\beta ,\left|S\right|)$.

Ensure: Optimized CRE bias $\beta$.

1: Initialize $Q_{\beta }(s,a)$, state s and n = 0;

2: Setting learning rate ${\lambda }_2$, greedy probability ${\epsilon }_2$, discount factor ${\gamma }_2$, and $threshol{d}_2$;

3: while n<= $threshol{d}_2$ do

4:  In state s, select the optimal action a with greedy probability ${\epsilon }_2$;

5:  Observe r;

6:  randomly transfer from s to $s^{{}^{\prime }}$;

7:  Update $Q_{\beta }(s,a)$ according to Formula (19);

8:  $s\leftarrow s^{{}^{\prime }}$;

9:  $n=n+1$;

10: end while

11: Output: $\beta =a$;