Algorithm 5: The algorithm for optimizing initial problems.

Require: $\alpha \in \mathrm{A}$, $\beta \in \mathrm{B}$, $\left|S\right|=\left\{0,1,\dots ,{\left|S\right|}_{max}\right\}$, Reward r, Learning rate $\lambda =\left\{{\lambda }_1,{\lambda }_2,{\lambda }_3\right\}$,

Greedy probability $\epsilon =\left\{{\epsilon }_1,{\epsilon }_2,{\epsilon }_3\right\}$ and Discount factor $\gamma =\left\{{\gamma }_1,{\gamma }_2,{\gamma }_3\right\}$.

Ensure: Optimal action configuration $\left\{\alpha ,\beta ,\left|S\right|\right\}$ in each state.

1: Initialize $U_t$, $\alpha$, $\beta$, $\left|S\right|$,

2: while n <= $threshol{d}_4$ do

3:  Fixed the CRE bias $\beta$ and the number of SBS activation $\left|S\right|$, calculate the ABS ratio $\alpha$

according to Algorithm 2. Pass the solved $\alpha$ to step (4) and step (5);

4:  Fixing the ABS ratio $\alpha$ and the number of SBS activation $\left|S\right|$, calculate the CRE bias $\beta$

according to Algorithm 3. Pass the solved $\beta$ to step (3) and step (4);

5:  Fix the ABS ratio $\alpha$ and the CRE bias $\beta$, calculate the number of SBS activation $\left|S\right|$

according to Algorithm 4. Pass the solved $\left|S\right|$ to step (4) and step (3);

6:  $n=n+1$;

7: end while

8: Output: $\alpha$, $\beta$, $\left|S\right|$;