Algorithm 1 The M-EXP3 Algorithm with the Modification.

Parameters: $\eta =\gamma \ast \alpha$ in $[0,1]$ where $\alpha >0$ is a discount factor initialization: $w_i\left(1\right)=1$ for all i = 1, …, K. For each time t = 1, 2, … At time t, Receive the experts’ advice vectors $B_i$ Calculate, for each action i, the probability $$P_i\left(t\right)=(1-\eta )\sum _{j=1}^N\frac{w_{i,j}\left(t\right)B_i\left(t\right)}{W_t}+\frac{\eta }{K}$$ (5) Calculate the sum of the weights of the actions at time t: $$W_t=\sum _{j=1}^K{w}_j\left(t\right)$$ (6) Choose action $I_t$ according to the max distribution $P_i\left(t\right)$, Receive a profit for the action i: $$g_i\left(t\right)\in [0,1],$$ (7) $$g{\ast }_i\left(t\right)=\left\{\begin{array}{c}{g}_{i_t}\left(t\right)/P_{i_t}\left(t\right) \mathrm{If} \mathrm{ACK} \left(r_j\left(i\right)\right) \mathrm{is} \mathrm{received}\\ 0 \mathrm{Otherwise}\end{array}\right.$$ (8) Update $B_i\left(t\right)$ as the reward (here the reward is a function of the expert in addition to the current action) $$y_i\left(t\right)=B_i\left(t\right)·g\left(t\right)=\sum _{i=1}^K{B}_i\left(t\right)g{\ast }_i\left(t\right)$$ (9) Update the weight of each expert $$w_j(t+1)=w_j\left(t\right)exp\left(\frac{\eta }{K}{y}_i\left(t\right)\right)$$ (10)