Algorithm 2: UpdateInformationofWinner.

1 //Input: user i; 2 Update the number of being randomly selected $n_i=n_i+1$; 3 Add the observed service quality $r_{it}$ to the total service quality $r_i=r_i+r_{it}$; 4 Update the learned expected service quality ${\widehat{q}}_i=max(\frac{r_i}{n_i}+\sqrt{\frac{8lnt}{n_i}},{\widehat{q}}_i$);