Algorithm 1. Dynamic variance sampling algorithm flow.

for $t=\mathrm{1,2},\cdots ,T$ do         for each node $i=1,\dots ,N$, sample ${\theta }_i\left(t\right)$ independently from the $\mathcal{N}\left({\widehat{\mu }}_i\left(t\right),\frac{1}{k_i\left(t\right)+1}\right)$ distribution         select node: $i\left(t\right)={argmax}_i{\theta }_i\left(t\right)$         observe reward: $r_{i\left(t\right)}$         update selected times: $k_{i\left(t\right)}\left(t+1\right)=k_{i\left(t\right)}\left(t\right)+1$         update mean benefit: ${\mu }_{i\left(t\right)}(t+1)=\frac{{\mu }_{i\left(t\right)}\left(t\right)\times k_{i\left(t\right)}\left(t\right)+r_{i\left(t\right)}}{k_{i\left(t\right)}+1}$ end for