Algorithm 2 Distributed Q-learning for device m with Boltzmann learning policy.

for$i=0$ to N do     $Q_m^i\left(0\right)=0$     $p_m^i\left(0\right)=1/(N+1)$ end for for$k=1$ to $k^{*}$ do      if $max{p}_m^i\left(k\right)\ge 0.99$ then        exit.     end if     Choose the action $a_m\left(k\right)$ according to its current action selection probability vector ${\mathit{p}}_m\left(k\right)$.     Based on $a_m\left(k\right)$, change the computing state and sub-channel.     Observe sub-channel interference and calculate the reward $r_m^{a_m\left(k\right)}\left(k\right)$ according to (26).     for $i=0$ to N do          Calculate the $Q_m^i\left(k\right)$ according to (25) with the learning rate formulated at (33)          Update the action selection probability vector according to (31).     end for end for