Algorithm 1 Distributed Q-learning for device m with $ϵ$-greedy learning policy.

for$i=0$ to N do     $Q_m^i\left(0\right)=0$ end for for$k=1$ to $k^{*}$ do     ${ϵ}_k={ϵ}_0{k}^{-1/N}$     Follow the $ϵ$-greedy policy (28) to get the next action $a_m\left(k\right)$.     Based on $a_m\left(k\right)$, change the computing state and sub-channel.     Observe sub-channel interference and calculate the reward $r_m^{a_m\left(k\right)}\left(k\right)$ according to (26).     for $i=0$ to N do          Calculate the $Q_m^i\left(k\right)$ according to (25) with the learning rate formulated at (30)     end for end for