Algorithm 1 DQN algorithm.

1:Initialize replay memory pool $\mathcal{M}$   2:Initialize neural network weight $\theta$ and target weight ${\theta }^{\prime }=\theta$   3:for episode i = 1, M do   4:      for $t=1,T$ do   5:            Obtain state $s_t$ from the environment   6:            Randomly select an action $a_t$ or determine $a_t=arg{max}_{a}Q(s_t,a_t;\theta )$   7:            Observe the reward $R_t$ with the action $a_t$ and obtain the next state $s_{t+1}$   8:            Store transition $(s_t,a_t,R_t,s_{t+1})$ in memory $\mathcal{M}$   9:            Randomly sample a mini-batch $\tilde{\mathcal{M}}$ of $\mathcal{M}$ 10:            Update the evaluation network and perform ${\nabla }_{{\theta }_i}LF\left({\theta }_i\right)$ 11:            Update the target network after C steps 12:      end for 13:end for 14:Output: offloading decision strategy $a_t$.