Algorithm 2: DQN-based dynamic SFC readjustment algorithm

Input: The network state $s_t$ the set of SFC $G_1,G_2\cdot \cdot \cdot G_m$, dynamic SFC deployment policy ${\mathrm{\Pi }}_1$. Output: Dynamic SFC readjustment policy ${\mathrm{\Pi }}_2$. 1: Initialize the action-value function $Q(s_t,a;\theta )$, where $\theta$ is the randomly generated neural network weights. 2: Initialize the target action-value function $\widehat{Q}(s_t,a;{\theta }^{-})$, where ${\theta }^{-}=\theta$. 3: Initialize the experience pool $D$ with memory $N$. 4: for episode in range (EPISODES): 5:  Generate a new collection of SFCs. 6:  Initialization state $s$. 7:  for step in range (STEPS): 8:     Generate the set of nodes that need to be readjusted based on the state of the underlying network. 9:     With probability $\epsilon$, select an action $a_t$ at random. 10:    Otherwise, select the action $a_t=\arg {\max }_{a}Q(s_t,a;\theta )$. 11:    Execute readjustment action $a_t$, $s_t\Rightarrow s_t{}^{\prime }$. 12:    Perform deployment with ${\mathrm{\Pi }}_1$. 13:    Observe reward $r_t$$, s_t{}^{\prime }\Rightarrow s_{t+1}$. 14:    Store transition $e_t=(s_t,a_t,r_t,s_{t+1})$ in $D$. 15:    Sample random minibatch of transitions $(s_j,a_j,r_j,s_{j+1})$ from D. 16:    Set $y_j=\left\{\begin{array}{l}{r}_j,r_j=end\\ r_j+\gamma {\max }_{a^{\prime }}\widehat{Q}(s_{j+1},a^{\prime };{\theta }^{\prime }),r_j\ne end\end{array}\right.$ 17:    Perform a gradient descent step on ${(y_j-Q(s_{j+1},a;\theta ))}^2$ with respect to the network parameters $\theta$. 18:    Every $C$ steps, reset $\widehat{Q}=Q$. 19:  End. 20: End.