Algorithm 1: DQN-based dynamic SFC deployment algorithm

Input: The underlying network state $s_t$ the set of dynamically arriving SFC requests $r_1,r_2\cdot \cdot \cdot r_m$. Output: Dynamic SFC deployment policy ${\mathrm{\Pi }}_1$. 1: Initialize the action-value function $Q(s_t,a;\theta )$ where $\theta$ are the randomly generated neural network weights. 2: Initialize the target action-value function $\widehat{Q}(s_t,a;{\theta }^{-})$, where ${\theta }^{-}=\theta$. 3: Initialize the experience pool $D$ with memory $N$. 4: for episode in range (EPISODES): 5:  Generate a new collection of SFCs. 6:  Initialize state $s\dots$. 7:  for step in range (STEPS):     8:     Select the nodes that satisfy the resource and delay requirements.   9:     Select m nodes that are closest to the last deployed node among the nodes that satisfy the deployment requirements and add them to set $\mathsf{\Phi }$.   10:    With probability $\epsilon$, select an action $a_t$ at random.   11:    Otherwise, select the action $a_t=\arg {\max }_{a}Q(s_t,a;\theta ), a\in \mathsf{\Phi }$.   12:    Execute action $a_t$ and observe reward $r_t$.   13:    Store transition $e_t=(s_t,a_t,r_t,s_{t+1})$ in $D$.   14:    Sample random minibatch of transitions $(s_j,a_j,r_j,s_{j+1})$ from D.   15:    Set $y_j=\left\{\begin{array}{l}{r}_j,r_j=end\\ r_j+\gamma {\max }_{a^{\prime }}\widehat{Q}(s_{j+1},a^{\prime };{\theta }^{\prime }),r_j\ne end\end{array}\right.$   16:    Perform a gradient descent step on ${(y_j-Q(s_{j+1},a;\theta ))}^2$ with respect to the network parameters $\theta$.   17:    Every $C$ steps, reset $\widehat{Q}=Q$.   18:  End.   19: End.