Algorithm 1 Reinforce with baseline (Rollout).

Require:  Initial network weight $\theta$, Input Instance x Ensure:  $<IsDone,\sum cost,OutputSequence$ P>     1:repeat     2:   Generate corresponding action $a_t$ through attention networks that consider traffic conditions     3:   if Decode strategy = greedy then     4:       Choose the point with the highest probability     5:       Rollout provides stable reference values based on determined strategies     6:   else if Decode strategy = sample then     7:       Select nodes based on probability value sampling     8:       The model enhances exploration capabilities through sampling     9:   end if   10:   Execute action $a_t$ and observe new observation $s_t$   11:   Put the node selected by action $a_t$ at into the Sequence P   12:   $\tau \leftarrow \tau +1$   13:until Service success or failure   14:Obtain the cumulative return $G_t^{greedy}$ of the model under the Rollout baseline   15:Obtain the cumulative return $G_t^{sample}$ of the model under the sampling strategy   16:${\theta }_{t+1}\leftarrow {\theta }_t+\alpha (G_t^{sample}-G_t^{greedy}){\nabla }_{\theta }log{P}_{\theta }\left(\pi \right|x)$   17:return$<IsDone,G_t^{greedy},\mathrm{P}>$