Algorithm 1 Adversarial attack trick (with the MADDPG).

1: $StateAttack\left(Q_i^{\mu ’}\left(x,a_1,a_2,\dots ,a_n\right),{\mu }_i\left(s_i|{\theta }_i^{\mu }\right),s_i\right)$

2: $a_i={\mu }_i\left(s_i|{\theta }_i^{\mu }\right),Q_i=Q_i^{\mu ’}\left(x,a_1,a_2,\dots ,a_n\right)$

3: for $i=1,N_a$do

4:   $s_{i\left(\mathit{noise}\right)}=s_i+\mathit{Gaussian}(0,{\sigma }_s^2)$

5:  $a_{i\left(\mathit{noise}\right)}={\mu }_i\left(s_{i\left(\mathit{noise}\right)}\right|{\theta }_i^{\mu })$

6:  $Q_{i\left(\mathit{noise}\right)}=Q_i^{\mu ’}\left(x,a_1,a_2,\dots ,a_{i\left(\mathit{noise}\right)},\dots ,a_n\right)$

7:  if $Q_{i\left(\mathit{noise}\right)}<Q_i$then

8:  $Q_i=Q_{i\left(\mathit{noise}\right)},s_i=s_{i\left(\mathit{noise}\right)}$

9: end if

10: end for

11: return $s_{i\left(\mathit{noise}\right)}$