Algorithm 2 MODDPG

Input:  State space $s_{rl}$ Output:  The action of UAV $a_{rl}$   1:Initialize Actor network parameters ${\theta }^{\mu }$, and Target Actor network parameters ${\theta }^{{\mu }^{\prime }}$, ${\theta }^{{\mu }^{\prime }}\leftarrow {\theta }^{\mu }$.   2:Initialize Critic network parameters ${\theta }^Q$, and Target Critic network parameters ${\theta }^{Q^{\prime }}$, ${\theta }^{Q^{\prime }}\leftarrow {\theta }^Q$.   3:Initialize experience Replay Buffer.   4:for $\mathbf{episode}=0\mathbf{to}$ M do   5:   Initialize $s_{rl}$.   6:   for time step = $t_1,t_2,\dots ,t_n$ do   7:     repeat   8:        With probability of $ϵ$ choose an action $a_{rl}=clip(\mu \left(s_{rl}\right|Q_{\mu })+ϵ,a_{low},a_{high})$.   9:        Perform action $a_{rl}$ and observe reward $r_{rl}$ and next state $s_{rl}^{\prime }$. 10:        Store the transition $(s_{rl},a_{rl},r_{rl},s_{rl}^{\prime })$ from P. 11:        if P ≥ Batch size then 12:           Randomly sample Mini batch transitions $(s_{rl},a_{rl},r_{rl},s_{rl}^{\prime })$ from P. 13:           Compute $y_{rl}$ (19). 14:           Update Critic network by minimizing the critic loss (20). 15:           Update Actor network by maximizing the actor loss (21). 16:           Soft Update Target Network Parameters. 17:           ${\theta }^{Q^{\prime }}\leftarrow \tau {\theta }^Q+(1-\tau ){\theta }^{Q^{\prime }}$            ${\theta }^{{\mu }^{\prime }}\leftarrow \tau {\theta }^{\mu }+(1-\tau ){\theta }^{{\mu }^{\prime }}$ 18:        end if 19:     until ${\sum }_{n=0}^N{t}_n\ge T$ 20:   end for 21: end for