Algorithm 2: Pseudocode for the DRL approach.

1.2 1:Initialize the experience replay buffer $\mathcal{B}$. 2:for each UAV i in N do 3:    Initialize the actor network $\pi \left(s_t\left|{\theta }^{\pi }\right.\right)$ with weights ${\theta }^{\pi }$. 4:    Initialize the critic network $Q\left(s,a\left|{\theta }^Q\right.\right)$. 5:    Initialize the target actor network ${\pi }^{{}^{\prime }}\left(s\left|{\theta }^{{\pi }^{{}^{\prime }}}\right.\right)$ with weights ${\theta }^{{\pi }^{{}^{\prime }}}$. 6:    Initialize the target critic network $Q^{{}^{\prime }}(s,a\left|{\theta }^{Q^{{}^{\prime }}}\right.)$ with weights ${\theta }^{Q^{{}^{\prime }}}$. 7:end for 8:for each episode in $\mathrm{H}$ do 9:    Initialize the locations of the UAVs. 10:    The initial speed is zero for the UAVs, and their battery energy is $E_{max}$. 11:    Initialize the environment. 12:    Receive the initial state $s_1$. 13:    for each time t in T do 14:        for each UAV i in N do 15:           Select action $a_t^i={\pi }^i\left(s_t\left|{\theta }^{\pi }\right.\right)+\mathcal{N}$, where $\mathcal{N}$ is the noise term. 16:        end for 17:        UAVs execute their actions $a_t=(a_t^1,\dots ,a_t^N)$. 18:        Update next state $s_{t+1}$, and obtain reward $r_t=(r_t^1,\dots ,r_t^N)$. 19:        for each UAV i in N do 20:           if UAV i moves outside the region or close to other UAVs then 21:               Find $r_t^i=U_t-f_t^i$. 22:               Neglect the new location and update $O_t^i$. 23:           end if 24:        end for 25:        Update $s_t⟵s_{t+1}$. 26:        Store $(s_t,a_t,r_t,s_{t+1})$ in the buffer. 27:        for each UAV i in N do 28:           Sample L random mini-batches $(s_t,a_t,r_t,s_{t+1})\in \mathcal{B}$. 29:           Find $y_t\left(b\right)=r_t\left(b\right)+\gamma Q^{{}^{\prime }}(s(b+1),{\pi }^{{}^{\prime }}\left(s(b+1)\right|{\theta }^{{\pi }^{{}^{\prime }}}.)|{\theta }^{Q^{{}^{\prime }}}.)$, where $b=1,\dots ,L$. 30:           Update weights ${\theta }^Q$ by minimizing: $L\left({\theta }^Q\right)=\frac{1}{L}{\sum }_{b=1}^L{\left[y_t\left(b\right)-Q\left(s\left(b\right),a\left(b\right)\left|{\theta }^Q\right.\right)\right]}^2$. 31:           Update weights ${\theta }^{\pi }$ by minimizing: $L\left({\theta }^{\pi }\right)=\frac{1}{L}{\sum }_{b=1}^L-Q\left(s\left(b\right),\pi \left(s\left(b\right)\left|{\theta }^{\pi }\right.\right)\left|{\theta }^Q\right.\right)$. 32:           Update the target network’s weights: ${\theta }^{Q^{{}^{\prime }}}=\epsilon {\theta }^{\mathrm{Q}}+(1-\epsilon ){\theta }^{Q^{{}^{\prime }}}$ and ${\theta }^{{\pi }^{{}^{\prime }}}=\epsilon {\theta }^{\pi }+(1-\epsilon ){\theta }^{{\pi }^{{}^{\prime }}}$. 33:        end for 34:    end for 35:end for