Algorithm 2 DRL-Based Multi-Modal Data Collection Algorithm

1:Input: Initialize the constants $k_1$, $k_2$ and $k_3$, maximum number of training sets E, reward discount factor $\gamma$, learning rate $l_r$, experience replay $\mathcal{B}$, minimum batch ${\mathrm{\Phi }}_b$, exploration probability $ϵ$, and update step $\chi$;   2:Initialize the current network $Q(s_t,a_t,\mathit{\theta })$ with weights $\mathit{\theta }$ and the target network $Q(s_t,a_t,{\mathit{\theta }}^{-})$ with weights ${\mathit{\theta }}^{-}$.   3:for$episode=1,\cdots ,E$do   4:    for $t=1,\cdots ,\mathcal{T}$ do   5:        Initialize the data collection network environment and observe the initial state $s_t$.   6:        Select a random action $a_t$ according to the $ϵ$-greedy algorithm.   7:        Determine the AUV steering angle with Algorithm 1.   8:        Execute action $a_t$ and observe the reward $r_t$ and the next state $s_{t+1}$.   9:        Store experience $(s_t,a_t,r_t,s_{t+1})$ in experience replay $\mathcal{B}$. 10:        Sample a random mini-batch of ${\mathrm{\Phi }}_b$ experiences from $\mathcal{B}$. 11:        Calculate the target value $y_t$ by (26). 12:        Update the current network weights $\mathit{\theta }$ by (27). 13:        Update the weights of the target network ${\mathit{\theta }}^{-}=\mathit{\theta }$ every $\chi$ steps. 14:        if $s_{t+1}$ is the collection stop $n_i$ then 15:           Remove the CH $c_i$ from $N_r$. 16:        end if 17:        Terminate the episode if $N_r=\varnothing$ holds. 18:    end for 19:end for 20:Output: The AUV trajectory $p_{a,t}$ and the AoI $A_i$.