Algorithm 1 FL-DDPG-based Offloading Method

Initialize actor network $\mu$ and critic network $Q$ Initialize target network with weights $\mu ’$ and critic network $Q’$ Initialize experience replay memory for each episode $e\in E$ do  Initialize parameters for simulation in VEC environment  Generate randomly an initial state $s_n^t$ for each vehicle agent $n\in N$  for each time slot $t\in T$ do   for each agent $n\in N$ do    Determine transmission power for offloading by selecting an action    $a_n^t=\mu \left(s^t\right)+\Delta \mu$, $\Delta \mu$ is a sampled noise    Execute the action $a_n^t,$receive reward $r_n^t$, and observe new state $s_n^{t+1}$    Store the tuple $(s_n^t, a_n^t,r_n^t,s_n^{t+1})$ into experience replay memory    Sample randomly a mini-batch of samples from experience replay memory    Update the critic and actor network via Equations (14) and (16)    Update target networks via Equations (17) and (18) If episode $e$ == $e_{aggregation}$ then   Upload the parameters to the VECS controller according to Equation (19)   Download the parameters from the VECS controller to each vehicle