Algorithm 1: DDPG
1.	Initialize: The Critic networks $Q(s,a|{\theta }^Q)$ and Actor network $\mu (s|{\theta }^{\mu })$; the weights are ${\theta }^Q$ and ${\theta }^{\mu }$. The Critic target networks $Q\prime (s,a|{\theta }^Q)$ and Actor target network $\mu \prime$ have weights ${\theta }^{Q\prime }\leftarrow {\theta }^Q$ and ${\theta }^{\mu \prime }\leftarrow {\theta }^{\mu }$ The experience playback buffer (R) has size n. Empty the experience playback buffer (R).
2.	for episode = 1, 2, …, T do
3.	Reset the simulation parameters of the energy dispatch system to obtain the initial observation state, $s_1$.
4.	for i = 1, 2, …, I do
5.	Normalize state $s_i$ to $s_i\prime$.
6.	Obtain Actor network action $a_i$ and noise $n_i$:                                                         $a_i=\min (\max (\mu (s_i\prime |{\theta }^{\mu })+n_i,-1),1)$
7.	Execute action $a_i$, obtain the reward, $r_i$, and observe the new state, $s_{i+1}$.
8.	Store transmission $(s_i\prime ,a_i,r_i,s_{i+1}\prime )$ to the Replay Buffer (R).
9.	Select a batch of transition $(s_j\prime ,a_j,r_j,s_{j+1}\prime )$ from R, $\forall j=1,2,\dots ,I$
10.	Calculate $Q_{t\arg et,j}=r_j+\gamma Q\prime (s_{j+1},a_j\prime |{\theta }^{Q\prime })$
11.	Update the Critic network parameters ${\theta }_j^Q$ based on the mean square     loss function:                                                         $L({\theta }^Q)=\frac{1}{N}{\displaystyle \sum _{N=1}^N({(Q_{t\arg et,j}-Q(s_j\prime ,a_j|{\theta }^Q))}^2)}$.
12.	Update the Actor network using the stochastic policy gradient:                                                         ${\nabla }_{\theta \mu }J\approx \frac{1}{N}{\displaystyle \sum _j{\nabla }_{a}Q(s,a|{\theta }^Q){|}_{s=sj,a=\mu (sj)}{\nabla }_{\theta \mu }\mu (s|{\theta }^{\mu })|s_j}$.
13.	Update the target network parameters:                                                         ${\theta }^{Q\prime }\leftarrow \tau {\theta }_i^Q+(1-\tau ){\theta }_i^{Q\prime }$,                                                         ${\theta }^{\mu \prime }\leftarrow \tau {\theta }^{\mu }+(1-\tau ){\theta }^{\mu }$.
14.	end for
15	end for