Table 2.

The optimization process for the Bayesian LSTM networks.

Step	Optimization Process
0	Set the scale parameter $\alpha$ as $\alpha \in (0,1)$.
1	Sample the random variable $\epsilon$ as $\epsilon \sim \mathcal{N}(0,1)$.
2	Set the initial value of the optimized parameters $(\mu ,\rho )$.
3	Sample all the parameters as $\theta =\mu +log(1+exp(\rho \left)\right)\otimes \epsilon$.
4	Set the cost function as $Loss=logq\left(\theta \right|\mu ,\rho )-logP(\theta )+logP(D\left|\theta \right)$.
5	Calculate the gradient by the mean with the training data D as $△\mu =\frac{\partial Loss}{\partial \theta }+\frac{\partial Loss}{\partial \mu }$.
6	Calculate the gradient by the standard deviation with the training data D as $△\rho =\frac{\partial Loss}{\partial \theta }\frac{\epsilon }{1+exp(-\rho )}+\frac{\partial Loss}{\partial \rho }$.
7	Update the parameters $(\mu ,\rho )$ as the following: $\mu \leftarrow \mu -\alpha △\mu ,\rho \leftarrow \rho -\alpha △\rho$.