Algorithm 1 Normalization

Input: CNN Network and mini-batch Bx Output: Normalized sample data By

1. Mini-batch mean: $\mu =\frac{1}{s}{\displaystyle {\sum }_{i=1}^s{x}_i}$

2. Mini-batch variance: ${\sigma }^2=\frac{1}{s}{{\displaystyle {\sum }_{i=1}^s(x_i-\mu )}}^2$

3. Normalized value: $y_i=\frac{x_i-\mu }{\sigma }$

4. Update the global mean: ${\mu }_B=(1-\phi )\ast {\mu }_B+\phi \ast \mu$

5. Update the global variance: ${\sigma }_B{}^2=(1-\psi )*{\sigma }_B{}^2+\psi *{\sigma }^2$

6. Update the momentum value φ: $\phi =\phi -\gamma \frac{\partial L}{\partial \phi }$

7. Update the momentum value ψ: $\psi =\psi -\gamma \frac{\partial L}{\partial \psi }$