Algorithm 1 IterNorm Batch Whitening Algorithm [30].

Input: Batch input $\mathbf{X}\in {\mathbb{R}}^{d\times m}$      Hyperparameters: constant $ϵ$ and number of iterations K      Output: Whitened activations ${\mathbf{X}}_{\mathrm{W}}$  1:Calculate batch mean $\mathit{\mu }$ using (10)  2:Calculate centered activations ${\mathbf{X}}_C$ using (9)  3:Calculate the covariance matrix $\mathbf{\Sigma }$ as $\frac{1}{m}{\mathbf{X}}_C{\mathbf{X}}_C^T+ϵ{\mathbf{I}}_d$  4:Calculate the trace-normalized covariance matrix ${\mathbf{\Sigma }}_N=\mathbf{\Sigma }/\mathrm{tr}\left(\mathbf{\Sigma }\right)$  5:${\mathbf{P}}_0={\mathbf{I}}_d$  6:for $k=1$ to K do  7:    ${\mathbf{P}}_k=\frac{1}{2}({3\mathbf{P}}_{k-1}-{\mathbf{P}}_{k-1}^3{\mathbf{\Sigma }}_N)$  8:end for  9:Calculate the whitening matrix: $\mathbf{W}={\mathbf{P}}_K/\sqrt{\mathrm{tr}\left(\mathbf{\Sigma }\right)}$ 10:Calculate whitened output: ${\mathbf{X}}_W=\mathbf{W}{\mathbf{X}}_C$