Algorithm A3 Overview of Vanilla-SLkGAN training

Require k, number of epochs $n_e$, batch size B, learning rate $\eta$       Initialize generator G with parameters ${\mathit{\theta }}_G$, discriminator D with parameters ${\mathit{\theta }}_D$.       for $i=1$ to $n_e$ do       Sample batch of real data $\mathbf{x}=\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_B\}$ from dataset       Sample batch of noise vectors $\mathbf{z}=\{{\mathbf{z}}_1,\dots ,{\mathbf{z}}_B\}\sim \mathcal{N}(\mathbf{0},\mathbf{I})$       Update the discriminator’s parameters using an Adam optimizer with learning rate $\eta$ by descending the gradient: $$\begin{array}{c}{\nabla }_{{\mathit{\theta }}_D}\left(-\frac{1}{B}\sum _{i=1}^B\left(log\left(D\left({\mathbf{x}}_i\right)\right)+log(1-D\left(G\left({\mathbf{z}}_i\right)\right))\right)\right)\end{array}$$       or update the discriminator’s parameters with a simplified (GP): $$\begin{array}{c}{\nabla }_{{\mathit{\theta }}_D}\left(-\frac{1}{B}\sum _{i=1}^B\left(log\left(D\left({\mathbf{x}}_i\right)\right)+log(1-D\left(G\left({\mathbf{z}}_i\right)\right))\right)\right.\\ \left.+5\left(\sum _{i=1}^B\left|\right|{\nabla }_{\mathbf{x}}log\left(\frac{D\left(\mathbf{x}\right)}{1-D\left(\mathbf{x}\right)}\right)|{|}_2^2\right)\right)\hfill \end{array}$$       Update the generator’s parameters using an Adam optimizer with learning rate $\eta$ and descending the gradient: $$\begin{array}{c}{\nabla }_{{\mathit{\theta }}_G}\left(\frac{1}{B}\sum _{i=1}^B\frac{1}{2}\left(\right|1-D\left(G\left({\mathbf{z}}_i\right)\right){|}^k-1)\right)\end{array}$$    end for