An Overview of Variational Autoencoders for Source Separation, Finance, and Bio-Signal Applications

. 2021 Dec 28;24(1):55. doi: 10.3390/e24010055

Algorithm 3: VAE-GAN training

$θ_{Enc}, θ_{Dec}, θ_{Dis} \leftarrow initialize network parameters for encoder, decoder, and discriminator$ networks
for $do # of training iterations :$
$X^{(M)} \leftarrow random mini - batch$
$Z^{(M)} \leftarrow Enc (X^{(M)})$
$L_{prior} \leftarrow D_{K L} (q (Z^{(M)} | X^{(M)}) ∥ p (Z^{(M)})$
${\tilde{X}}^{(M)} \leftarrow Dec (Z^{(M)})$
$L_{Llike}^{Disl} \leftarrow - E_{q (Z^{(M)} | X^{(M)})} [p ({Dis}_{l} (X^{(M)}) | Z^{(M)})]$
$Z_{p}^{(M)} \leftarrow samples from prior N (0, I)$
$X_{p}^{(M)} \leftarrow Dec (Z_{p}^{(M)})$
$L_{GAN} \leftarrow log (Dis (X^{(M)})) + log (1 - Dis ({\tilde{X}}^{(M)}))$
$+ log (1 - Dis (X_{p}^{(M)}))$
$Update the network parameters with their stochastic gradients :$
$θ_{Enc} \overset{+}{\leftarrow} - \nabla_{θ_{Enc}} (L_{prior} + L_{llike}^{Dis_{l}})$
$θ_{Dec} \overset{+}{\leftarrow} - \nabla_{θ_{Dec}} (η L_{llike}^{Dis_{l}} - L_{GAN})$
$θ_{Dis} \overset{+}{\leftarrow} - \nabla_{θ_{Dis}} L_{GAN}$
end for