Algorithm 1 VIB-GMM algorithm for unsupervised learning.

1:input: Dataset $\mathcal{D}:={\left\{{\mathbf{x}}_i\right\}}_{i=1}^N$, parameter $s\ge 0$. 2:output: Optimal DNN weights ${\theta }^{\star },{\varphi }^{\star }$ and GMM parameters ${\psi }^{\star }=\{{\pi }_c^{\star }$, ${\mathit{\mu }}_c^{\star }$, ${\mathsf{\Sigma }}_c^{\star }{\}}_{c=1}^{\left|\mathcal{C}\right|}$. 3:initialization Initialize $\theta ,\varphi ,\psi$. 4:repeat 5:    Randomly select b mini-batch samples ${\left\{{\mathbf{x}}_i\right\}}_{i=1}^b$ from $\mathcal{D}$. 6:    Draw m random i.i.d samples ${\left\{{\mathbf{z}}_j\right\}}_{j=1}^m$ from $P_{\mathbf{Z}}$. 7:    Compute m samples ${\mathbf{u}}_{i,j}={\tilde{f}}_{\theta }({\mathbf{x}}_i,{\mathbf{z}}_j)$ 8:    For the selected mini-batch, compute gradients of the empirical cost (20). 9:    Update $\theta ,\varphi ,\psi$ using the estimated gradient (e.g., with SGD or ADAM). 10:until convergence of $\theta ,\varphi ,\psi$.