Table 2:

Model details. Here, hierarchical means that there are parallel pathways for information to flow from the encoder to the decoder (Fig. 2), which is slightly different from the conventional notion. For variational models, this implies hierarchical dependencies between latents in a statistical sense [68]. This hierarchical dependence is reflected in the KL term for the cNVAE, where L is the number of hierarchical latent groups. See Supplementary section 9.3 for more details and section 9.1 for a derivation. All models have an equal # of latent dimensions (420, see Table 4), approximately the same # of convolutional layers, and # of parameters (~ 24 M). EPE, endpoint error.

Model	Architecture	Loss	Kullback–Leibler term (KL)
cNVAE	Hierarchical	EPE +β * KL	$\mathrm{K}\mathrm{L}={\sum }_{\ell =1}^L {\mathbb{E}}_{q\left({\mathit{z}}_{<\ell }\mid \mathit{x}\right)}\left[{\mathrm{K}\mathrm{L}}_{\ell }\right]$, where ${\mathrm{K}\mathrm{L}}_{\ell }:={𝒟}_{\mathrm{K}\mathrm{L}}\left[q\left({\mathit{z}}_{\ell }\mid \mathit{x},{\mathit{z}}_{<\ell }\right)\parallel p\left({\mathit{z}}_{\ell }\mid {\mathit{z}}_{<\ell }\right)\right]$
VAE	Non-hierarchical	EPE +β * KL	$\mathrm{K}\mathrm{L}={𝒟}_{\mathrm{K}\mathrm{L}}\left[q\right(\mathit{z}\mid \mathit{x})\parallel p(\mathit{z}\left)\right]$
cNAE	Hierarchical	EPE	-
AE	Non-hierarchical	EPE	-