Table 1.

Data distillation: The relationship between principal component analysis (PCA), canonical correlation analysis (CCA), nonlinear autoencoders and nonlinear latent representations.

Random Vectors	What Is Distilled?	Probability Distribution
		Gaussian	Non-Gaussian
1	Entropy	PCA	Autoencoder
	$H\left(X\right)={\sum }_{i}H\left(Z_i\right)$	$\mathbf{z}=\mathbf{F}\mathbf{x}$	$Z=f\left(X\right)$
2	Mutual information	CCA	Latent reps
	$I(X,Y)={\sum }_{i}I(Z_i,Z_i^{\prime })$	$\mathbf{z}=\mathbf{F}\mathbf{x}$	$Z=f\left(X\right)$
		${\mathbf{z}}^{\prime }=\mathbf{G}\mathbf{y}$	$Z^{\prime }=g\left(Y\right)$