Figure 4.

Example of value convergence exploitation with the shifted exponential Lagrangian with $\eta =200$. In the top row, for the MNIST dataset aiming for a compression level $r^{\ast }=2$ and in the bottom row, for the TREC-6 dataset aiming for a compression level of $r^{\ast }=16$. In each row, from left to right it is shown (i) the information plane, where the region of possible solutions of the IB problem is shadowed in light orange and the information-theoretic limits are the dashed orange line; (ii) $I(T;Y)$ as a function of ${\beta }_u$; and (iii) the compression $I(X;T)$ as a function of ${\beta }_u$. In all plots, the red crosses joined by a dotted line represent the values computed with the training set, the blue dots the values computed with the validation set and the green stars the theoretical values computed as dictated by Proposition 3. Moreover, in all plots, it is indicated $H\left(Y\right)$ in a dashed, orange line. All values are shown in bits.