Figure 5.

Various mutual information between variables change with training iterations. (a) shows the change of mutual information $I(x_t,y_t^{q=4})$ , $I(x_t,y_t^{q=3})$ , $I(x_t,y_t^{q=2})$ and $I(x_t,{\widehat{x}}_t)$ with the increase of the number of iterations. From the figure, we can see that within the specified number of iterations, $I(x_t,{\widehat{x}}_t)\le I(x_t,y_t^{q=2})\le I(x_t,y_t^{q=3})\le I(x_t,y_t^{q=4})$ . Among them, q is the dimension of the coarse-grained system. (b) verifies the theory of information bottleneck on NIS when Scale (q) = 2. (c,d) show the change of mutual information $I(x_t,x_{t+1})$ , $I(y_t,y_{t+1})$ and $I(x_t,{\widehat{x}}_t)$ with the increase of the number of iterations. It can be seen that under different scales, the three mutual information values are close to each other. The standard deviations of the three variables gradually decrease with iteration. Therefore, $I(x_t,x_{t+1})\approx I(y_t,y_{t+1})=I(x_t,{\widehat{x}}_t)$ is reflected.