Figure 12.

A BYOL architecture. BYOL reduces similarity loss between $q_{\theta }$($z_{\theta }$ ) and $sg$ ($z_{\xi }^{\text{'}}$ ), where $\theta$ represents the trained weights, ξ represents an exponential moving average of $\theta$, and sg means the stop-gradient. After training, everything but $f_{\theta }$ is discarded. $y_{\theta }$ represents the image representation.