Figure 7.

Impact of varied pretraining hyperparameters on downstream test accuracy and pretext loss

(A) The effect of hyperparameter variations on downstream test accuracy.

(B) The effect of hyperparameter variations on pretext loss during pretraining. Regarding batch sizes, temperature parameters, thresholds, and dictionary sizes, blue indicates that the parameter value has been doubled, while red indicates that it has been halved. ∗For lighting conditions, blue represents the default setting, whereas red represents the use of multiple skyboxes.