Figure 3:

(A) Each heatmap shows a brute-force search over the shift parameters along the width and height dimensions of a pair of convolutional layers compared across two networks. The optimal shifts are typically close to zero (red lines). (B) Impact of sample size, $m$, on flattened and convolutional metrics with orthogonal invariance. The convolutional metric approaches its final value faster than the flattened metric, which is still increasing even at the full size of the CIFAR-10 test set $\left(m={10}^4\right)$. (C) Impact of sample density, $m/n$, on metrics invariant to permutation, orthogonal, regularized linear $(\alpha =0.5)$, and linear transformations. Shaded regions mark the 10^th and 90^th percentiles across shuffled repeats. Further details are provided in Supplement E.