Figure 2:

(A) Schematic illustration of metrics with rotational invariance (top), and linear invariance (bottom). Red and blue dots represent a pair of network representations ${\mathit{X}}_i$ and ${\mathit{X}}_j$, which correspond to $m$ points in $n$-dimensional space. (B) Demonstration of convolutional metric on toy data. Flattened metrics (e.g. [6, 9]) that ignore convolutional layer structure treat permuted images (${\mathit{X}}_k$, right) as equivalent to images with coherent spatial structure (${\mathit{X}}_i$ and ${\mathit{X}}_j$, left and middle). A convolutional metric, Eq. (11), distinguishes between these cases while still treating ${\mathit{X}}_i$ and ${\mathit{X}}_j$ as equivalent (obeying translation invariance).