Figure 4.

Attention mechanisms: (left) In the global attention model output decision for a particular slice is affected by all the slice in the stack, whereas (right) in the Toeplitz attention model the output is only affected by a local neighborhood of slices (e.g D = 1 in the exemplar). Note that this figure is intended to explain only the attention layers, the encoder and decoder structures can be seen in Fig. 3.