Figure 3.

Visualizing the weights of the fully connected layers in MLP-Mixer and ResMLP

Visualizing the weights of the fully connected layers in MLP-Mixer (A) and ResMLP (B). Each weight matrix is resized to $14\times 14$ pixel images. In MLP-Mixer, white denotes that weight is 0, red means positive weights, blue means negative weights, and the brighter, the greater the weight. In ResMLP, black indicates that weight is 0, and the brighter, the greater the weight’s absolute value. The results are from Tolstikhin et al.¹⁵ and Touvron et al.,⁶⁰ respectively.