Figure 1.

Network architectures. The input, $s$, is a vector of pixels represented by the lower row of black dots. The neurons are the upper row of black dots and have internal states, $u$. The dark arrows are connections for neuron $k$. All other connections are denoted by dotted lines. (a) A standard architecture for a feedforward rectified (ReLU) encoder. (b) The architecture for the LCA network, which includes feedforward driving connections, $\Phi$, as well as a lateral connectivity matrix, $G$.