Figure 2.

A recurrent layer with a single node. A) A compressed representation of the recurrent node. B) An unfolded version illustrating the cycling of weights inside a recurrent node at different time steps. Note that the weights within each layer are shared but applied to different time steps. The first node $l_{i,t=1}^2$ simply inputs the information from the previous layer as there is no previous time step.