Fig. 5. Enhanced PredRNN++.

a Framework of the enhanced PredRNN++ model with five convolutional layers, and (b) visual illustration of the flow of input data in the spatial memory. In Fig. 5a, the Gradient Highway Unit (GHU; blue) is embedded between the first and the second convolutional layers, the horizontal red arrows denote the deep transition paths of the spatial memory, the vertical black arrows represent the updating direction of the temporal memory, and the blue parts indicate the gradient highway connecting the current time step directly with previous inputs. In Fig. 5b, “$⊚$” denotes the convolution, $C_n^k$ the temporal memory, $M_n^k$ the spatial memory, and $H_n^k$ the hidden state. $M_n^{k-1}$, $C_n^k$ and $H_n^{k-1}$ are concatenated to form a larger tensor, and then $H_n^k$ is generated by the convolution. The numbers below each memory indicate the dimensions of the corresponding tensors.