Fig. 4.

The structure of an LSTM. The system at each time-point is updated based on the current input data, the status of the system at the previous time-point, and the content of the memory. Here, $\varphi$ and $\sigma$ are hyperbolic tangent and sigmoid functions, respectively, and $\odot$ stands for the element-wise multiplication. ${\mathbf{i}}_t$, ${\mathbf{f}}_t$, ${\mathbf{o}}_t$ and $\mathbf{c}(t)$ denote input gate, forget gate, output gate and memory cell respectively