Figure 2.
Simplified diagram of long short-term memory (LSTM) neural networks. LSTMs are comprised of cells. In this figure, Cell 2 provides more detail on the inner workings of a cell and will serve as the reference point. Cell 2 can receive input from the data as x2, from Cell 1’s state as C1, and from the hidden layer output of the prior cell h1. Within Cell 2, these inputs can undergo further processing to generate a new hidden output h2 or be passed through the cell along with data from x2 and h1 with relatively little processing into a new cell state C2. This allows information encountered early to be passed along the network as “memory” to assist in calculations later in the network.