Figure 2: Real-time training graphs of training and validation set accuracy/cross-entropy.

Training set curve is visualized in red; validation set in orange. Note that the model achieves perfect performance on the training set, but is unable to fully generalize this success to the validation set. Cross-entropy error is the quantity the network attempts to minimize after each iteration of training. Improvements in cross-entropy on the validation set plateau during the last half of training; thus, the network is “learning less” with each training cycle.