Figure 3.

A graphical example of one stratum training: Given one stratum of training data and factor matrices A⁽¹⁾, A⁽²⁾, A⁽³⁾, we run SGD on each block in parallel. Then factor matrices A⁽¹⁾, A⁽²⁾, A⁽³⁾ are updated and used as the initialization for the next stratum training.