Algorithm 3 Layer-by-Layer Training Method

1:$\mathbf{Initialize}{\mathbf{B}}_0={\mathbf{A}}^T$ and ${\mathit{\theta }}_0$    2:Learn ${\mathbf{B}}_0$ by minimizing $L_t^{\mathrm{linear}}$    3:Learn ${\mathit{\theta }}_0$ with fixed ${\mathbf{B}}_0$ by minimizing $L_t^{\mathrm{non}-\mathrm{linear}}$    4:Relearn ${\mathbf{\Omega }}_0=\left\{{\mathbf{B}}_0,{\mathit{\theta }}_0\right\}$ to minimize $L_0^{\mathrm{non}-\mathrm{linear}}$    5:$\mathbf{for}$$t=1,2,\cdots ,T-1$$\mathbf{do}$    6:${\mathbf{B}}_t={\mathbf{B}}_{t-1},{\mathit{\theta }}_t={\mathit{\theta }}_{t-1}$    7:Learn ${\mathbf{B}}_t$ with fixed ${\mathbf{\Omega }}_{t-1}$ by minimizing $L_t^{\mathrm{linear}}$    8:Relearn $\left\{{{\mathbf{B}}_t,\mathbf{\Omega }}_{t-1}\right\}$ by minimizing $L_t^{\mathrm{linear}}$    9:Learn ${\mathit{\theta }}_t$ with fixed $\left\{{{\mathbf{B}}_t,\mathbf{\Omega }}_{t-1}\right\}$ by minimizing $L_t^{\mathrm{non}-\mathrm{linear}}$  10:Relearn $\left\{{{\mathbf{B}}_t,\mathbf{\Omega }}_{t-1},{\mathit{\theta }}_t\right\}$ by minimizing $L_t^{\mathrm{non}-\mathrm{linear}}$  11:$\mathbf{end}\mathbf{for}$  12:return${\mathbf{\Omega }}_{T-1}$