Table 2. Regularized NMF and convNMF: cost functions and algorithms.

NMF
$ℒ=\frac{1}{2}{||\widetilde{\mathbf{𝐗}}-\mathbf{𝐗}||}_2^2+ℛ$ $\widetilde{\mathbf{𝐗}}=\mathbf{𝐖𝐇}$	${\displaystyle \mathbf{W}\leftarrow \mathbf{W}\times \frac{\mathbf{X}{\mathbf{H}}^{\mathrm{\top }}}{\tilde{\mathbf{X}}{\mathbf{H}}^{\mathrm{\top }}+\frac{\mathrm{\partial }\mathcal{ℛ}}{\mathrm{\partial }\mathbf{W}}}}$ ${\displaystyle \mathbf{H}\leftarrow \mathbf{H}\times \frac{{\mathbf{W}}^{\mathrm{\top }}\mathbf{X}}{{\mathbf{W}}^{\mathrm{\top }}\tilde{\mathbf{X}}+\frac{\mathrm{\partial }\mathcal{ℛ}}{\mathrm{\partial }\mathbf{H}}}}$
convNMF
$ℒ=\frac{1}{2}{||\widetilde{\mathbf{𝐗}}-\mathbf{𝐗}||}_2^2+ℛ$ $\widetilde{\mathbf{𝐗}}=\mathbf{𝐖}⊛\mathbf{𝐇}$	${\displaystyle {\mathbf{W}}_{\cdot \cdot \ell }\leftarrow {\mathbf{W}}_{\cdot \cdot \ell }\times \frac{\mathbf{X}{\stackrel{\ell \to }{\mathbf{H}}}^{\mathrm{\top }}}{\tilde{\mathbf{X}}{\stackrel{\ell \to }{\mathbf{H}}}^{\mathrm{\top }}+\frac{\mathrm{\partial }\mathcal{ℛ}}{\mathrm{\partial }{\mathbf{W}}_{\cdot \cdot \ell }}}}$ ${\displaystyle \mathbf{H}\leftarrow \mathbf{H}\times \frac{\mathbf{W}\stackrel{\mathrm{\top }}{{\displaystyle ⊛}}\mathbf{X}}{\mathbf{W}\stackrel{\mathrm{\top }}{{\displaystyle ⊛}}\tilde{\mathbf{X}}+\frac{\mathrm{\partial }\mathcal{ℛ}}{\mathrm{\partial }\mathbf{H}}}}$
$L1$ regularization for $\mathbf{𝐇}$ ( $L1$ for $\mathbf{𝐖}$ is analogous)
$ℛ=\lambda {||\mathbf{𝐇}||}_1$	$\frac{\partial ℛ}{\partial {\mathbf{𝐖}}_{\cdot \cdot \mathrm{\ell }}}=0$ $\frac{\partial ℛ}{\partial \mathbf{𝐇}}=\lambda \mathrm{𝟏}$
Orthogonality cost for $\mathbf{𝐇}$
$ℛ=\frac{\lambda }{2}{||{\mathbf{𝐇𝐇}}^{\top }||}_{1,i\ne j}$	$\frac{\partial ℛ}{\partial {\mathbf{𝐖}}_{\cdot \cdot \mathrm{\ell }}}=0$ $\frac{\partial ℛ}{\partial \mathbf{𝐇}}=\lambda (\mathrm{𝟏}-\mathbf{𝐈})\mathbf{𝐇}$
Smoothed orthogonality cost for $\mathbf{𝐇}$ (favors ‘events-based’)
$ℛ=\frac{\lambda }{2}{||{\mathbf{𝐇𝐒𝐇}}^{\top }||}_{1,i\ne j}$	$\frac{\partial ℛ}{\partial {\mathbf{𝐖}}_{\cdot \cdot \mathrm{\ell }}}=0$ $\frac{\partial ℛ}{\partial \mathbf{𝐇}}=\lambda (\mathrm{𝟏}-\mathbf{𝐈})\mathbf{𝐇𝐒}$
Smoothed orthogonality cost for $\mathbf{𝐖}$ (favors ‘parts-based’)
$ℛ=\frac{\lambda }{2}{||{\mathbf{𝐖}}_{flat}^{\top }{\mathbf{𝐖}}_{flat}||}_{1,i\ne j}$  where ${({\mathbf{𝐖}}_{flat})}_{nk}={\sum }_{\mathrm{\ell }}{\mathbf{𝐖}}_{nk\mathrm{\ell }}$	$\frac{\partial ℛ}{\partial {\mathbf{𝐖}}_{\cdot \cdot \mathrm{\ell }}}=\lambda {\mathbf{𝐖}}_{flat}(\mathrm{𝟏}-\mathbf{𝐈})$ $\frac{\partial ℛ}{\partial \mathbf{𝐇}}=0$
Smoothed cross-factor orthogonality (x-ortho penalty)
$ℛ=\lambda {||(\mathbf{𝐖}\stackrel{\top }{⊛}\mathbf{𝐗}){\mathbf{𝐒𝐇}}^{\top }||}_{1,i\ne j}$	$\frac{\partial ℛ}{\partial {\mathbf{𝐖}}_{\cdot \cdot \mathrm{\ell }}}=\lambda \stackrel{\leftarrow \mathrm{\ell }}{\mathbf{𝐗}}{\mathbf{𝐒𝐇}}^{\top }(\mathrm{𝟏}-\mathbf{𝐈})$ $\frac{\partial ℛ}{\partial \mathbf{𝐇}}=\lambda (\mathrm{𝟏}-\mathbf{𝐈})\mathbf{𝐖}\stackrel{\top }{⊛}\mathbf{𝐗𝐒}$