Algorithm 2 First-Order Method for Solving (6). (PenNMF)

Require:$f:{\mathbb{R}}^{n\times p}\mapsto \mathbb{R}$, $\beta >0$;

1: Randomly choose $W^0$ satisfies ${W^0}^{\top }{W}^0=I_p$, set $k=0$;

2: while not terminate do

3:  Compute inexact gradient $$D\left(W^k\right)=\nabla f\left(W^k\right)-W^k\mathsf{\Phi }({W^k}^{\top }\nabla f\left(W^k\right))+\beta W^k({W^k}^{\top }{W}^k{W^k}^{\top }{W}^k-I_p);$$

4:  Compute stepsize ${\eta }^k$;

5:  $W^{k+1}=W^k-{\eta }^{k}D\left(W^k\right)$;

6:  $k++$;

7: end while

8: Return $W^k$