Algorithm 3.1.

Rank-Restricted Efficient Maximum-Margin Matrix Factorization: softImpute-ALS

$$\begin{array}{l}\text{1}.\text{Initialize}A=UD\text{where}{U}_{m\times r}\text{is a randomly chosen matrix with orthonormal columns and}D=I_r,\text{the}r\times r\text{identity matrix, and}B=VD\text{with}V=0.\text{Alternatively, any prior solution}A=UD\text{and}B=VD\text{could be used as a warm start}.\\ \text{2}.\text{Given}A=UD\text{and}B=VD,\text{approximately solve}\\ \underset{\tilde{B}}{\mathrm{minimize}}\frac{1}{2}{\Vert P_{\Omega }\left(X-A{\tilde{B}}^T\right)\Vert }_F^T+\frac{\lambda }{2}\Vert \tilde{B}{\Vert }_F^2(20)\\ \text{to update}B.\text{We achieve that with the following steps:}\\ \text{(a) Let}{X}^{*}=\left(P_{\Omega }(X)-P_{\Omega }\left(A{B}^T\right)\right)+A{B}^T,\text{stored as}sparsepluslow-rank.\\ \text{(b) Solve}\\ \underset{\tilde{B}}{\mathrm{minimize}}\frac{1}{2}{\Vert X^{*}-A{\tilde{B}}^T\Vert }_F^2+\frac{\lambda }{2}\Vert \tilde{B}{\Vert }_F^2,(21)\\ \text{with solution}\\ {\tilde{B}}^T={\left(D^2+\lambda I\right)}^{-1}D{U}^T{X}^{*}\\ ={\left(D^2+\lambda I\right)}^{-1}D{U}^T\left(P_{\Omega }(X)-P_{\Omega }\left(A{B}^T\right)\right)(22)\\ +{\left(D^2+\lambda I\right)}^{-1}{D}^2{B}^T.(23)\\ \text{(c) Use this solution for}\tilde{B}\text{and update}V\text{and}D:\\ \text{i}.\text{Compute the SVD decomposition}\tilde{B}D=\tilde{U}{\tilde{D}}^2{\tilde{V}}^T;\\ \text{ii}.V\leftarrow \tilde{U},\text{and}D\leftarrow \tilde{D}.\\ \text{3}.\text{Given}B=VD,\text{solve for}A.\text{By symmetry, this is equivalent to step}2,\text{with}{X}^T\text{replacing}X,\text{and}B\text{and}A\text{interchanged}.\\ \text{4}.\text{Repeat steps}(2)-(3)\text{until convergence}.\\ 5.\text{Compute}M=X^{*}V,\text{and}\text{then}\text{it's}\text{SVD}:M\text{=}U{D}_{\sigma }{R}^T.\text{Then output}U,V\leftarrow VR\text{and}{D}_{\sigma ,\lambda }=\text{diag}\left[{\left({\sigma }_1-\lambda \right)}_{+},\dots ,{\left({\sigma }_r-\lambda \right)}_{+}\right].\end{array}$$