Algorithm 2.1.

Rank-Restricted Soft SVD

$$\begin{array}{l}\text{1}.\text{Initialize}A=UD\text{where}{U}_{m\times r}\text{is a randomly chosen matrix with orthonormal columns}\text{and}D=I_r,\text{the}r\times r\text{identity matrix}.\\ 2.\text{Given}A,\text{solve for}B:\\ \underset{B}{\mathrm{minimize}}{\Vert X-A{B}^T\Vert }_F^T+\lambda \Vert B{\Vert }_F^2.(15)\\ \text{This is a multiresponse ridge regression, with solution}\\ {\tilde{B}}^T={\left(D^2+\lambda I\right)}^{-1}D{U}^{T}X.(16)\\ \text{This is simply matrix multiplication followed by coordinate-wise shrinkage}.\\ \text{3}.\text{Compute the SVD of}\tilde{B}D=\tilde{V}{\tilde{D}}^2{R}^T,\text{and let}V\leftarrow \tilde{V},D\leftarrow \tilde{D},\text{and}B=VD.\\ \text{4}.\text{Given}B,\text{solve for}A:\\ \underset{A}{\mathrm{minimize}}{\Vert X-A{B}^T\Vert }_F^T+\lambda \Vert A{\Vert }_F^2.(17)\\ \text{This is also a multiresponse ridge regression, with solution}\\ \tilde{A}=XVD{\left(D^2+\lambda I\right)}^{-1}.(18)\\ \text{Again matrix multiplication followed by coordinate-wise shrinkage}.\\ \text{5}.\text{Compute the SVD of}\tilde{A}D=\tilde{U}{\tilde{D}}^2{R}^T,\text{and let}U\leftarrow \tilde{U},D\leftarrow \tilde{D},\text{and}A=UD.\\ \text{6}.\text{Repeat steps}(2)-(5)\text{until convergence of}A{B}^T.\\ \text{7}.\text{Compute}M=XV,\text{and}\text{then}\text{it's}\text{SVD}:M=U{D}_{\sigma }{R}^T.\text{Then output}U,V\leftarrow VR\text{and}{S}_{\lambda }\left(D_{\sigma }\right)=\text{diag}\left[{\left({\sigma }_1-\lambda \right)}_{+},\dots ,{\left({\sigma }_r-\lambda \right)}_{+}\right].\end{array}$$