Algorithm 9.

Proximal linearized 2-block ADMM with backtracking when the loss is differentiable and gradient is Lipschitz continuous

Input: $\mathbf{\text{X}},\gamma ,\mathbf{\text{w}},\alpha ,\zeta$

Initialize: U(0), V(0), Λ(0), t

Precompute: Difference matrix D, ${\tilde{x}}_j$

while not converged do

for j = 1 to p do

t = 1

${\tilde{\mathbf{u}}}_j^{\left(k-1\right)}={\mathbf{\text{U}}}_{.j}^{\left(k-1\right)}-{\tilde{x}}_j\cdot {\mathbf{1}}_n$

$\nabla g({\tilde{\mathbf{u}}}_j^{(k-1)})=\nabla \ell ({\mathbf{\text{X}}}_{.j},{\tilde{\mathbf{u}}}_j^{(k-1)}+{\tilde{x}}_j\cdot {\mathbf{1}}_n)+\rho {\mathbf{\text{D}}}^T(\mathbf{\text{D}}({\tilde{\mathbf{u}}}_j^{(k-1)}+{\tilde{x}}_j\cdot {\mathbf{1}}_n)-{\mathbf{\text{V}}}_{.j}^{(k-1)}+{\mathbf{\Lambda }}_{.j}^{(k-1)})$

$\mathbf{z}={\text{prox}}_{t\alpha {\zeta }_j{\Vert \cdot \Vert }_2}({\tilde{\mathbf{u}}}_j^{(k-1)}-t\nabla g({\tilde{\mathbf{u}}}_j^{(k-1)}))$

while $g(\mathbf{z})>g({\tilde{\mathbf{u}}}_j^{(k-1)}-\nabla g{({\tilde{\mathbf{u}}}_j^{(k-1)})}^T({\tilde{\mathbf{u}}}_j^{(k-1)}-\mathbf{z})+\frac{1}{2t}{\Vert \mathbf{z}-{\tilde{\mathbf{u}}}_j^{(k-1)}\Vert }_2^2\mathbf{do}$

t = βt

$\mathbf{z}={\text{prox}}_{t\alpha {\zeta }_j{\Vert \cdot \Vert }_2}({\tilde{\mathbf{u}}}_j^{(k-1)}-t\nabla g({\tilde{\mathbf{u}}}_j^{(k-1)}))$

end while

${\mathbf{\text{U}}}_{.j}^{(k)}=\mathbf{z}+{\tilde{x}}_j\cdot {\mathbf{1}}_n$

end for

${\mathbf{\text{V}}}^{(k)}={\text{prox}}_{\gamma /\rho P_1(\cdot ;\mathbf{\text{w}})}({\mathbf{\text{DU}}}^{(k)}+{\mathbf{\Lambda }}^{(k-1)})$

${\mathbf{\Lambda }}^{(k)}={\mathbf{\Lambda }}^{(k-1)}+({\mathbf{\text{DU}}}^{(k)}-{\mathbf{\text{V}}}^{(k)})$

end while

Output: U(k).