Algorithm 1.

Proximal gradient descent with line search algorithm for solving Equation (8). Proximal steps given in Equation (17) for the group lasso penalty, in Algorithm 2 for the group sparse group lasso penalty, and in Algorithm 3 for the hierarchical penalty.

Require: λ > 0

m = 0, initialize W(0)

while not converged do

m = m + 1

determine γ by line search

for j = 1 to p do

$W_{:j}^{1(m+1)}={\text{prox}}_{\gamma \lambda \text{Ω}}\left(W_{:j}^{1(m)}-\gamma {\nabla }_{W_{:j}^1}𝓛\left({\mathbf{W}}^{(m)}\right)\right)$

end for

for l = 2 to L do

$W^{l(m+1)}=W^{l(m)}-\gamma {\nabla }_{W^l}𝓛\left({\mathbf{W}}^{(m)}\right)$

end for

end while

return (W(m))