Algorithm 1.

WEAK–HIERNET: Generalized gradient descent to solve weak hierarchical lasso, (7), with elastic net penalty ε.

Inputs: X ∈ ℝn×p, Z ∈ ℝn×p(p−1), λ > 0. Initialize (β̂+(0), β̂−(0), Θ̂(0)).

For k = 1, 2, … until convergence:

Compute residual: r̂(k−1) ← y − X(β̂+(k−1) − β̂− (k−1)) − ZΘ̂(k−1)/2.

For j = 1, …, p: $$\begin{array}{r}({\widehat{\beta }}_j^{+(k)},{\widehat{\beta }}_j^{-(k)},{\widehat{\mathrm{\Theta }}}_j^{(k)})\leftarrow \text{ONEROW}(\delta {\widehat{\beta }}_j^{+(k-1)}-{tX}_j^T{\widehat{r}}^{(k-1)}),\\ \delta {\widehat{\beta }}^{-(k-1)}+{tX}_j^T{\widehat{r}}^{(k-1)}),\\ \delta {\widehat{\mathrm{\Theta }}}_j^{(k-1)}-{tZ}_{(j,·)}^T{\widehat{r}}^{(k-1)}),\end{array}$$

where ONEROW is given in Algorithm 3, δ = 1 − tε, and Z(j, ·) ∈ ℝn×(p−1) denotes the columns of Z involving Xj.