Algorithm 2.

Hierarchical Proximal Operator

1:	procedure Hier-Prox(θ, W(1); λ, M)
2:	for j ∈ {1,..., d} do
3:	Sort the entries of $W_j^{(1)}$ into $|W_{(j,1)}^{(1)}|\ge \dots \ge |W_{(j,K)}^{(1)}|$
4:	for m ∈ {0, ..., K} do
5:	Compute $w_m\equiv \frac{M}{1+m{M}^2}\cdot {\mathcal{S}}_{\lambda }\left(\left|{\theta }_j\right|+M·{\sum }_{i=1}^m|W_{(j,i)}^{(1)}|\right)$
6:	Find the first m such that $|W_{(j,m+1)}^{(1)}|\le w_m\le |W_{(j,m)}^{(1)}|$
7:	end for
8:	${\tilde{\theta }}_j\leftarrow \frac{1}{M}\cdot \mathrm{sign}\left({\theta }_j\right)\cdot w_m$
9:	${\tilde{W}}_j^{(1)}\leftarrow \mathrm{sign}\left(W_j^{(1)}\right)\cdot \min \left(w_m,W_j^{(1)}\right)$
10:	end for
11:	return $\left(\tilde{\theta },{\tilde{W}}^{(1)}\right)$
12:	end procedure
13:	Notation: d denotes the number of features; K denotes the size of the first hidden layer.
14:	Conventions: Ln. 6, $W_{(j,K+1)}^{(1)}=0$, $W_{(j,0)}^{(1)}=+\infty$; Ln. 9, minimum is applied coordinate-wise.