Algorithm 1 OGA + HDIC + Trim

Require: response vector ${\mathit{y}}_n\in {\mathbb{R}}^n$, regressor matrix ${\mathit{X}}_n\in {\mathbb{R}}^{n\times r_n}$.  Initialzation: set $d=0$, ${\mathit{r}}^{\left(0\right)}={\mathit{y}}_n$ and ${\widehat{J}}_0=\varnothing$.  While $d=0,\dots ,D_n$ do     Compute ${\widehat{j}}_{d+1}=arg{max}_{1\le j\le r_n,j\notin {\widehat{J}}_{d-1}}|{\mathit{z}}_j^{\top }{\mathit{r}}^{\left(d\right)}|/(n^{1/2}\parallel {\mathit{z}}_j\parallel )$ and update ${\widehat{J}}_{d+1}={\widehat{J}}_d\bigcup {\widehat{j}}_{d+1}$;     Compute ${\mathit{z}}_{{\widehat{j}}_{d+1}}^{\perp }$ via ${\mathit{z}}_{{\widehat{j}}_{d+1}}^{\perp }=({\mathit{I}}_n-{\sum }_{\ell =1}^d{\mathit{H}}_{{\widehat{j}}_{\ell }}^{\perp }){\mathit{z}}_{{\widehat{j}}_{d+1}}$, where ${\mathit{H}}_{{\widehat{j}}_{\ell }}^{\perp }={\mathit{z}}_{{\widehat{j}}_{\ell }}{\mathit{z}}_{{\widehat{j}}_{\ell }}^{\top }/{\parallel {\mathit{z}}_{{\widehat{j}}_{\ell }}\parallel }^2$;     Compute ${\mathit{r}}^{(d+1)}$ via ${\mathit{r}}^{(d+1)}=({\mathit{I}}_n-{\mathit{H}}_{{\widehat{j}}_{d+1}}^{\perp }){\mathit{r}}^{\left(d\right)}$.  end     Compute the minimum of HDIC via $$\begin{array}{cc}\hfill \mathrm{HDIC}\left({\widehat{J}}_d\right)& =log{\widehat{\sigma }}_{{\widehat{J}}_d}^2+♯\left({\widehat{J}}_d\right)c_{n}log\left(r_n\right)/n,\hfill \\ \hfill {\widehat{d}}_n& =arg\underset{1\le d\le D_n}{min}\mathrm{HDIC}\left({\widehat{J}}_d\right).\hfill \end{array}$$ (5)  return ${\widehat{\mathcal{J}}}_n$ via $${\widehat{\mathcal{J}}}_n=\left\{\begin{array}{cc}\{{\widehat{j}}_{\ell }:\mathrm{HDIC}\left({\widehat{J}}_d-\left\{{\widehat{j}}_{\ell }\right\}\right)>\mathrm{HDIC}\left({\widehat{J}}_d\right),1\le \ell \le {\widehat{d}}_n\},\hfill & \mathrm{if}{\widehat{d}}_n>1;\hfill \\ \left\{{\widehat{j}}_1\right\},\hfill & \mathrm{if}{\widehat{d}}_n=1.\hfill \end{array}\right.$$