Algorithm 1: Proposed channel estimator

1 Obtain the initial channel estimate $\mathbf{H}\leftarrow \widehat{\mathbf{h}}=\left[{\widehat{\mathbf{h}}}_1,\cdots ,{\widehat{\mathbf{h}}}_{N_r}\right]$ from (3) 2 Initialize the state ${\mathsf{S}}_1=\left({\mathbf{X}}^p,{\mathbf{X}}^p,\varphi \right)$. 3 for$d=1$ to $M_d$ do$\phantom{(}$ 4   for$n\in {\mathcal{M}}_d$do$\phantom{(}$ 5    Compute the optimal policy $a^{★}={\pi }^{★}\left({\mathsf{S}}_n\right)$ from (26). 6    Set the optimal values $j^{★}=0$ for $a^{★}=0$ and ${\mathbf{x}}_{j^{★}}=\widehat{\mathbf{x}}\left[n\right]$ for $a^{★}=1$. 7    Update ${\mathsf{S}}_{n+1}\leftarrow {\mathsf{U}}_{n+1}^{(a^{★},j^{★})}\left({\mathsf{S}}_n\right)$ from (12). 8    if$a^{★}==1$ and $N_w<\left|\mathcal{C}\right|$then$\phantom{(}$ 9      Remove the state elements in ${\mathsf{S}}_{n+1}$. 10      ${\mathbf{X}}_{n+1}\leftarrow {\mathbf{X}}_{n+1}\setminus {\mathbf{X}}_{n+1}\left[\mathcal{C}\left(1\right)\right]$, 11      ${\widehat{\mathbf{X}}}_{n+1}\leftarrow {\widehat{\mathbf{X}}}_{n+1}\setminus {\widehat{\mathbf{X}}}_n\left[\mathcal{C}\left(1\right)\right]$, 12      $\mathcal{C}\leftarrow \mathcal{C}\setminus \mathcal{C}\left(1\right)$. 13    end$\phantom{(}$ 14    Refine the state elements in ${\mathbf{X}}_{n+1}\leftarrow {\sqrt{1-{ϵ}^2}}^{-1}{\mathbf{X}}_{n+1}$ and      ${\widehat{\mathbf{X}}}_{n+1}\leftarrow {\sqrt{1-{ϵ}^2}}^{-1}{\widehat{\mathbf{X}}}_{n+1}$. 15  end$\phantom{(}$ 16  Update the channel estimate $\mathbf{H}\leftarrow \widehat{\mathbf{h}}=\left[{\widehat{\mathbf{h}}}_1\left({\mathsf{S}}_n\right),\cdots ,{\widehat{\mathbf{h}}}_{N_r}\left({\mathsf{S}}_n\right)\right]$ from (10). 17 end$\phantom{(}$