The Noise Collector for sparse recovery in high dimensions

Significance

The ability to detect sparse signals from noisy, high-dimensional data is a top priority in modern science and engineering. For optimal results, current approaches need to tune parameters that depend on the level of noise, which is often difficult to estimate. We develop a parameter-free, computationally efficient, ${\ell }_1$-norm minimization approach that has a zero false discovery rate (no false positives) with high probability for any level of noise while it detects the exact location of sparse signals when the noise is not too large.

Abstract

The ability to detect sparse signals from noisy, high-dimensional data is a top priority in modern science and engineering. It is well known that a sparse solution of the linear system $\mathcal{A}\mathit{\rho }={\mathit{b}}_0$ can be found efficiently with an ${\ell }_1$-norm minimization approach if the data are noiseless. However, detection of the signal from data corrupted by noise is still a challenging problem as the solution depends, in general, on a regularization parameter with optimal value that is not easy to choose. We propose an efficient approach that does not require any parameter estimation. We introduce a no-phantom weight $\tau$ and the Noise Collector matrix $\mathcal{C}$ and solve an augmented system $\mathcal{A}\mathit{\rho }+\mathcal{C}\mathit{\eta }={\mathit{b}}_0+\mathit{e}$, where $\mathit{e}$ is the noise. We show that the ${\ell }_1$-norm minimal solution of this system has zero false discovery rate for any level of noise, with probability that tends to one as the dimension of ${\mathit{b}}_0$ increases to infinity. We obtain exact support recovery if the noise is not too large and develop a fast Noise Collector algorithm, which makes the computational cost of solving the augmented system comparable with that of the original one. We demonstrate the effectiveness of the method in applications to passive array imaging.

We want to find sparse solutions $\mathit{\rho }\in {\mathbb{R}}^K$ for

$$\mathcal{A}\hspace{0.17em}\mathit{\rho }=\mathit{b}$$ [1]

from highly incomplete measurement data $\mathit{b}={\mathit{b}}_0+\mathit{e}\in {\mathbb{R}}^N$ corrupted by noise $\mathit{e}$, where $1\ll N<K$. In the noiseless case, $\mathit{\rho }$ can be found exactly by solving the optimization problem (1)

$${\mathit{\rho }}_{*}=\arg \underset{\mathit{\rho }}{min}{\Vert \mathit{\rho }\Vert }_{{\ell }_1},\text{subject to}\mathcal{A}\hspace{0.17em}\mathit{\rho }=\mathit{b},$$ [2]

provided that the measurement matrix $\mathcal{A}\in {\mathbb{R}}^{N\times K}$ satisfies additional conditions (e.g., decoherence or restricted isometry properties) (2, 3) and that the solution vector $\mathit{\rho }$ has a small number $M$ of nonzero components or degrees of freedom. When measurements are noisy, exact recovery is no longer possible. However, the exact support of $\mathit{\rho }$ can still be determined if the noise is not too strong. The most commonly used approach is to solve the ${\ell }_2$-relaxed form of Eq. 2:

$${\mathit{\rho }}_{\lambda }=\arg \underset{\mathit{\rho }}{min}\left(\lambda {\Vert \mathit{\rho }\Vert }_{{\ell }_1}+{\Vert \mathcal{A}\mathit{\rho }-\mathit{b}\Vert }_{{\ell }_2}^2\right),$$ [3]

which is known as Lasso in the statistics literature (4). There are sufficient conditions for the support of ${\mathit{\rho }}_{\lambda }$ to be contained within the true support [e.g., the works of Fuchs (5), Tropp (6), Wainwright (7), and Maleki et al. (8)]. These conditions depend on the signal-to-noise ratio (SNR), which is not known and must be estimated, and on the regularization parameter $\lambda$, which must be carefully chosen and/or adaptively changed (9). Although such an adaptive procedure improves the outcome, the resulting solutions tend to include a large number of “false positives” in practice (10). Belloni et al. (11) proposed to solve the square root Lasso minimization problem instead of Eq. 3, which makes the regularization parameter $\lambda$ independent of the SNR. Our contribution is a computationally efficient method for exact support recovery with no false positives in noisy settings. It also does not require an estimate on SNR.

Main Results

Suppose that $\mathit{\rho }$ is an $M$-sparse solution of system [1] with no noise, where the columns of $\mathcal{A}$ have unit length. Our main result ensures that we can still recover the support of $\mathit{\rho }$ when the data are noisy by looking at the support of ${\mathit{\rho }}_{\tau }$ found as

$$\begin{array}{l}\hfill \left({\mathit{\rho }}_{\tau },{\mathit{\eta }}_{\tau }\right)=\arg \underset{\mathit{\rho },\mathit{\eta }}{min}\left(\tau {\Vert \mathit{\rho }\Vert }_{{\ell }_1}+{\Vert \mathit{\eta }\Vert }_{{\ell }_1}\right),\\ \hfill \text{subject to}\mathcal{A}\mathit{\rho }+\mathcal{C}\mathit{\eta }={\mathit{b}}_0+\mathit{e},\end{array}$$ [4]

with an $O\left(1\right)$ weight $\tau$ and an appropriately chosen Noise Collector matrix $\mathcal{C}\in {\mathbb{R}}^{N\times \mathrm{\Sigma }}$, $\mathrm{\Sigma }\gg K$. The minimization problem [4] can be understood as a relaxation of [2] as it works by absorbing all of the noise and possibly, some signal in $\mathcal{C}{\mathit{\eta }}_{\tau }$.

The following theorem shows that, if the signal is pure noise and the columns of $\mathcal{C}$ are chosen independently and at random on the unit sphere ${\mathbb{S}}^{N-1}=\left\{x\in {\mathbb{R}}^N,{\Vert x\Vert }_{{\ell }_2}=1\right\}$, then $\mathcal{C}{\mathit{\eta }}_{\tau }=\mathit{e}$ for any level of noise, with large probability.

Theorem 1 (No-Phantom Signal). Suppose that ${\mathit{b}}_0=0$ and that $\mathit{e}/{\Vert \mathit{e}\Vert }_{{\ell }_2}$ is uniformly distributed on ${\mathbb{S}}^{N-1}$. Fix $\beta >1$, and draw $\mathrm{\Sigma }=N^{\beta }$ columns for $\mathcal{C}$ independently from the uniform distribution on ${\mathbb{S}}^{N-1}$. For any $\kappa >0$, there are constants $\tau =\tau \left(\kappa ,\beta \right)$ and $N_0=N_0\left(\kappa ,\beta \right)$ such that, for all $N>N_0$, ${\mathit{\rho }}_{\tau }$, the solution of Eq. 4, is 0 with probability $1-1/N^{\kappa }$.

This theorem guarantees with large probability a zero false discovery rate in the absence of signals with meaningful information. The key to a zero false discovery rate is the choice of a no-phantom weight $\tau$. Next, we generalize this result for the case in which the recorded signals carry useful information.

Theorem 2 (Zero False Discoveries). Let $\mathit{\rho }$ be an $M$-sparse solution of the noiseless system $\mathcal{A}\mathit{\rho }={\mathit{b}}_0$. Assume that $\kappa$, $\beta$, the Noise Collector, and the noise are the same as in Theorem 1. In addition, assume that the columns of $\mathcal{A}$ are incoherent in the sense that $\left|⟨{\mathit{a}}_i,{\mathit{a}}_j⟩\right|\le \frac{1}{3M}$. Then, there are constants $\tau =\tau \left(\kappa ,\beta \right)$ and $N_0=N_0\left(\kappa ,\beta \right)$ such that $\text{supp}\left({\mathit{\rho }}_{\tau }\right)\subseteq \text{supp}\left(\mathit{\rho }\right)$ for all $N>N_0$ with probability $1-1/N^{\kappa }$.

This theorem holds for any level of noise and the same value of $\tau$ as in Theorem 1. The incoherence conditions in Theorem 2 are needed to guarantee that the true signal does not create false positives elsewhere. Theorem 2 guarantees that the support of ${\mathit{\rho }}_{\tau }$ is inside the support of $\mathit{\rho }$. The next theorem shows that, if the noise is not too large, then ${\mathit{\rho }}_{\tau }$ and $\mathit{\rho }$ have exactly the same support.

Theorem 3 (Exact Support Recovery). Keep the same assumptions as in Theorem 2. Let $\gamma =\underset{i\in \text{supp}\left(\mathit{\rho }\right)}{min}\left|{\rho }_i\right|/{\Vert \rho \Vert }_{{\ell }_{\infty }}$. There are constants $\tau =\tau \left(\kappa ,\beta \right)$, $c_1=c_1\left(\kappa ,\beta ,\gamma \right)$, and $N_0=N_0\left(\kappa ,\beta \right)$ such that, if the noise level satisfies ${\Vert \mathit{e}\Vert }_{{\ell }_2}\le c_1{\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}^2{\Vert \mathit{\rho }\Vert }_{{\ell }_1}^{-1}\sqrt{N}/\sqrt{\ln N}$, then for all $N>N_0$, $\text{supp}\left({\mathit{\rho }}_{\tau }\right)=\text{supp}\left(\mathit{\rho }\right)$ with probability $1-1/N^{\kappa }$.

To elucidate an interpretation of the last theorem, consider a model case where $\mathcal{A}$ is the identity matrix and all coefficients of ${\mathit{b}}_0=\mathit{\rho }$ are either one or zero. Then, ${\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}^2={\Vert \mathit{\rho }\Vert }_{{\ell }_1}=M$. In this case, an acceptable relative level of noise is

$${\Vert \mathit{e}\Vert }_{{\ell }_2}/{\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}\lesssim \sqrt{N}/\sqrt{M\ln N}.$$ [5]

This means that ${\Vert \mathit{e}\Vert }_{{\ell }_2}\lesssim \sqrt{N}/\sqrt{\ln N}$, and it implies that each coefficient of ${\mathit{b}}_0$ may be corrupted by $O\left(1/\sqrt{\ln N}\right)$ on average and that some coefficients of ${\mathit{b}}_0$ may be corrupted by $O\left(1\right)$.

Motivation

We are interested in imaging sparse scenes accurately using limited and noisy data. Such imaging problems arise in many areas, such as medical imaging (12), structural biology (13), radar (14), and geophysics (15). In imaging, the ${\ell }_1$-norm minimization method in Eq. 2 is often used (16–21) as it has the desirable property of superresolution: that is, the enhancement of the fine-scale details of the images. This has been analyzed in different settings by Donoho (22), Candès and Fernandez-Granda (23), Fannjiang and Liao (24), and Borcea and Kocyigit (25) among others. We want to retain this property in our method when the data are corrupted by additive noise.

However, noise fundamentally limits the quality of the images formed with almost all computational imaging techniques. Specifically, ${\ell }_1$-norm minimization produces images that are unstable for low SNR due to the ill conditioning of superresolution reconstruction schemes. The instability emerges as clutter noise in the image, or grass, that degrades the resolution. Our initial motivation to introduce the Noise Collector matrix $\mathcal{C}$ was to regularize the matrix $\mathcal{A}$ and thus, to suppress the clutter in the images. We proposed in ref. 26 to seek the minimal ${\ell }_1$-norm solution of the augmented linear system $\mathcal{A}\mathit{\rho }+\mathcal{C}\mathit{\eta }=\mathit{b}$. The idea was to choose the columns of $\mathcal{C}$ almost orthogonal to those of $\mathcal{A}$. Indeed, the condition number of $\left[\mathcal{A}\hspace{0.17em}|\hspace{0.17em}\mathcal{C}\right]$ becomes $O\left(1\right)$ when $O\left(N\right)$ columns of $\mathcal{C}$ are taken at random. This essentially follows from the bounds on the largest and the smallest nonzero singular values of random matrices (theorem 4.6.1 in ref. 27).

The idea to create a dictionary for noise is not new. For example, the work by Laska et al. (28) considers a specific version of the measurement noise model so that $\mathit{b}=\mathcal{A}\mathit{\rho }+\mathcal{C}\mathit{e}$, where $\mathcal{C}$ is a matrix with fewer (orthonormal) columns than rows and the noise vector $\mathit{e}$ is sparse. $\mathcal{C}$ represents the basis in which the noise is sparse and it is assumed to be known. Then, they show that it is possible to recover sparse signals and sparse noise exactly. We stress that we do not assume here that the noise is sparse. In our work, the noise is large (SNR can be small) and is evenly distributed across the data, and therefore, it cannot be sparsely accommodated.

To suppress the clutter, our theory in ref. 26 required exponentially many columns, and therefore, $\mathrm{\Sigma }\lesssim e^N$. This seemed to make the Noise Collector impractical, but the numerical experiments suggested that $O\left(N\right)$ columns were enough to obtain excellent results. We address this issue here and explain why the Noise Collector matrix $\mathcal{C}$ only needs algebraically many columns. Moreover, to absorb the noise completely and thus, improve the algorithm in ref. 26, we introduce now the no-phantom weight $\tau$ in Eq. 4. Indeed, by weighting the columns of the Noise Collector matrix $\mathcal{C}$ with respect to those in the model matrix $\mathcal{A}$, the algorithm now produces images with no clutter at all regardless of how much noise is added to the data.

Finally, we want the Noise Collector to be efficient, with almost no extra computational cost with respect to the Lasso problem in Eq. 3. To this end, the Noise Collector is constructed using circulant matrices that allow for efficient matrix vector multiplications using fast Fourier transforms (FFTs).

We now explain how the Noise Collector works and reduce our theorems to basic estimates in high-dimensional probability.

The Noise Collector

The method has two main ingredients: the Noise Collector matrix $\mathcal{C}$ and the no-phantom weight $\tau$. The construction of the Noise Collector matrix $\mathcal{C}$ starts with the following three key properties. First, its columns should be sufficiently orthogonal to the columns of $\mathcal{A}$, and therefore, it does not absorb signals with “meaningful” information. Second, the columns of $\mathcal{C}$ should be uniformly distributed on the unit sphere ${\mathbb{S}}^{N-1}$ so that we could approximate well a typical noise vector. Third, the number of columns in $\mathcal{C}$ should grow slower than exponential with $N$; otherwise, the method is impractical.

One way to guarantee all three properties is to impose

$$\left|⟨{\mathit{a}}_i,{\mathit{c}}_j⟩\right|<\frac{\alpha }{\sqrt{N}}\hspace{0.17em}\forall i,j\hspace{0.17em}\text{and}\left|⟨{\mathit{c}}_i,{\mathit{c}}_j⟩\right|<\frac{\alpha }{\sqrt{N}}\hspace{0.17em}\forall i\hspace{0.17em}\ne \hspace{0.17em}j$$ [6]

with $\alpha >1$ and fill out $\mathcal{C}$ drawing ${\mathit{c}}_i$ at random with rejections until the rejection rate becomes too high. Then, by construction, the columns of $\mathcal{C}$ are almost orthogonal to the columns of $\mathcal{A}$, and when the rejection rate becomes too high, this implies that we cannot pack more N-dimensional unit vectors into $\mathcal{C}$; thus, we can approximate well a typical noise vector. Finally, the Kabatjanskii–Levenstein inequality (discussed in ref. 29) implies that the number $\mathrm{\Sigma }$ of columns in $\mathcal{C}$ grows at most polynomially: $\mathrm{\Sigma }\le N^{{\alpha }^2}$. The first estimate in Eq. 6 implies that any solution $\mathcal{C}\mathit{\eta }={\mathit{a}}_i$ satisfies, for any $i\le K$, ${\Vert \mathit{\eta }\Vert }_{{\ell }_1}\gtrsim \sqrt{N}$. This estimate measures how expensive it is to approximate columns of $\mathcal{A}$ (i.e., the meaningful signal) with the Noise Collector. In turn, the no-phantom weight $\tau$ should be chosen so that it is expensive to approximate noise using columns of $\mathcal{A}$. It cannot be taken too large, however, because we may lose the signal. In fact, one can prove that, if $\tau \ge \sqrt{N}/\alpha$, then ${\mathit{\rho }}_{\tau }\equiv 0$ for any $\mathit{\rho }$ and any level of noise. Intuitively, $\tau$ characterizes the rate at which the signal is lost as the noise increases. The most important property of the no-phantom weight $\tau$ is that it does not depend on the level of noise, and therefore, it is chosen before we start using the Noise Collector.

It is, however, more convenient for the proofs to use a probabilistic version of Eq. 6. Suppose that the columns of $\mathcal{C}$ are drawn independently at random. Then, the dot product of any two random unit vectors is still typically of order $1/\sqrt{N}$ (27). If the number of columns grows polynomially, we only have to sacrifice an asymptotically negligible event where our Noise Collector does not satisfy the three key properties, and the decoherence constraints in Eq. 6 are weakened by a logarithmic factor only. This follows from basic estimates in high-dimensional probability. We will state them in the next lemma after we interpret problem [4] geometrically.

Consider the convex hulls

$$H_1=\left\{x\in {\mathbb{R}}^N\left|x=\sum _{i=1}^{\mathrm{\Sigma }}{\xi }_i{\mathit{c}}_i,\hspace{0.17em}\sum _{i=1}^{\mathrm{\Sigma }}\left|{\xi }_i\right|\le 1\right.\right\},$$ [7]

$$H_2=\left\{x\in {\mathbb{R}}^N\left|x=\sum _{i=1}^K{\xi }_i{\mathit{a}}_i,\hspace{0.17em}\sum _{i=1}^{\mathrm{\Sigma }}\left|{\xi }_i\right|\le 1\right.\right\},$$ [8]

and $H\left(\tau \right)=\left\{\xi h_1/\tau +\left(1-\xi \right)h_2,0\le \xi \le 1,h_i\in H_i\right\}$. Theorem 1 states that, for a typical noise vector $\mathit{e}$, we can find ${\lambda }_0>0$ such that $\mathit{e}\in {\lambda }_0\partial H_1$ and $\mathit{e}\notin \lambda \partial H\left(\tau \right)$ for any $\lambda <{\lambda }_0$.

Lemma 1 (Typical Width of Convex Hulls $H_i$). Suppose that $\mathrm{\Sigma }=N^{\beta }$, $\beta >1$; vectors ${\mathit{c}}_i\in {\mathbb{S}}^{N-1}$, $i=\mathrm{1,2},\dots ,\mathrm{\Sigma }$, are drawn at random and independently; and $\mathit{e}\in {\mathbb{S}}^{N-1}$. Then, for any $\kappa >0$, there are constants $c_0=c_0\left(\kappa ,\beta \right)$, $\alpha =\sqrt{\left(\beta -1\right)/2}$, and $N_0=N_0\left(\kappa ,\beta \right)$ such that, for all $N\ge N_0$,

$$max\left(\underset{i\le K}{max}\left(\left|⟨{\mathit{a}}_i,\mathit{e}⟩\right|\right),\underset{i\le \mathrm{\Sigma }}{max}\left(\left|⟨{\mathit{c}}_i,\mathit{e}⟩\right|\right)\right)<c_0\sqrt{\ln N}/\sqrt{N}$$ [9]

and

$$\alpha \sqrt{\ln N}\mathit{e}/\sqrt{N}\in H_1$$ [10]

with probability $1-1/N^{\kappa }$.

We sketch the proof of estimates [9] and [10] in Proofs. Estimate [9] can also be derived from Milman’s version (30) of Dvoretzky’s theorem. Informally, inequality [9] states that $H_1$ and $H_2$ are contained in the ${\ell }_2$ ball of radius $c_0\sqrt{\ln N}/\sqrt{N}$ except for a few spikes in statistically insignificant directions (Fig. 1, Left). Inequality [10] states that $H_1$ contains an ${\ell }_2$ ball of radius $\alpha \sqrt{\ln N}/\sqrt{N}$ except for a few statistically insignificant directions.

Fig. 1.

(Left) A convex hull $H_1$ is an ${\ell }_2$ ball of radius $O\left(\sqrt{\ln N}/\sqrt{N}\right)$ with few spikes. (Right) An intersection of $H\left(\tau \right)$ with the span $\left({\mathit{a}}_1,\mathit{e}\right)$ is a rounded rhombus.

These inequalities immediately imply Theorem 1. We just need to explain how to choose the no-phantom weight $\tau$. There will be no phantoms if $H_2/\tau$ is strictly inside the ${\ell }_2$ ball of radius $\alpha \sqrt{\ln N}/\sqrt{N}$. This could be done if $\tau >c_0/\alpha$.

If columns of $\mathcal{A}$ are orthogonal to each other, then Theorem 2 follows from Theorem 1. We just need to project the linear system in Eq. 4 on the span of ${\mathit{a}}_i$, $i\notin \text{supp}\left(\mathit{\rho }\right)$, and apply Theorem 1 to the projections. If, in addition, we assume that ${\mathit{b}}_0={\mathit{a}}_1{\rho }_1$, then Proof of Theorem 3 is illustrated in Fig. 1, Right. In detail, a typical intersection of $V=\text{span}\left({\mathit{a}}_1,\mathit{e}\right)$, and $H\left(\tau \right)$ is a rounded rhombus because it is the convex hull of ${\mathit{a}}_1/\tau$ and the ${\ell }_2$ ball of radius $c_0\sqrt{\ln N}/\sqrt{N}$. If ${\mathit{a}}_1{\rho }_1+\mathit{e}\in {\lambda }_0\partial H\left(\tau \right)$, then there are two options: 1) ${\mathit{a}}_1{\rho }_1+\mathit{e}$ lies on the curved boundary of the rounded rhombus, and then, $\text{supp}\left({\mathit{\rho }}_{\tau }\right)=\varnothing$; or 2) ${\mathit{a}}_1{\rho }_1+\mathit{e}$ lies on the flat boundary of the rounded rhombus, and then, $\text{supp}\left({\mathit{\rho }}_{\tau }\right)=\text{supp}\left(\mathit{\rho }\right)$. The second option happens if the vector ${\mathit{a}}_1{\rho }_1+\mathit{e}$ intersects the flat boundary of $\partial H\left(\tau \right)$. This gives the support recovery estimate in Theorem 3.

In the general case, the columns of the combined matrix $\left[\mathcal{A}\hspace{0.17em}|\hspace{0.17em}\mathcal{C}\right]$ are incoherent. This property allows us to prove Theorems 2 and 3 in Proofs using known techniques (26). In particular, we automatically have exact recovery using ref. 2 applied to $\left[\mathcal{A}\hspace{0.17em}|\hspace{0.17em}\mathcal{C}\right]$ if the data are noiseless.

Lemma 2 (Exact Recovery). Suppose that $\mathit{\rho }$ is an $M$-sparse solution of $\mathcal{A}\mathit{\rho }=\mathit{b}$ and that there is no noise so that $\mathit{e}=0$. In addition, assume that the columns of $\mathcal{A}$ are incoherent: $\left|⟨{\mathit{a}}_i,{\mathit{a}}_j⟩\right|\le \frac{1}{3M}$. Then, the solution to Eq. 4 satisfies ${\mathit{\rho }}_{\tau }=\mathit{\rho }$ for all

$$M<\frac{2\sqrt{N}}{3c_0\tau \sqrt{\ln N}}\hspace{0.17em}\mathit{\text{with probability}}1-\frac{1}{N^{\kappa }}.$$ [11]

Fast Noise Collector Algorithm

To find the minimizer in Eq. 4, we consider a variational approach. We define the function

$$\begin{array}{ll}\hfill & F\left(\mathit{\rho },\mathit{\eta },\mathit{z}\right)=\lambda \hspace{0.17em}\left(\tau {\Vert \mathit{\rho }\Vert }_{{\ell }_1}+{\Vert \mathit{\eta }\Vert }_{{\ell }_1}\right)\hfill \\ \hfill & +\frac{1}{2}{\Vert \mathcal{A}\mathit{\rho }+\mathcal{C}\mathit{\eta }-\mathit{b}\Vert }_{{\ell }_2}^2+⟨\mathit{z},\mathit{b}-\mathcal{A}\mathit{\rho }-\mathcal{C}\mathit{\eta }⟩\hfill \end{array}$$ [12]

for a no-phantom weight $\tau$ and determine the solution as

$$\underset{\mathit{z}}{max}\underset{\mathit{\rho },\mathit{\eta }}{min}F\left(\mathit{\rho },\mathit{\eta },\mathit{z}\right).$$ [13]

The key observation is that this variational principle finds the minimum in Eq. 4 exactly for all values of the regularization parameter $\lambda$. Hence, the method has no tuning parameters. To determine the exact extremum in Eq. 13, we use the iterative soft thresholding algorithm GeLMA (generalized Lagrangian multiplier algorithm) (31) that works as follows.

First, pick a value for $\beta$ and $\tau$. For optimal results, one can calibrate $\tau$ to be the smallest constant such that Theorem 1 holds: that is, we see no-phantom signals when the algorithm is fed with pure noise. In our numerical experiments, we use $\beta =1.5$ and $\tau =2$.

Second, pick a value for the regularization parameter $\lambda$ (e.g., $\lambda =1$). Choose step sizes $\mathrm{\Delta }{t}_1<2/{\Vert \left[\mathcal{A}\hspace{0.17em}|\hspace{0.17em}\mathcal{C}\right]\Vert }^2$ and $\mathrm{\Delta }{t}_2<\lambda /\Vert \mathcal{A}\Vert$.* Set ${\mathit{\rho }}_0=\mathbf{0}$, ${\mathit{\eta }}_0=\mathbf{0}$, and ${\mathit{z}}_{\mathbf{0}}=\mathbf{0}$, and iterate for $k\ge 0$:

$$\begin{array}{ll}\hfill & \mathit{r}=\mathit{b}-\mathcal{A}\hspace{0.17em}{\mathit{\rho }}_k-\mathcal{C}\hspace{0.17em}{\mathit{\eta }}_k,\hfill \\ \hfill & {\mathit{\rho }}_{k+1}={\mathcal{S}}_{\hspace{0.17em}\tau \hspace{0.17em}\lambda \mathrm{\Delta }{t}_1}\left({\mathit{\rho }}_k+\mathrm{\Delta }{t}_1\hspace{0.17em}{\mathcal{A}}^{*}\left({\mathit{z}}_k+\mathit{r}\right)\right),\hfill \\ \hfill & {\mathit{\eta }}_{k+1}={\mathcal{S}}_{\lambda \mathrm{\Delta }{t}_1}\left({\mathit{\eta }}_k+\mathrm{\Delta }{t}_1\hspace{0.17em}{\mathcal{C}}^{*}\left({\mathit{z}}_k+\mathit{r}\right)\right),\hfill \\ \hfill & {\mathit{z}}_{k+1}={\mathit{z}}_k+\mathrm{\Delta }{t}_2\hspace{0.17em}\mathit{r},\hfill \end{array}$$ [14]

where ${\mathcal{S}}_r\left(y_i\right)=\text{sign}\left(y_i\right)max\left\{0,\left|y_i\right|-r\right\}$.

The Noise Collector matrix $\mathcal{C}$ is computed by drawing $N^{\beta -1}$ normally distributed $N$-dimensional vectors normalized to unit length. These are the generating vectors of the Noise Collector. From each of them, a circulant $N\times N$ matrix ${\mathcal{C}}_i$, $i=1,\dots ,N^{\beta -1}$, is constructed. The Noise Collector matrix is obtained by concatenation, and therefore, $\mathcal{C}=\left[{\mathcal{C}}_1\left|{\mathcal{C}}_2\left|\dots \left|{\mathcal{C}}_{N^{\beta -1}}\right.\right.\right.\right]$. Exploiting the circulant structure of the matrices ${\mathcal{C}}_i$, we perform the matrix vector multiplications $\mathcal{C}{\mathit{\eta }}_k$ and ${\mathcal{C}}^{*}\left({\mathit{z}}_k+\mathit{r}\right)$ in Eq. 14 using the FFT (32). This makes the complexity associated with the Noise Collector $O\left(N^{\beta }\log \left(N\right)\right)$. Note that only the $N^{\beta -1}$ generating vectors are stored and not the entire $N\times N^{\beta }$ Noise Collector matrix. In practice, we use $\beta \approx 1.5$, which makes the cost of using the Noise Collector negligible as typically, $K\gg N^{\beta -1}$. The columns of the Noise Collector matrix $\mathcal{C}$ with this circulant structure are uniformly distributed on ${\mathbb{S}}^{N-1}$, and they satisfy Lemma 1. This implies that the theorems of this paper are still valid for such $\mathcal{C}$.

Application to Imaging

We consider passive array imaging of point sources. The problem consists of determining the positions ${\overrightarrow{\mathit{z}}}_j$ and the complex^† amplitudes ${\alpha }_j$, $j=1,\dots ,M$, of a few point sources from measurements of polychromatic signals on an array of receivers (Fig. 2). The imaging system is characterized by the array aperture $a$, the distance $L$ to the sources, the bandwidth $B$, and the central wavelength ${\lambda }_0$.

Fig. 2.

General setup for passive array imaging. The source at ${\overrightarrow{\mathit{z}}}_j$ emits a signal that is recorded at all array elements ${\overrightarrow{\mathit{x}}}_r$, $r=1,\dots ,N_r$.

The sources are located inside an image window (IW), which is discretized with a uniform grid of points ${\overrightarrow{\mathit{y}}}_k$, $k=1,\dots ,K$. The unknown is the source vector $\mathit{\rho }={\left[{\rho }_1,\dots ,{\rho }_K\right]}^{⊺}\in {\mathbb{C}}^K$, with components ${\rho }_k$ that correspond to the complex amplitudes of the $M$ sources at the grid points ${\overrightarrow{\mathit{y}}}_k$, $k=1,\dots ,K$, with $K\gg M$. For the true source vector, we have ${\rho }_k={\alpha }_j$ if ${\overrightarrow{\mathit{y}}}_k={\overrightarrow{\mathit{z}}}_j$ for some $j=1,\dots ,M$, while ${\rho }_k=0$ otherwise.

Denoting by $G\left(\overrightarrow{\mathit{x}},\overrightarrow{\mathit{y}};\omega \right)$ Green’s function for the propagation of a signal of angular frequency $\omega$ from point $\overrightarrow{\mathit{y}}$ to point $\overrightarrow{\mathit{x}}$, we define the single-frequency Green’s function vector that connects a point $\overrightarrow{\mathit{y}}$ in the IW with all points ${\overrightarrow{\mathit{x}}}_r$, $r=1,\dots ,N_r$, on the array as

$$\mathit{g}\left(\overrightarrow{\mathit{y}};\omega \right)={\left[G\left({\overrightarrow{\mathit{x}}}_1,\overrightarrow{\mathit{y}};\omega \right),G\left({\overrightarrow{\mathit{x}}}_2,\overrightarrow{\mathit{y}};\omega \right),\dots ,G\left({\overrightarrow{\mathit{x}}}_N,\overrightarrow{\mathit{y}};\omega \right)\right]}^{⊺}\in {\mathbb{C}}^{N_r}.$$

In three dimensions, $G\left(\overrightarrow{\mathit{x}},\overrightarrow{\mathit{y}};\omega \right)=\frac{\exp \left\{\mathrm{i}\omega |\overrightarrow{\mathit{x}}-\overrightarrow{\mathit{y}}|/c_0\right\}}{4\pi |\overrightarrow{\mathit{x}}-\overrightarrow{\mathit{y}}|}$ if the medium is homogeneous. The data for the imaging problem are the signals $b\left({\overrightarrow{\mathit{x}}}_r,{\omega }_l\right)={\sum }_{j=1}^M{\alpha }_{j}G\left({\overrightarrow{\mathit{x}}}_r,{\overrightarrow{\mathit{z}}}_j;{\omega }_l\right)$ recorded at receiver locations ${\overrightarrow{\mathit{x}}}_r$, $r=1,\dots ,N_r$, at frequencies ${\omega }_l$, $l=1,\dots ,S$. These data are stacked in a column vector

$$\mathit{b}={\left[\mathit{b}{\left({\omega }_1\right)}^{⊺},\mathit{b}{\left({\omega }_2\right)}^{⊺},\dots ,\mathit{b}{\left({\omega }_S\right)}^{⊺}\right]}^{⊺}\in {\mathbb{C}}^N;N=N_{r}S,$$ [15]

with $\mathit{b}\left({\omega }_l\right)={\left[b\left({\overrightarrow{\mathit{x}}}_1,{\omega }_l\right),b\left({\overrightarrow{\mathit{x}}}_2,{\omega }_l\right),\dots ,b\left({\overrightarrow{\mathit{x}}}_N,{\omega }_l\right)\right]}^{⊺}\in {\mathbb{C}}^{N_r}$. Then, $\mathcal{A}\hspace{0.17em}\mathit{\rho }=\mathit{b}$, with $\mathcal{A}$ being the $N\times K$ measurement matrix with columns ${\mathit{a}}_k$ that are the multiple-frequency Green’s function vectors

$${\mathit{a}}_k={\left[\mathit{g}{\left({\overrightarrow{\mathit{y}}}_k;{\omega }_1\right)}^{⊺},\mathit{g}{\left({\overrightarrow{\mathit{y}}}_k;{\omega }_2\right)}^{⊺},\dots ,\mathit{g}{\left({\overrightarrow{\mathit{y}}}_k;{\omega }_S\right)}^{⊺}\right]}^{⊺}\in {\mathbb{C}}^N$$ [16]

normalized to have length 1. The system $\mathcal{A}\hspace{0.17em}\mathit{\rho }=\mathit{b}$ relates the unknown vector $\mathit{\rho }\in {\mathbb{C}}^K$ to the data vector $\mathit{b}\in {\mathbb{C}}^N$.

Next, we illustrate the performance of the Noise Collector in this imaging setup. The most important features are that 1) no calibration is necessary with respect to the level of noise, that 2) exact support recovery is obtained for relatively large levels of noise [i.e., ${\Vert \mathit{e}\Vert }_{{\ell }_2}\le c_1{\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}^2\sqrt{N}/\left({\Vert \mathit{\rho }\Vert }_{{\ell }_1}\sqrt{\ln N}\right)$], and that 3) we have zero false discovery rates for all levels of noise with high probability.

We consider a high-frequency microwave imaging regime with central frequency $f_0=60$ GHz corresponding to ${\lambda }_0=5$ mm. We make measurements for $S=25$ equally spaced frequencies spanning a bandwidth $B=20$ GHz. The array has $N=25$ receivers and an aperture $a=50$ cm. The distance from the array to the center of the imaging window is $L=50$ cm. Then, the resolution is ${\lambda }_{0}L/a=5$ mm in the cross-range (direction parallel to the array) and $c_0/B=15$ mm in range (direction of propagation). These parameters are typical in microwave scanning technology (33).

We seek to image a source vector with sparsity $M=12$ (Fig. 3, Left). The size of the imaging window is 20 $\times$ 60 cm, and the pixel spacing is 5 $\times$ 15 mm. The number of unknowns is, therefore, $K=\mathrm{1,681}$, and the number of data is $N=625$. The size of the Noise Collector is taken to be $\mathrm{\Sigma }=10^4$, and therefore, $\beta \approx 1.5$. When the data are noiseless, we obtain exact recovery as expected (Fig. 3, Right).

Fig. 3.

Noiseless data. The exact solution is recovered for any value of $\lambda$ in algorithm [14]. (Left) The true image. (Right) The recovered solution vector, ${\mathit{\rho }}_{\tau }$, is plotted with red stars, and the true solution vector, $\mathit{\rho }$, is plotted with green circles.

In Fig. 4, we display the imaging results with and without the Noise Collector when the data are corrupted by additive noise. The SNR$\hspace{0.17em}=\hspace{0.17em}1$, and therefore, the ${\ell }_2$ norms of the signals and the noise are equal. In column 1 of Fig. 4, we show the recovered image using ${\ell }_1$-norm minimization without the Noise Collector. There is a lot of grass in this image, with many nonzero values outside the true support. When the Noise Collector is used, the level of the grass is reduced, and the image improves (column 2 of Fig. 4). Still, there are several false discoveries because we use $\tau =1$ in algorithm [14].

Fig. 4.

High level of noise; SNR$\hspace{0.17em}=\hspace{0.17em}1$. (Column 1) ${\ell }_1$-norm minimization without the noise collector. (Column 2) ${\ell }_1$-norm minimization with a noise collector with $\mathrm{\Sigma }=10^4$ columns and $\tau =1$ in algorithm [14]. (Column 3) ${\ell }_1$-norm minimization with a noise collector and the correct $\tau =2$ in algorithm [14]. (Column 4) ${\ell }_2$-norm solution restricted to the support. In Upper, we show the images. In Lower, we show the solution vector with red stars and the true solution vector with green circles. NC = noise collector.

In column 3 of Fig. 4, we show the image obtained with a weight $\tau =2$ in algorithm [14]. With this weight, there are no false discoveries, and the recovered support is exact. This simplifies the imaging problem dramatically as we can now restrict the inverse problem to the true support just obtained and then, solve an overdetermined linear system using a classical ${\ell }_2$ approach. The results are shown in column 4 of Fig. 4. Note that this second step largely compensates for the signal that was lost in the first step due to the high level of noise.

In Fig. 5, we illustrate the performance of the Noise Collector for different sparsity levels $M$ and ${\Vert \mathit{e}\Vert }_{{\ell }_2}/{\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}$ values. Success in recovering the true support of the unknown corresponds to a value of one (yellow in Fig. 5), and failure corresponds to a value of zero (blue in Fig. 5). The small phase transition zone (green in Fig. 5) contains intermediate values. The black lines in Fig. 5 are the theoretical prediction Eq. 5. These results are obtained by averaging over 10 realizations of noise. We show results for three values of data sizes $N=342$, $N=625$, and $N=961$. In our experiments, the nonzero components of the unknown $\mathit{\rho }$ take values in $\left[0.6,0.8\right]$, and therefore, $\hspace{0.17em}{\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}/{\Vert \mathit{\rho }\Vert }_{{\ell }_1}=cst/\sqrt{M}$.

Fig. 5.

(Left) Algorithm performance for exact support recovery. Success corresponds to a value of one (yellow), and failure corresponds to a value of zero (blue). The small phase transition zone (green) contains intermediate values. The black lines are the theoretical estimate $\sqrt{N}/\sqrt{M\ln N}$. Ordinate and abscissa are the sparsity $M$ and ${\Vert \mathit{e}\Vert }_2/{\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}$. The data sizes are (Left) $N=342$, (Center) $N=625$, and (Right) $N=961$.

Remark 1: We considered passive array imaging for ease of presentation. The same results hold for active array imaging with or without multiple scattering; ref. 34 discusses the detailed analytical setup.

Remark 2: We have considered a microwave imaging regime. Similar results can be obtained in other regimes.

Proofs

Proof of Lemma 1: Using the rotational invariance of all of our probability distributions, inequality [9] is true if

$$\mathbb{P}\left(\underset{i}{max}\left|⟨{\mathit{d}}_i,\mathit{e}⟩\right|\ge c_0\sqrt{\ln N}/\sqrt{N}\right)\le 1/N^{\kappa },$$

where ${\mathit{d}}_i$, $i=\mathrm{1,2},\dots ,K+\mathrm{\Sigma }$ are (possibly dependent) uniformly distributed on ${\mathbb{S}}^{N-1}$, and we can assume that $\mathit{e}=\left(\mathrm{1,0},\dots ,0\right)$. Denote the event

$${\mathrm{\Omega }}_t=\left\{\underset{i}{max}\left|⟨{\mathit{d}}_i,\mathit{e}⟩\right|\ge t/\sqrt{N}\right\}.$$

$\mathbb{P}\left(\left|⟨{\mathit{d}}_i,\mathit{e}⟩\right|\ge t/\sqrt{N}\right)\le 2\exp \left(-t^2/2\right)$ for each ${\mathit{d}}_i$. We obtain $\mathbb{P}\left({\mathrm{\Omega }}_t\right)\le 2\left(K+\mathrm{\Sigma }\right)\exp \left(-t^2/2\right)$ using the union bound. Choosing $t=c_0\sqrt{\ln N}$ for sufficiently large $c_0$, we get $\mathbb{P}\left({\mathrm{\Omega }}_t\right)\le C{N}^{\beta }{N}^{-c_0^2/2}\le N^{-\kappa }$, where $c_0^2>2\left(\beta +\kappa \right)$ and $N\ge N_0$. Hence, Eq. 9 holds with probability $1-N^{-\kappa }$.

If $N$ columns ${\mathit{c}}_j$, $j\in S$ of $\mathcal{C}$ satisfy

$$\underset{j\in S}{min}\left|⟨{\mathit{c}}_j,\mathit{e}⟩\right|\ge \theta ,\theta =\alpha \sqrt{\ln N}/\sqrt{N},$$ [17]

then their convex hull will contain $\theta \mathit{e}$ with probability ${\left(1/2\right)}^N$. Therefore, inequality [10] follows if [17] holds with probability $1-1/N^{\kappa }$. Using the rotational invariance of all of our probability distributions, we can assume that $\mathit{e}=\left(\mathrm{1,0},\dots ,0\right)$. For each ${\mathit{c}}_i$,

$$\mathbb{P}\left(\left|⟨{\mathit{c}}_i,\mathit{e}⟩\right|\ge \frac{t}{\sqrt{N}}\right)=\frac{2}{\sqrt{2\pi }}{\int }_t^{\infty }{\text{e}}^{-\frac{x^2}{2}}dx\ge \frac{1}{2}{\text{e}}^{-t^2}.$$

Split the index set $\mathrm{1,2},\dots ,\mathrm{\Sigma }$ into $N$ nonoverlapping subsets $S_k$, $k=\mathrm{1,2},\dots ,N$ of size $N^{\beta -1}$. For each $S_k$,

$$\mathbb{P}\left(\underset{i\in S_k}{max}\left|⟨{\mathit{c}}_i,\mathit{e}⟩\right|\le \frac{\alpha \sqrt{\ln N}}{\sqrt{N}}\right)\le {\left(1-\frac{1}{2\hspace{0.17em}{N}^{{\alpha }^2}}\right)}^{N^{\beta -1}}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\le {\text{e}}^{-\frac{1}{2}{N}^{\frac{\beta -1}{2}}}$$

for $\alpha =\sqrt{\left(\beta -1\right)/2}$. By independence,

$$\mathbb{P}\left(\left[\mathit{17}\right]\text{holds}\right)\hspace{0.17em}\ge \hspace{0.17em}{\mathrm{\Pi }}_{k=1}^N\mathbb{P}\left(\underset{i\in S_k}{max}\left|⟨{\mathit{c}}_i,\mathit{e}⟩\right|\ge \alpha \sqrt{\ln N}/\sqrt{N}\right).$$

Then, $\mathbb{P}\left(\left[\mathit{17}\right]\text{holds}\right)\ge {\left(1-{\text{e}}^{-\frac{1}{2}{N}^{\frac{\beta -1}{2}}}\right)}^N\ge 1-N{\text{e}}^{-\frac{1}{2}{N}^{\frac{\beta -1}{2}}}$. Choosing $N_0$ sufficiently large, we obtain [10]. $\square$

Proof of Theorem 2: When columns of $\mathcal{A}$ are not orthogonal, we will choose a $\tau$ smaller than that in Theorem 1 by a factor of two. Suppose that the $M$-dimensional space $V$ is the span of the column vectors ${\mathit{a}}_j$, with $j$ in the support of $\mathit{\rho }$. Say that $V$ is spanned by ${\mathit{a}}_1\dots {\mathit{a}}_M$. Let $W=V^{\perp }$ be the orthogonal complement to $V$. Consider the orthogonal decomposition ${\mathit{a}}_i={\mathit{a}}_i^v+{\mathit{a}}_i^w$ for all $i\ge M+1$. Incoherence of ${\mathit{a}}_i$ implies that ${\Vert {\mathit{a}}_i^w\Vert }_{{\ell }_2}\ge 1/2$ for all $i\ge M+1$. Indeed, fix any $i\ge M+1$. Suppose that ${\mathit{a}}_i^v={\sum }_{k=1}^M{\xi }_k{\mathit{a}}_k$ and that $\left|{\xi }_j\right|=\underset{k\le M}{max}\left|{\xi }_k\right|={\Vert \mathit{\xi }\Vert }_{l_{\infty }}$. Thus, $\frac{1}{3M}\ge \left|⟨{\mathit{a}}_j,{\mathit{a}}_i^v⟩\right|\ge \left|⟨{\mathit{a}}_j,{\sum }_{k=1}^M{\xi }_k{\mathit{a}}_k⟩\right|\ge {\Vert \mathit{\xi }\Vert }_{l_{\infty }}\left(1-\frac{M-1}{3M}\right).$ Then, ${\Vert \mathit{\xi }\Vert }_{l_{\infty }}\le 1/\left(2M\right)$. Therefore, ${\Vert {\mathit{a}}_i^v\Vert }_{{\ell }_2}\le {\Vert \mathit{\xi }\Vert }_{{\ell }_1}\le M{\Vert \mathit{\xi }\Vert }_{l_{\infty }}\le 1/2$, and ${\Vert {\mathit{a}}_i^w\Vert }_{{\ell }_2}\ge {\Vert {\mathit{a}}_i\Vert }_{{\ell }_2}-{\Vert {\mathit{a}}_i^v\Vert }_{{\ell }_2}\ge 1/2$.

Project system [4] on $W$. Then, we obtain a new system [4]. The ${\ell }_2$ norms of the columns of new $\mathcal{A}$ are at least $1/2$. Otherwise, the new system satisfies all conditions of Theorem 1. Indeed, ${\mathit{b}}_0$ is projected to zero. All ${\mathit{c}}_i$ and $\mathit{e}/{\Vert \mathit{e}\Vert }_{{\ell }_2}$ are projected to vectors uniformly distributed on ${\mathbb{S}}^{N-M-1}$ by the concentration of measure (27). If any ${\mathit{a}}_i$, $i\ge M+1$, was used in an optimal approximation of ${\mathit{b}}_0+\mathit{e}$, then its projection ${\mathit{a}}_i^w$ is used in an optimal approximation of the projection of ${\mathit{b}}_0+\mathit{e}$ on $W$. This is a contradiction to Lemma 1 if we choose $\tau <c_0/\left(2\alpha \right)$ and recall that ${\Vert {\mathit{a}}_i^w\Vert }_{{\ell }_2}\ge 1/2$. $\square$

Proof of Theorem 3: Choose $\tau$ as in Theorem 2. Incoherence of ${\mathit{a}}_i$ implies that we can argue as in Proof of Theorem 2 and assume that $⟨{\mathit{a}}_i,{\mathit{a}}_j⟩=0$ for $i\hspace{0.17em}\ne \hspace{0.17em}j$, $i,j\in \text{supp}\left(\mathit{\rho }\right)$. Suppose that $V^i$ are the two-dimensional (2D) spaces spanned by $\mathit{e}$ and ${\mathit{a}}_i$ for $i\in \text{supp}\left(\mathit{\rho }\right)$. By Lemma 1, all $\lambda H\left(\tau \right)\cap V^i$ look like the rounded rhombi depicted in Fig. 1, Right, and $\lambda H_1\cap V^i\subset B_{\lambda }^i$ with probability $1-N^{-\kappa }$, where $B_{\lambda }^i$ is a 2D ${\ell }_2$ ball of radius $\lambda c_0\sqrt{\ln N}/\sqrt{N}$. Thus, $\lambda H\left(\tau \right)\cap V^i\subset H_{\lambda }^i$ with probability $1-N^{-\kappa }$, where $H_{\lambda }^i$ is the convex hull of $B_{\lambda }^i$ and a vector $\lambda {\mathit{f}}_i$, ${\mathit{f}}_i={\rho }_i{\Vert \mathit{\rho }\Vert }_{{\ell }_1}^{-1}{\tau }^{-1}{\mathit{a}}_i$. Then, $\text{supp}\left({\mathit{\rho }}_{\tau }\right)=\text{supp}\left(\mathit{\rho }\right)$ if there exists ${\lambda }_0$ so that ${\rho }_i{\mathit{a}}_i+\mathit{e}$ lies on the flat boundary of $H_{{\lambda }_0}^i$ for all $i\in \text{supp}\left(\mathit{\rho }\right)$.

If $\underset{i\in \text{supp}\left(\mathit{\rho }\right)}{min}\left|{\rho }_i\right|\ge \gamma {\Vert \mathit{\rho }\Vert }_{\infty }$, then there is a constant $c_2=c_2\left(\gamma \right)$ such that, if ${\rho }_i{\mathit{a}}_i+\mathit{e}$ lies on the flat boundary of $H_{\lambda }^i$ for some $i$ and some $\lambda$, then there exists ${\lambda }_0$ so that ${\rho }_i{\mathit{a}}_i+c_2\mathit{e}$ lies on the flat boundary of $H_{{\lambda }_0}^i$ for all $i\in \text{supp}\left(\mathit{\rho }\right)$. Suppose that $V$ is spanned by $\mathit{e}$ and ${\mathit{b}}_0$, $H_{\lambda }\subset V$ is the convex hull of $B_{\lambda }$ and $\lambda \mathit{f}$, and $\mathit{f}={\mathit{b}}_0{\Vert \mathit{\rho }\Vert }_{{\ell }_1}^{-1}{\tau }^{-1}$ where $B_{\lambda }\subset V$ is an ${\ell }_2$ ball of radius $\lambda c_0\sqrt{\ln N}/\sqrt{N}$. If ${\mathit{b}}_0+c_2\mathit{e}$ lies on the flat boundary of $H_{\lambda }$, then there must be an $i\in \text{supp}\left(\mathit{\rho }\right)$ such that ${\rho }_i{\mathit{a}}_i+c_2\mathit{e}$ lies on the flat boundary of $H_{\lambda }^i$. If

$$\frac{\left|⟨{\mathit{b}}_0,{\mathit{b}}_0+c_2\mathit{e}⟩\right|}{{\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}{\Vert {\mathit{b}}_0+c_2\mathit{e}\Vert }_{{\ell }_2}}\ge \frac{c_0\sqrt{\ln N}}{\sqrt{N}{\Vert \mathit{f}\Vert }_{{\ell }_2}},$$ [18]

then ${\mathit{b}}_0+c_2\mathit{e}$ lies on the flat boundary of $H_{\lambda }$. Since $\left|⟨{\mathit{b}}_0,\mathit{e}⟩\right|\le c_0{\Vert \mathit{e}\Vert }_{{\ell }_2}{\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}/\sqrt{N}$ with probability $1-N^{-\kappa }$, Eq. 18 holds if ${\Vert \mathit{e}\Vert }_{{\ell }_2}/{\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}\le {\Vert \mathit{f}\Vert }_{{\ell }_2}\sqrt{N/}\left(c_2{c}_0\sqrt{\ln N}\right)\le c_1{\Vert {\mathit{b}}_0\Vert }_{{\ell }_2}{\Vert \mathit{\rho }\Vert }_{{\ell }_1}^{-1}\sqrt{N}/\sqrt{\ln N}$. $\square$

Data Availability Statement.

There are no data in this paper; we present an algorithm, its theoretical analysis, and some numerical simulations.