The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale’s 18th problem

Significance

Instability is the Achilles’ heel of modern artificial intelligence (AI) and a paradox, with training algorithms finding unstable neural networks (NNs) despite the existence of stable ones. This foundational issue relates to Smale’s 18th mathematical problem for the 21st century on the limits of AI. By expanding methodologies initiated by Gödel and Turing, we demonstrate limitations on the existence of (even randomized) algorithms for computing NNs. Despite numerous existence results of NNs with great approximation properties, only in specific cases do there also exist algorithms that can compute them. We initiate a classification theory on which NNs can be trained and introduce NNs that—under suitable conditions—are robust to perturbations and exponentially accurate in the number of hidden layers.

Abstract

Deep learning (DL) has had unprecedented success and is now entering scientific computing with full force. However, current DL methods typically suffer from instability, even when universal approximation properties guarantee the existence of stable neural networks (NNs). We address this paradox by demonstrating basic well-conditioned problems in scientific computing where one can prove the existence of NNs with great approximation qualities; however, there does not exist any algorithm, even randomized, that can train (or compute) such a NN. For any positive integers $K>2$ and L, there are cases where simultaneously 1) no randomized training algorithm can compute a NN correct to K digits with probability greater than 1/2; 2) there exists a deterministic training algorithm that computes a NN with K –1 correct digits, but any such (even randomized) algorithm needs arbitrarily many training data; and 3) there exists a deterministic training algorithm that computes a NN with K –2 correct digits using no more than L training samples. These results imply a classification theory describing conditions under which (stable) NNs with a given accuracy can be computed by an algorithm. We begin this theory by establishing sufficient conditions for the existence of algorithms that compute stable NNs in inverse problems. We introduce fast iterative restarted networks (FIRENETs), which we both prove and numerically verify are stable. Moreover, we prove that only $\mathcal{O}(|\log (\mathrm{ϵ})|)$ layers are needed for an ϵ-accurate solution to the inverse problem.

Deep learning (DL) has demonstrated unparalleled accomplishments in fields ranging from image classification and computer vision (1–3), to voice recognition and automated diagnosis in medicine (4–6), to inverse problems and image reconstruction (7–12). However, there is now overwhelming empirical evidence that current DL techniques typically lead to unstable methods, a phenomenon that seems universal and present in all of the applications listed above (13–21) and in most of the new artificial intelligence (AI) technologies. These instabilities are often detected by what has become commonly known in the literature as “adversarial attacks.” Moreover, the instabilities can be present even in random cases and not just worst-case scenarios (22)—see Fig. 1 for an example of AI-generated hallucinations. There is a growing awareness of this problem in high-stakes applications and society as a whole (20, 23, 24), and instability seems to be the Achilles’ heel of modern AI and DL (Fig. 2, Top row). For example, this is a problem in real-world clinical practice. Facebook and New York University’s 2019 FastMRI challenge reported that networks that performed well in terms of standard image quality metrics were prone to false negatives, failing to reconstruct small, but physically relevant image abnormalities (25). Subsequently, the 2020 FastMRI challenge (26) focused on pathologies, noting, “Such hallucinatory features are not acceptable and especially problematic if they mimic normal structures that are either not present or actually abnormal. Neural network models can be unstable as demonstrated via adversarial perturbation studies (19).” For similar examples in microscopy, see refs. 27 and 28. The tolerance level for false positives/negatives varies within different applications. However, for scenarios with a high cost of misanalysis, it is imperative that false negatives/positives be avoided. AI-generated hallucinations therefore pose a serious danger in applications such as medical diagnosis.

Fig. 1.

AI-generated hallucinations. A trained NN, based on a U-net architecture and trained on a set of ellipses images, generates a black area in a white ellipse (Top Left image, shown as green arrow) when reconstructing the original image x from noiseless measurements. By adding random Gaussian noise e₁ and e₂ (where $||e_1|{|}_{l^2}/||e_2|{|}_{l^2}\approx 2/5$) to the measurements, we see that the trained NN removes the aspiring black ellipse (Top row, Center Left to Center Right). FIRENET on the other hand is completely stable with and without random Gaussian noise (Bottom row, Left to Center Right). In Right column, we show the original image x, with a red square (Top Right) indicating the cropped area. In this example, $A\in {ℂ}^{m\times N}$ is a subsampled discrete Fourier transform with $m/N\approx 0.12$.

Fig. 2.

Top row (unstable neural network in image reconstruction): The neural network AUTOMAP (60) represents the tip of the iceberg of DL in inverse problems. Ref. 60, pp. 1 and 487, promises that one can “…observe superior immunity to noise….” Moreover, the follow-up announcement (ref. 83, pp. 1 and 309) proclaims “A deep-learning-based approach improves speed, accuracy and robustness of biomedical image reconstruction.” However, as we see in Top row, the AUTOMAP reconstruction $\mathrm{\Psi }(Ax+e_j)$ from the subsampled noisy Fourier MRI data $Ax+e_j$ is completely unstable. Here, $A\in {ℂ}^{m\times N}$ is a subsampled Fourier transform, x is the original image, and the e_j s are perturbations meant to simulate the worst-case effect. Note that the condition number $\text{cond}(A{A}^{*})=1$, so the instabilities are not caused by poor condition. The network weights were provided by the authors of ref. 60, which trained and tested it on brain images from the Massachusetts General Hospital Human Connectome Project (MGH-USC HCP) dataset (84). The image x is taken from this dataset. Bottom row (the FIRENET is stable to worst-case perturbations): Using the same method, we compute perturbations ${\tilde{e}}_j$ to simulate the worst-case effect for the FIRENET $\mathrm{\Phi }:{ℂ}^m\to {ℂ}^N$. As can be seen, FIRENET is stable to these worst-case perturbations. Here x and $A\in {ℂ}^{m\times N}$ are the same image and sampling matrix as for AUTOMAP. Moreover, for each j = 1, 2, 3 we have ensured that $||{\tilde{e}}_j|{|}_{l^2}\ge ||e_j|{|}_{l^2}$, where the e_j s are the perturbations for AUTOMAP (we have denoted the perturbations for FIRENET by ${\tilde{e}}_j$ to emphasize that these adversarial perturbations are sought for FIRENET and have nothing to do with the perturbations for AUTOMAP).

Nevertheless, classical approximation theorems show that a continuous function can be approximated arbitrarily well by a neural network (NN) (29, 30). Thus, stable problems described by stable functions can always be solved stably with a NN. This leads to the following fundamental question:

Question.

Why does DL lead to unstable methods and AI-generated hallucinations, even in scenarios where one can prove that stable and accurate neural networks exist?

Foundations of AI for Inverse Problems.

To answer the above question we initiate a program on the foundations of AI, determining the limits of what DL can achieve in inverse problems. It is crucial to realize that an existence proof of suitable NNs does not always imply that they can be constructed by a training algorithm. Furthermore, it is not difficult to compute stable NNs. For example, the zero network is stable, but not particularly useful. The big problem is to compute NNs that are both stable and accurate (30, 31). Scientific computing itself is based on the pillars of stability and accuracy. However, there is often a trade-off between the two. There may be barriers preventing the existence of stable and accurate algorithms, and sometimes accuracy must be sacrificed to secure stability.

Main Results.

We consider the canonical inverse problem of an underdetermined system of linear equations:

$$\text{Given} \text{measurements}\hspace{0.17em}y=Ax+e\in {ℂ}^m,\text{recover}\hspace{0.17em}x\in {ℂ}^N.$$ [1]

Here, $A\in {ℂ}^{m\times N}$ represents a sampling model (m < N), such as a subsampled discrete Fourier transform in MRI, and x the unknown quantity. The problem in Eq. 1 forms the basis for much of inverse problems and image analysis. The vector e models noise or perturbations. Our results demonstrate fundamental barriers preventing NNs (despite their existence) from being computed by algorithms. This helps shed light on the intricate question of why current algorithms in DL produce unstable networks, despite the fact that stable NNs often exist in the particular application. We show the following:

• 1)Theorems 1 and 2: There are well-conditioned problems (suitable condition numbers bounded by 1) where, paradoxically, mappings from training data to suitable NNs exist, but no training algorithm (even randomized) can compute approximations of the NNs from the training data.
• 2)Theorem 2: The existence of algorithms computing NNs depends on the desired accuracy. For any $K\in {\mathbb{Z}}_{\ge 3}$, there are well-conditioned classes of problems where simultaneously 1) algorithms may compute NNs to K –1 digits of accuracy, but not K; 2) achieving K –1 digits of accuracy requires arbitrarily many training data; and 3) achieving K –2 correct digits requires only one training datum.
• 3)Theorems 3 and 4: Under specific conditions that are typically present in, for example, MRI, there are algorithms that compute stable NNs for the problem in Eq. 1. These NNs, which we call fast iterative restarted networks (FIRENETs), converge exponentially in the number of hidden layers. Crucially, we prove that FIRENETs are robust to perturbations (Fig. 2, Bottom row), and they can even be used to stabilize unstable NNs (Fig. 3).
• 4)There is a trade-off between stability and accuracy in DL, with limits on how well a stable NN can perform in inverse problems. Fig. 4 demonstrates this with a U-net trained on images consisting of ellipses that is quite stable. However, when a detail not in the training set is added, it washes it out almost entirely. FIRENETs offer a blend of both stability and accuracy. However, they are by no means the end of the story. Tracing out the optimal stability vs. accuracy trade-off is crucial for applications and will no doubt require a myriad of different techniques to tackle different problems and stability tolerances.

Fig. 3.

Adding a few FIRENET layers at the end of AUTOMAP makes it stable. The FIRENET $\mathrm{\Phi }:{ℂ}^m\times {ℂ}^N\to {ℂ}^N$ takes as input measurements $y\in {ℂ}^m$ and an initial guess for x, which we call $x_0\in {ℂ}^N$. We now concatenate a 25-layer (p = 5, n = 5) FIRENET $\mathrm{\Phi }$ and the AUTOMAP network $\mathrm{\Psi }:{ℂ}^m\to {ℂ}^N$, by using the output from AUTOMAP as initial guess x₀; i.e., we consider the neural network mapping $y\mapsto \mathrm{\Phi }(y,\mathrm{\Psi }(y))$. In this experiment, we consider the image x from Fig. 2 and the perturbed measurements $\tilde{y}=Ax+e_3$ (here A is as in Fig. 2). Left shows the reconstruction of AUTOMAP from Fig. 2. Center Left shows the reconstruction of FIRENET with $x_0=\mathrm{\Psi }(\tilde{y})$. Center Right shows the reconstruction of FIRENET from Fig. 2. Right shows the reconstruction of the concatenated network with a worst-case perturbation ${\widehat{e}}_3$ such that $||{\widehat{e}}_3|{|}_{l^2}\ge ||e_3|{|}_{l^2}$. In all other experiments we set $x_0=0$ and consider $\mathrm{\Phi }$ as a mapping $\mathrm{\Phi }:{ℂ}^m\to {ℂ}^N$.

Fig. 4.

Trained neural networks with limited performance can be stable. We examine the accuracy/stability trade-off for linear inverse problems by considering three reconstruction networks ${\mathrm{\Phi }}_j:{ℂ}^m\to {ℂ}^N$, j = 1, 2, 3. Here ${\mathrm{\Phi }}_1$ is a FIRENET, whereas ${\mathrm{\Phi }}_2$ and ${\mathrm{\Phi }}_3$ are the U-nets mentioned in the main text, trained without and with noisy measurements, respectively. For each network, we compute a perturbation $w_j\in {ℂ}^N$ meant to simulate the worst-case effect, and we show a cropped version of the perturbed images $x+w_j$ in Left column (rows 2 to 4). In Center column (rows 2 to 4), we show the reconstructed images ${\mathrm{\Phi }}_j(A(x+w_j))$ from each of the networks. In Right column (rows 2 to 4) we test the networks’ ability to reconstruct a tiny detail h₁, in the form of the text “Can u see it?”. As we see, the network trained on noisy measurements is stable to worst-case perturbations, but it is not accurate. Conversely, the network trained without noise is accurate but not stable. The FIRENET is balancing this trade-off and is accurate for images that are sparse in wavelets and stable to worst-case perturbations.

Fundamental Barriers

We first consider basic mappings used in modern mathematics of information, inverse problems, and optimization. Given a matrix $A\in {ℂ}^{m\times N}$ and a vector $y\in {ℂ}^m$, we consider the following three popular minimization problems:

$$\begin{array}{l}(P_1)\hspace{1em}{\text{argmin}}_{x\in {ℂ}^N}{F}_1^A(x)≔\left|\right|x|{|}_{l_w^1},\text{s}.\text{t}.\hspace{0.17em}\hspace{0.17em}\text{||}Ax-y{\text{||}}_{{\text{l}}^2}\text{}\le \hspace{0.17em}\text{}\mathrm{ϵ},\\ (P_2)\hspace{1em}{\text{argmin}}_{x\in {ℂ}^N}{F}_2^A(x,y,\lambda )≔\lambda \left|\right|x|{|}_{l_w^1}+||Ax-y|{|}_{l^2}^2,\\ (P_3)\hspace{1em}{\text{argmin}}_{x\in {ℂ}^N}{F}_3^A(x,y,\lambda )≔\lambda \left|\right|x|{|}_{l_w^1}+||Ax-y|{|}_{l^2},\end{array}$$

known respectively as quadratically constrained basis pursuit [we always assume existence of a feasible x for $(P_1)$], unconstrained least absolute shrinkage and selection operator (LASSO), and unconstrained square-root LASSO. Such sparse regularization problems are often used as benchmarks for Eq. 1, and we prove impossibility results for computing the NNs that can approximate these mappings. Our results initiate a classification theory on which NNs can be computed to a certain accuracy.

The parameters λ and ϵ are positive rational numbers, and the weighted $l_w^1$ norm is given by $||x|{|}_{l_w^1}≔{\displaystyle {\sum }_{l=1}^N{w}_l}|x_l|,$ where each weight w_j is a positive rational. Throughout, we let

$${\mathrm{\Xi }}_j(A,y)\hspace{0.17em}\text{be} \text{the} \text{set} \text{of} \text{minimizers} \text{for}\hspace{0.17em}(P_j).$$ [2]

Let $A\in {ℂ}^{m\times N}$ and let $\mathcal{S}={\left\{y_k\right\}}_{k=1}^R\subset {ℂ}^m$ be a collection of samples ($R\in \mathbb{N}$). We consider the following key question:

Question.

Given a collection Ω of pairs $(A,\mathcal{S})$, does there exist a neural network approximating Ξ_j, and if so, can such an approximation be trained or determined by an algorithm?

To make this question precise, note that A and samples in $\mathcal{S}$ will typically never be exact, but can be approximated/stored to arbitrary precision. For example, this would occur if A was a subsampled discrete cosine transform. Thus, we assume access to rational approximations ${\left\{y_{k,n}\right\}}_{k=1}^R$ and A_n with

$$||y_{k,n}-y_k|{|}_{l^2}\le 2^{-n},\hspace{1em}||A_n-A||\le 2^{-n},\hspace{1em}\forall n\in \mathbb{N},$$ [3]

where $\left|\right|·\left|\right|$ refers to the usual Euclidean operator norm. The bounds $2^{-n}$ are simply for convenience and can be replaced by any other sequence converging to zero. We also assume access to rational ${\left\{x_{k,n}\right\}}_{k=1}^R$ with

$$\underset{x^{*}\in {\mathrm{\Xi }}_j(A_n,y_{k,n})}{\inf }||x_{k,n}-x^{*}|{|}_{l^2}\le 2^{-n},\hspace{1em}\forall n\in \mathbb{N}.$$ [4]

Hence, the training data associated with $(A,\mathcal{S})\in \mathrm{\Omega }$ must be

$${\iota }_{A,\mathcal{S}}≔\left\{\left(y_{k,n},A_n,x_{k,n}\right)\hspace{0.17em}|\hspace{0.17em}k=1,\dots ,R,\text{and}\hspace{0.17em}n\in \mathbb{N}\right\}.$$ [5]

This set is formed of arbitrary precision rational approximations of finite collections of data associated with $(A,\mathcal{S})$. Given a collection Ω of pairs $(A,\mathcal{S})$, the class of all such admissible training data is denoted by

$${\mathrm{\Omega }}_{\mathcal{T}}≔\left\{{\iota }_{A,\mathcal{S}}\hspace{0.17em}\text{as} \text{in} \text{Eq}.\hspace{0.17em}\hspace{0.17em}5\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\text{|}\hspace{0.17em}(A,\mathcal{S})\in \mathrm{\Omega },\text{Eqs}.3 \text{to} 4 \text{hold}\right\}.$$

Statements addressing the above question are summarized in Theorems 1 and 2. We use ${\mathcal{N}}_{m,N}$ to denote the class of NNs from ${ℂ}^m$ to ${ℂ}^N$. We use standard definitions of feedforward NNs (32), precisely given in SI Appendix.

Theorem 1

For any collection Ω of such $(A,\mathcal{S})$ described above, there exists a mapping

$$\begin{array}{l}\mathcal{K}:{\mathrm{\Omega }}_{\mathcal{T}}\to {\mathcal{N}}_{m,N},\hspace{1em}\mathcal{K}({\iota }_{A,\mathcal{S}})={\phi }_{A,\mathcal{S}},\\ s.t.\hspace{1em}{\phi }_{A,\mathcal{S}}(y)\in {\mathrm{\Xi }}_j(A,y),\hspace{1em}\forall y\in \mathcal{S}.\end{array}$$

In words, $\mathcal{K}$ maps the training data ${\mathrm{\Omega }}_{\mathcal{T}}$ to NNs that solve the optimization problem $(P_j)$ for each $(A,\mathcal{S})\in \mathrm{\Omega }$.

Despite the existence of NNs guaranteed by Theorem 1, computing or training such a NN from training data is most delicate. The following is stated precisely and proved in SI Appendix. We also include results for randomized algorithms, which are common in DL (e.g., stochastic gradient descent).

Theorem 2

Consider the optimization problem $(P_j)$ for fixed parameters $\lambda \in (0,1]$ or $\mathrm{ϵ}\in (0,1/2]$ and w_l = 1, where $N\ge 2$ and m < N. Let K > 2 be a positive integer and let $L\in \mathbb{N}$. Then there exists an infinite class $\mathrm{\Omega }=\mathrm{\Omega }(K,L)$ of elements $(A,\mathcal{S})$ as above, with the following properties. The class Ω is well-conditioned with relevant condition numbers bounded by 1 independent of all parameters. However, the following hold simultaneously (where accuracy is measured in the l² norm):

• 1)(K digits of accuracy impossible) There does not exist any algorithm that, given a training set ${\iota }_{A,\mathcal{S}}\text{}\in \text{}{\mathrm{\Omega }}_{\mathcal{T}}$, produces a NN with K digits of accuracy for any element of $\mathcal{S}$. Furthermore, for any p > 1∕2, no probabilistic algorithm (Blum–Shub–Smale [BSS], Turing, or any model of computation) can produce a NN with K digits of accuracy with probability at least p.
• 2)(K –1 digits of accuracy possible but requires arbitrarily many training data) There does exist a deterministic Turing machine that, given a training set ${\iota }_{A,\mathcal{S}}\text{}\in \text{}{\mathrm{\Omega }}_{\mathcal{T}}$, produces a NN accurate to K –1 digits over $\mathcal{S}$. However, for any probabilistic Turing machine, $M\in \mathbb{N}$ and $p\in \left[0,\frac{N-m}{N+1-m}\right)$ that produces a NN, there exists a training set ${\iota }_{A,\mathcal{S}}\in {\mathrm{\Omega }}_{\mathcal{T}}$ such that for all $y\in \mathcal{S}$, the probability of failing to achieve K –1 digits or requiring more than M training data is greater than p.
• 3)(K –2 digits of accuracy possible with L training data) There does exist a deterministic Turing machine that, given a training set ${\iota }_{A,\mathcal{S}}\in {\mathrm{\Omega }}_{\mathcal{T}}$ and using only L training data from each ${\iota }_{A,\mathcal{S}}$, produces a NN accurate to K –2 digits over $\mathcal{S}$.

Remark 1 (condition and class size).

The statement in Theorem 2 refers to the standard condition numbers used in optimization and scientific computing. For precise definitions, see SI Appendix. The class Ω we construct is infinite. Similarly, one can design a finite class Ω with the same conclusion by allowing the sample size R to be infinite.

Remark 2 (distributions on training data).

In DL it is often the case that one assumes some probability distribution on the training data. This is not needed for Theorem 2. However, having a probability distribution on the training data ${\iota }_{A,\mathcal{S}}$ would not invalidate statement 1 in Theorem 2. In particular, there is no (computable) probability distribution that would make statement 1 in Theorem 2 cease to be true. This follows from the probabilistic part of statement 1 in Theorem 2, as the existence of such a (computable) distribution and an algorithm would yield a randomized algorithm violating statement 1 in Theorem 2.

Remark 3 (on the role of K in Theorem 2).

The result should be understood as fixing an integer K (and L) and then $\mathrm{\Omega }=\mathrm{\Omega }(K,L)$ depends on K and L. However, given a particular Ω one can ask, what is the largest K such that one can compute K correct digits? Note that we typically have $K=\lfloor \log ({\mathrm{ϵ}}^{-1})\rfloor$, where $\mathrm{ϵ}>0$ is the so-called breakdown epsilon of the problem (33), i.e., the largest $\mathrm{ϵ}>0$ for which all algorithms will fail to provide ϵ accuracy. When the breakdown epsilon $\mathrm{ϵ}>0$, it is typically impossible to check whether an algorithm fails (33). Thus, even if an algorithm would succeed with probability 1/2, one could never trust the output.

Remark 4 (Gödel, Turing, Smale, and Theorem 2).

Theorems 1 and 2 demonstrate basic limitations on the existence of algorithms that can compute NNs despite their existence. This relates to Smale’s 18th problem, “What are the limits of intelligence, both artificial and human?”, from the list of mathematical problems for the 21st century (34), which echoes the Turing test from 1950 (35). Smale’s discussion is motivated by the results of Gödel (36) and Turing (37) establishing impossibility results on what mathematics and digital computers can achieve (38). Our results are actually stronger, however, than what can be obtained with Turing’s techniques. Theorem 2 holds even for any randomized Turing or BSS machine that can solve the halting problem. It immediately opens up for a classification theory on which NNs can be computed by randomized algorithms. Theorem 3 is a first step in this direction. See also the work by Niyogi, Smale, and Weinberger (39) on existence results of algorithms for learning.

Numerical Example.

To highlight the impossibility of computing NNs (Theorem 2)—despite their existence by Theorem 1—we consider the following numerical example: Consider the problem $(P_3)$, with w_l = 1 and $\lambda =1$. Theorem 2 is stated for a specific input class $\mathrm{\Omega }=\mathrm{\Omega }(K)$ depending on the accuracy parameter K, and in this example we consider three different such classes. In Theorem 2, we required that K > 2 so that $K-2>0$, but this is not necessary to show the impossibility statement 1, so we consider K = 1, 3, 6. Full details of the following experiment are given in SI Appendix.

To show that it is impossible to compute NNs that can solve $(P_3)$ to arbitrary accuracy we consider FIRENETs ${\mathrm{\Phi }}_{A_n}$ (the NNs in Theorem 3) and learned ISTA (LISTA) networks ${\mathrm{\Psi }}_{A_n}$ based on the architecture choice from ref. 40. The networks are trained to high accuracy on training data on the form of Eq. 5 with R = 8, 000 training samples and n given as in Table 1. In all cases N = 20, $m=N-1$, and the $x_{k,n}$ s minimizing $(P_3)$ with input data $(y_{k,n},A_n)$ are all 6-sparse. The choice of N, m, and sparsity is to allow for fast training; other choices are certainly possible.

Table 1.

Impossibility of computing approximations of the existing neural network to arbitrary accuracy

${\mathrm{\Psi }}_{A_n}$	${\mathrm{\Phi }}_{A_n}$	$\left|\right|A_n-A\left|\right|\le 2^{-n}$	${10}^{-K}$	$\mathrm{\Omega }(K)$
		$\left|\right|y_n-y|{|}_{l^2}\le 2^{-n}$
0.2999690	0.2597827	n = 10	10–1	K = 1
0.3000000	0.2598050	n = 20	10–1	K = 1
0.3000000	0.2598052	n = 30	10–1	K = 1
0.0030000	0.0025980	n = 10	10–3	K = 3
0.0030000	0.0025980	n = 20	10–3	K = 3
0.0030000	0.0025980	n = 30	10–3	K = 3
0.0000030	0.0000015	n = 10	10–6	K = 6
0.0000030	0.0000015	n = 20	10–6	K = 6
0.0000030	0.0000015	n = 30	10–6	K = 6

We demonstrate statement 1 from Theorem 2 on FIRENETs ${\mathrm{\Phi }}_{A_n}$ and LISTA networks ${\mathrm{\Psi }}_{A_n}$. Shown is the shortest l² distance between the output from the networks and the true solution of the problem $(P_3)$, with w_l = 1 and λ = 1, for different values of n and K. Note that none of the networks can compute the existing correct NN (that exists by Theorem 1 and coincides with ${\mathrm{\Xi }}_3$) to ${10}^{-K}$ digits accuracy, while all of them are able to compute approximations that are accurate to ${10}^{-K+1}$ digits [for the input class $\mathrm{\Omega }(K)$]. This agrees exactly with Theorem 2.

Table 1 shows the errors for both LISTA and FIRENETs. Both network types are given input data (y_n, A_n), approximating the true data (y, A). As is clear from Table 1, none of the networks are able to compute an approximation to the true minimizer in ${\mathrm{\Xi }}_3(A,y)$ to K digits accuracy. However, both networks compute an approximation with K –1 digits accuracy. These observations agree precisely with Theorem 2.

The Subtlety and Difficulty of Removing Instabilities and the Need for Additional Assumptions.

Theorem 2 shows that the problems $(P_j)$ cannot, in general, be solved by any training algorithm. Hence, any attempt at using the problems $(P_j)$ as approximate solution maps of the general inverse problem in Eq. 1, without additional assumptions, is doomed to fail. This is not just the case for reconstruction using sparse regularization, but also applies to other methods. In fact, any stable and accurate reconstruction procedure must be “kernel aware” (22), a property that most DL methods do not enforce. A reconstruction method $\mathrm{\Psi }:{ℂ}^m\to {ℂ}^N$ lacks kernel awareness if it approximately recovers two vectors

$$\left|\right|\mathrm{\Psi }(Ax)-x\left|\right|\le \mathrm{ϵ}\hspace{1em}\text{and}\hspace{1em}\left|\right|\mathrm{\Psi }(Ax\prime )-x\prime \left|\right|\le \mathrm{ϵ}$$ [6]

whose difference $\left|\right|x-x\prime \left|\right|\gg 2\mathrm{ϵ}$ is large, but where the difference lies close to the null space of A (which is nontrivial due to m < N) so that $\left|\right|A(x-x\prime )\left|\right|<\mathrm{ϵ}$. In particular, by applying Eq. 6 and the triangle inequality twice, we have that

$$\left|\right|\mathrm{\Psi }(Ax)-\mathrm{\Psi }(Ax\prime )\left|\right|\ge \left|\right|x-x\prime \left|\right|-2\mathrm{ϵ}$$ [7]

implying instability, as it requires only a perturbation $e=A(x\prime -x)$ of size $\left|\right|e\left|\right|<\mathrm{ϵ}$ for $\mathrm{\Psi }(Ax+e)=\mathrm{\Psi }(Ax\prime )$ to reconstruct the wrong image. The issue here is that if we want to accurately recover x and $x\prime$, i.e., we want Eq. 6 to hold, then we cannot simultaneously have that $x-x\prime$ lies close to the kernel. Later we shall see conditions that circumvent this issue for our model class, thereby allowing us to compute stable and accurate NNs.

While training can encourage the conditions in Eq. 6 to hold, it is not clear how many of the defense techniques in DL, simultaneously, will protect against the condition $\left|\right|A(x-x\prime )\left|\right|<\mathrm{ϵ}$. One standard attempt to remedy instabilities is adversarial training (41). However, while this strategy can potentially avoid Eq. 6, it may yield poor performance. For example, consider the following optimization problem, which generates a reconstruction in the form of a NN given samples $\mathrm{\Theta }=\{(y_s,x_s):s=1,\dots ,R,A{x}_s=y_s\}$ and $\mathrm{ϵ},\lambda >0$:

$$\underset{{}_{\varphi \in {\mathcal{N}}_{m,N}}}{\min }\text{}{\displaystyle \sum _{s=1}^R\text{}}\underset{{}_{\left|\right|e|{|}_{l^2}\le \mathrm{ϵ}}}{\max }\text{}\left\{\text{}\left|\right|x_s\text{}-\text{}\varphi (y_s)|{|}_{l^2}^2\text{}+\text{}\lambda ||x_s\text{}-\text{}\varphi (y_s\text{}+\text{}e)|{|}_{l^2}^2\text{}\right\}\text{}.\text{}$$ [8]

In other words, for each training point $(y,x)\in \mathrm{\Theta }$ we find the worst-case perturbation e in the ϵ -ball around y. This is a simplified model of what one might do using generative adversarial networks (GANs) to approximate adversarial perturbations (42, 43). For simplicity, assume that A has full row rank m and that we have access to exact measurements y_s = Ax_s. Suppose that our sample is such that ${\min }_{i\ne j}||y_i-y_j|{|}_{l^2}>2\mathrm{ϵ}$. In this case, $\varphi$ minimizes Eq. 8 if and only if $\varphi (y_s+e)=x_s$ for all e with $\left|\right|e|{|}_{l^2}\le \mathrm{ϵ}.$ A piecewise affine network achieving this can easily be constructed using ReLU (rectified linear unit) activation functions. Now suppose that x₂ is altered so that $x_1-x_2$ lies in the kernel of A. Then for any minimizer $\varphi$, we must have $\varphi (y_1+e)=\varphi (y_2+e)=(x_1+x_2)/2$ for any e with $\left|\right|e|{|}_{l^2}\le \mathrm{ϵ}$, and hence we can never be more than $\left|\right|x_1-\varphi (y_1)\left|\right|=\left|\right|x_1-x_2|{|}_{l^2}/2$ accurate over the whole test sample. Similar arguments apply to other methods aimed at improving robustness such as adding noise to training samples (known as “jittering”) (Fig. 4). Given such examples and Theorem 2, we arrive at the following question:

Question.

Are there sufficient conditions on A that imply the existence of an algorithm that can compute a neural network that is both stable and accurate for the problem in Eq. 1?

Sufficient Conditions for Algorithms to Compute Stable and Accurate NNs

Sparse regularization, such as the problems $(P_j)$, forms the core of many start-of-the-art reconstruction algorithms for inverse problems. We now demonstrate a sufficient condition (from compressed sensing) guaranteeing the existence of algorithms for stable and accurate NNs. Sparsity in levels is a standard sparsity model for natural images (44–47) as images are sparse in levels in X-lets (wavelets, curvelets, shearlets, etc.).

Definition 1

(Sparsity in Levels)

Let $\mathbf{M}=(M_1,\dots ,M_r)\in {\mathbb{N}}^{\text{r}},\hspace{0.17em}1\le M_1<\dots <M_r=N$, and $\mathbf{s}=(s_1,\dots ,s_r)\in {\mathbb{N}}_0^r,$ where $s_l\le M_l-M_{l-1}$ for $l=1,\dots ,r$ ($M_0=0$). $x\in {ℂ}^N$ is $(\mathbf{s},\mathbf{M})$ -sparse in levels if $\mid \text{supp}(x)\cap \{M_{l-1}+1,\dots ,M_l\}\mid \le s_l$ for $l=1,\dots ,r.$ The total sparsity is $s=s_1+\dots +s_r$. We denote the set of $(\mathbf{s},\mathbf{M})$ -sparse vectors by ${\mathrm{\Sigma }}_{\mathbf{s},\mathbf{M}}$. We also define the following measure of distance of a vector x to ${\mathrm{\Sigma }}_{\mathbf{s},\mathbf{M}}$ by

$${\sigma }_{\mathbf{s},\mathbf{M}}{(x)}_{l_w^1}=\inf \{||x-z|{|}_{l_w^1}:z\in {\mathrm{\Sigma }}_{\mathbf{s},\mathbf{M}}\}.$$

This model has been used to explain the effectiveness of compressed sensing (46, 48–52) in real-life applications (53). For simplicity, we assume that each $s_l>0$ and that $w_i=w_{(l)}$ if $M_{l-1}+1\le i\le M_l$ (the weights in the $l_w^1$ norm are constant in each level). For a vector c that is compressible in the wavelet basis, ${\sigma }_{\mathbf{s},\mathbf{M}}{(x)}_{l_w^1}$ is expected to be small if x is the vector of wavelet coefficients of c and the levels correspond to wavelet levels (54). In general, the weights are a prior on anticipated support of the vector (55), and we discuss some specific optimal choices in SI Appendix.

For $\mathcal{I}\subset \{1,\dots ,N\}$, let $P_{\mathcal{I}}\in {ℂ}^{N\times N}$ denote the projection ${(P_{\mathcal{I}}x)}_i=x_i$ if $i\in \mathcal{I}$ and ${(P_{\mathcal{I}}x)}_i=0$ otherwise. The key kernel-aware property that allows for stable and accurate recovery of $(\mathbf{s},\mathbf{M})$ -spare vectors for the inverse problem Eq. 1 is the weighted robust null space property in levels (wrNSPL):

Definition 2

(wrNSPL)

Let $(\mathbf{s},\mathbf{M})$ be local sparsities and sparsity levels, respectively. For weights ${\left\{w_i\right\}}_{i=1}^N,\hspace{0.17em}A\in {ℂ}^{m\times N}$ satisfies the wrNSPL of order $(\mathbf{s},\mathbf{M})$ with constants $0<\rho <1$ and $\gamma >0$ if for any $(\mathbf{s},\mathbf{M})$ support set $\mathcal{I}\subset \{1,\dots ,N\}$ (with complement ${\mathcal{I}}^c=\{1,\dots ,N\}\backslash \mathcal{I}$),

$$\left|\right|P_{\mathcal{I}}x|{|}_{l^2}\le \frac{\rho \left|\right|P_{{\mathcal{I}}^c}x|{|}_{l_w^1}}{\sqrt{{\displaystyle \sum _{l=1}^r{w}_{(l)}^2}{s}_l}}+\gamma \left|\right|Ax|{|}_{l^2},\hspace{1em}\hspace{1em}\mathit{for}\hspace{0.17em}\mathit{all}\hspace{0.17em}x\in {ℂ}^N.$$

We highlight that if A satisfies the wrNSPL, then

$$\left|\right|x-x\prime |{|}_{l^2}\le C||A(x-x\prime )|{|}_{l^2},\hspace{1em}\forall x,x\prime \in {\mathrm{\Sigma }}_{\mathbf{s},\mathbf{M}},$$

where $C=C(\rho ,\gamma )>0$ is a constant depending only on ρ and γ (SI Appendix). This ensures that if $\left|\right|x-x\prime |{|}_{{\ell }^2}\gg 2\mathrm{ϵ}$, then we cannot, simultaneously, have that $\left|\right|A(x-x\prime )\left|\right|<\mathrm{ϵ}$, causing the instability in Eq. 7. Below, we give natural examples of sampling in compressed imaging where such a property holds, for known ρ and γ, with large probability. We can now state a simplified version of our result (the full version with explicit constants is given and proved in SI Appendix):

Theorem 3

There exists an algorithm such that for any input sparsity parameters $(\mathbf{s},\mathbf{M})$, weights ${\left\{w_i\right\}}_{i=1}^N,\hspace{0.17em}A\in {ℂ}^{m\times N}$ (with the input A given by $\left\{A_l\right\}$) satisfying the wrNSPL with constants $0<\rho <1$ and $\gamma >0$ (also input), and input parameters $n\in \mathbb{N}$ and $\{\delta ,b_1,b_2\}\subset {\mathbb{Q}}_{>0}$, the algorithm outputs a neural network ${\varphi }_n$ with $\mathcal{O}(n)$ hidden layers and $\mathcal{O}(N)$ width with the following property: For any $x\in {ℂ}^N,\hspace{0.17em}y\in {ℂ}^m$ with

$${\sigma }_{\mathbf{s},\mathbf{M}}{(x)}_{l_w^1}+||Ax-y|{|}_{l^2}\mathrm{\lesssim }\delta ,\hspace{1em}||x|{|}_{l^2}\mathrm{\lesssim }{b}_1,\hspace{1em}||y|{|}_{l^2}\mathrm{\lesssim }{b}_2,$$

we have $||{\varphi }_n(y)-x|{|}_{l^2}\mathrm{\lesssim }\delta +e^{-n}.$

Hence, up to the small error term ${\sigma }_{\mathbf{s},\mathbf{M}}{(x)}_{l_w^1}$, as $n\to \infty$ (with exponential convergence), we recover x stably with an error proportional to the measurement error $\left|\right|Ax-y|{|}_{l^2}$. The explicit constant in front of the $\left|\right|Ax-y|{|}_{l^2}$ term can be thought of as an asymptotic local Lipschitz constant for the NNs as $n\to \infty$ and thus measures stability of inexact input y. The error of order ${\sigma }_{\mathbf{s},\mathbf{M}}{(x)}_{l_w^1}$ measures how close the vector x is from the model class of sparse in levels vectors. In the full version of Theorem 3, we also bound the error when we only approximately apply the nonlinear maps of the NNs and show that these errors can only accumulate slowly as n increases. In other words, we also gain a form of numerical stability of the forward pass of the NN. We call our NNs FIRENETs.

Remark 5 (unrolling does not in general yield an algorithm producing an accurate network).

Unrolling iterative methods has a rich history in DL (9, 56). Note, however, that Theorem 2 demonstrates that despite the existence of an accurate neural network, there are scenarios where no algorithm exists that can compute it. Thus, unrolling optimization methods can work only under certain assumptions. Our results are related to the work of Ben-Tal and Nemirovski (57) (see also ref. 58), which shows how key assumptions such as the robust nullspace property help bound the error of the approximation to a minimizer in terms of error bounds on the approximation to the objective function. This is related to robust optimization (59).

In the case that we do not know ρ or γ (the constants in the definition of wrNSPL), we can perform a log-scale grid search for suitable parameters. By increasing the width of the NNs to $\mathcal{O}(N\log (n))$, we can still gain exponential convergence in n by choosing the parameters in the grid search that lead to the vector with minimal $F_3^A$ [the objective function of $(P_3)$]. In other cases, such as Theorem 4 below, it is possible to prove probabilistic results where ρ and γ are known.

Examples in Image Recovery.

As an application, we consider Fourier and Walsh (binary) sampling, using Haar wavelets as a sparsifying transform. Our results can also be generalized to infinite-dimensional settings via higher-order Daubechies wavelets. Theorem 3 is quite general and there are numerous other applications where problem-dependent results similar to Theorem 4 can be shown.

Let $K=2^r$ for $r\in \mathbb{N}$, and set $N=K^d$ so that the objective is to recover a vectorized d-dimensional tensor $c\in {ℂ}^N$. Let $V\in {ℂ}^{N\times N}$ correspond to the d-dimensional discrete Fourier or Walsh transform (SI Appendix). Let $\mathcal{I}\subset \{1,\dots ,N\}$ be a sampling pattern with cardinality $m=\left|\mathcal{I}\right|$ and let $D=\text{diag}(d_1,\dots ,d_m)\in {ℂ}^{m\times m}$ be a suitable diagonal scaling matrix, whose entries along the diagonal depend only on $\mathcal{I}$. We assume we can observe the subsampled, scaled and noisy measurements $y=D{P}_{\mathcal{I}}Vc+e\in {ℂ}^m$, where projection $P_{\mathcal{I}}$ is treated as an $m\times N$ matrix by ignoring the zero entries.

To recover a sparse representation of c, we consider Haar wavelet coefficients. Denote the discrete d-dimensional Haar wavelet transform by $\mathrm{\Psi }\in {ℂ}^{N\times N}$ and note that ${\mathrm{\Psi }}^{*}={\mathrm{\Psi }}^{-1}$ since $\mathrm{\Psi }$ is unitary. To recover the wavelet coefficients $x=\mathrm{\Psi }c$ of c, we consider the matrix $A=D{P}_{\mathcal{I}}V{\mathrm{\Psi }}^{*}$ and observe that $y=Ax+e=D{P}_{\mathcal{I}}Vc+e$. A key result in this work is that we can design a probabilistic sampling strategy (SI Appendix), for both Fourier and Walsh sampling in d dimensions, requiring no more than $m\mathrm{\gtrsim }(s_1+\dots +s_r)·\mathcal{L}$ samples, that can ensure with high probability that A satisfies the wrNSPL with certain constants. The sparsity in levels structure (Definition 1) is chosen to correspond to the r wavelet levels. Here $\mathcal{L}$ is a logarithmic term in N, m, s, and ${\mathrm{ϵ}}_{\mathrm{\mathbb{P}}}^{-1}$ [where ${\mathrm{ϵ}}_{\mathrm{\mathbb{P}}}\in (0,1)$ is a probability]. This result is crucial, as it makes A kernel aware for vectors that are approximately $(\mathbf{s},\mathbf{M})$ -sparse and allows us (using Theorem 3) to design NNs that can stably and accurately recover approximately $(\mathbf{s},\mathbf{M})$ -sparse vectors. Moreover, due to the exponential convergence in Theorem 3, the depth of these NNs depends only logarithmically on the error δ. Below follows a simplified version of our result (the full precise version is given and proved in SI Appendix).

Theorem 4

Consider the above setup of recovering wavelet coefficients $x=\mathrm{\Psi }c$ of a tensor $c\in {ℂ}^{K^d}$ from subsampled, scaled and noisy Fourier or Walsh measurements $y=D{P}_{\mathcal{I}}Vc+e$. Let $A=D{P}_{\mathcal{I}}V{\mathrm{\Psi }}^{*},\hspace{0.17em}m=\left|\mathcal{I}\right|$, and ${\mathrm{ϵ}}_{\mathrm{\mathbb{P}}}\in (0,1)$. We then have the following:

• 1)
  If $\mathcal{I}\subset \{1,\dots ,N\}$ is a random sampling pattern drawn according to the strategy specified in SI Appendix, and

  $$m\mathrm{\gtrsim }(s_1+\cdots +s_r)·\mathcal{L},$$

  then with probability $1-{\mathrm{ϵ}}_{\mathrm{\mathbb{P}}}$, A satisfies the wrNSPL of order $(\mathbf{s},\mathbf{M})$ with constants $(\rho ,\gamma )=(1/2,\sqrt{2}),\hspace{0.17em}{w}_{(l)}=\sqrt{s/s_l},\hspace{0.17em}s=s_1+\cdots +s_r$. Here $\mathcal{L}$ denotes a term logarithmic in ${\mathrm{ϵ}}_{\mathrm{\mathbb{P}}}^{-1},N,m$ and s.
• 2)
  Suppose $\mathcal{I}$ is chosen as above. For any $\delta \in (0,1)$, let $\mathcal{J}(\delta ,\mathbf{s},\mathbf{M},w)$ be the set of all $y\text{}=\text{}Ax\text{}+\text{}e\text{}\in {ℂ}^m$ where

  $$||x|{|}_{l^2}\le 1,\hspace{1em}\max \left\{{\sigma }_{\mathbf{s},\mathbf{M}}{(x)}_{l_w^1},||e|{|}_{l^2}\right\}\le \delta .$$ [9]
  We provide an algorithm that constructs a neural network $\varphi$ with $\mathcal{O}(\log ({\delta }^{-1}))$ hidden layers [and width bounded by $2(N+m)$] such that with probability at least $1-{\mathrm{ϵ}}_{\mathrm{\mathbb{P}}}$,

  $$\mid |\varphi (y)-c\mid {|}_{l^2}\mathrm{\lesssim }\delta ,\hspace{1em}\forall y=Ax+e\in \mathcal{J}(\delta ,\mathbf{s},\mathbf{M},w).$$

Balancing the Stability and Accuracy Trade-Off

Current DL methods for image reconstruction can be unstable in the sense that 1) a tiny perturbation, in either the image or the sampling domain, can cause severe artifacts in the reconstructed image (Fig. 2, Top row) and/or 2) a tiny detail in the image domain might be washed out in the reconstructed image (lack of accuracy), resulting in potential false negatives. Inevitably, there is a stability–accuracy trade-off for this type of linear inverse problem, making it impossible for any reconstruction method to become arbitrarily stable without sacrificing accuracy or vice versa. Here, we show that the NNs computed by our algorithm (FIRENETs) are stable with respect to adversarial perturbations and accurate for images that are sparse in wavelets (cf. Theorem 4). As most images are sparse in wavelets, these networks also show great generalization properties to unseen images.

Adversarial Perturbations for AUTOMAP and FIRENETs.

Fig. 2 (Top row) shows the stability test, developed in ref. 19, applied to the automated transform by manifold approximation (AUTOMAP) (60) network used for MRI reconstruction with 60% subsampling. The stability test is run on the AUTOMAP network to find a sequence of perturbations $\left|\right|e_1|{|}_{l^2}<|\left|e_2\right|{|}_{l^2}<\left|\right|e_3|{|}_{l^2}$. As can be seen from Fig. 2, Top row, the network reconstruction completely deforms the image and the reconstruction is severely unstable (similar results for other networks are demonstrated in ref. 19).

In contrast, we have applied the stability test, but now for the FIRENETs reported in this paper. Fig. 2 (Bottom row) shows the results for the constructed FIRENETs, where we rename the perturbations ${\tilde{e}}_j$ to emphasize the fact that these perturbations are sought for the FIRENETs and have nothing to do with the adversarial perturbations for AUTOMAP. We now see that despite the search for adversarial perturbations, the reconstruction remains stable. The error in the reconstruction was also found to be at most of the same order of the perturbation (as expected from the stability in Theorem 3). In applying the test to FIRENETs, we tested/tuned the parameters in the gradient ascent algorithm considerably (much more so than was needed for applying the test to AUTOMAP, where finding instabilities was straightforward) to find the worst reconstruction results, yet the reconstruction remained stable. Note also that this is just one form of stability test and it is likely that there are many other tests for creating instabilities for NNs for inverse problems. This highlights the importance of results such as Theorem 3, which guarantees stability regardless of the perturbation.

To demonstrate the generalization properties of our NNs, we show the stability test applied to FIRENETs for a range of images in SI Appendix. This shows stability across different types of images and highlights that conditions such as Definition 2 allow great generalization properties.

Stabilizing Unstable NNs with FIRENETs.

Our NNs also act as a stabilizer. For example, Fig. 3 shows the adversarial example for AUTOMAP (taken from Fig. 2), but now shows what happens when we take the reconstruction from AUTOMAP as an input to our FIRENETs. Here we use the fact that we can view our networks as approximations of unrolled and restarted iterative methods, allowing us to use the output of AUTOMAP as an additional input for the reconstruction. We see that FIRENETs fix the output of AUTOMAP and stabilize the reconstruction. Moreover, the full concatenation itself of the networks remains stable to adversarial attacks.

The Stability vs. Accuracy Trade-Off and False Negatives.

It is easy to produce a perfectly stable network: The zero network is the obvious candidate! However, this network would obviously have poor performance and produce many false negatives. The challenge is to simultaneously ensure performance and stability. Fig. 4 highlights this issue. Here we have trained two NNs to recover a set of ellipses images from noise-free and noisy Fourier measurements. The noise-free measurements are generated as y = Ax, where $A\in {ℂ}^{m\times N}$ is a subsampled discrete Fourier transform, with $m/N=0.15$ and $N=1,{024}^2$. The noisy measurements are generated as $y=Ax+ce$, where A is as before, and the real and imaginary components of $e\in {ℂ}^m$ are drawn from a zero mean and unit variance normal distribution $\mathcal{N}(0,1)$, and $c\in \mathbb{R}$ is drawn from the uniform distribution $\text{Unif}([0,100])$. The noise $ce\in {ℂ}^m$ is generated on the fly during the training process.

The trained networks use a standard benchmarking architecture for image reconstruction and map $y\mapsto \varphi (A^{*}y)$, where $\varphi :{ℂ}^N\to {\mathbb{R}}^N$ is a trainable U-net NN (8, 61). Training networks with noisy measurements, using for example this architecture, have previously been used as an example of how to create NNs that are robust toward adversarial attacks (62). As we can see from Fig. 4 (Bottom row) this is the case, as it does indeed create a NN that is stable with respect to worst-case perturbations. However, a key issue is that it is also producing false negatives due to its inability to reconstruct details. Similarly, as reported in the 2019 FastMRI challenge, trained NNs that performed well in terms of standard image quality metrics were prone to false negatives: They failed to reconstruct small, but physically relevant image abnormalities (25). Pathologies, generalization, and AI-generated hallucinations were subsequently a focus of the 2020 challenge (26). FIRENET, on the other hand, has a guaranteed performance (on images approximately sparse in wavelet bases) and stability, given specific conditions on the sampling procedure. The challenge is to determine the optimal balance between accuracy and stability, a well-known problem in numerical analysis.

Concluding Remarks

• 1)(Algorithms may not exist—Smale’s 18th problem) There are well-conditioned problems where accurate NNs exist, but no algorithm can compute them. Understanding this phenomenon is essential to addressing Smale’s 18th problem on the limits of AI. Moreover, limitations established in this paper suggest a classification theory describing the conditions needed for the existence of algorithms that can compute stable and accurate NNs (remark 5).
• 2)(Classifications and Hilbert’s program) The strong optimism regarding the abilities of AI is comparable to the optimism surrounding mathematics in the early 20th century, led by D. Hilbert. Hilbert believed that mathematics could prove or disprove any statement and, moreover, that there were no restrictions on which problems could be solved by algorithms. Gödel (36) and Turing (37) turned Hilbert’s optimism upside down by their foundational contributions establishing impossibility results on what mathematics and digital computers can achieve.

  Hilbert’s program on the foundations of mathematics led to a rich mathematical theory and modern logic and computer science, where substantial efforts were made to classify which problems can be computed. We have sketched a similar program for modern AI, where we provide certain sufficient conditions for the existence of algorithms to produce stable and accurate NNs. We believe that such a program on the foundations of AI is necessary and will act as an invaluable catalyst for the advancement of AI.

• 3)(Trade-off between stability and accuracy) For inverse problems there is an intrinsic trade-off between stability and accuracy. We demonstrated NNs that offer a blend of both stability and accuracy, for the sparsity in levels class. Balancing these two interests is crucial for applications and will no doubt require a myriad of future techniques to be developed.
• 4)Tracing out the optimal stability vs. accuracy trade-off remains largely an open problem and depends on several factors such as the model class one wishes to recover, the error tolerance of the application, and the error metric used. We have shown stability and accuracy results in the l² norm, since it is common in the literature to measure noise via this norm. We expect a program quantifying the stability and accuracy trade-off to be of particular relevance in the increasing number of real-world implementations of machine learning in inverse problems.
• 5)(Inverse problems vs. classification problems) The mathematical techniques used in this paper are applied to inverse problems. However, the mathematical framework of ref. 33 can be used to produce similar impossibility results for computing NNs in classification problems (63).
• 6)(Future work—Which NNs can be computed?) There is an enormous literature (29, 30, 64–66) on the existence of NNs with great approximation qualities. However, Theorem 2 shows that only certain accuracy may be computationally achievable. Our results are just the beginning of a mathematical theory studying which NNs can be computed by algorithms. This opens up for a theory covering other sufficient (and potentially necessary) conditions guaranteeing stability and accuracy and extensions to other inverse problems such as phase retrieval (67, 68). One can also prove similar computational barriers in other settings via the tools developed in this paper.

Methods: The Solvability Complexity Index Hierarchy

Our proof techniques for fundamental barriers in Theorem 2 stem from the mathematics behind the solvability complexity index (SCI) hierarchy (33, 69–78). The SCI hierarchy generalizes the fundamental problems of Smale (79, 80) on existence of algorithms and work by McMullen (81) and Doyle and McMullen (82). We extend and refine these techniques, in particular those of ref. 33, and generalize the mathematics behind the extended Smale’s ninth problem (33, 34)—which also builds on the SCI hierarchy. More precisely, to prove our results we develop the concept of sequential general algorithms. General algorithms are a key tool in the mathematics of the SCI hierarchy. Sequential general algorithms extend this concept and capture the notion of adaptive and/or probabilistic choice of training data. The architectures of the NNs in Theorem 3 are based on unrolled primal–dual iterations for $(P_3)$. In addition to providing stability, the wrNSPL allows us to prove exponential convergence through a careful restarting and reweighting scheme. Full theoretical derivations are given in SI Appendix.

Data Availability

All the code and data used to produce the figures in this paper are available from GitHub, https://www.github.com/Comp-Foundations-and-Barriers-of-AI/firenet.