Fast multiscale reconstruction for Cryo-EM

Abstract

We present a multiscale reconstruction framework for single-particle analysis (SPA). The representation of three-dimensional (3D) objects with scaled basis functions permits the reconstruction of volumes at any desired scale in the real-space. This multiscale approach generates interesting opportunities in SPA for the stabilization of the initial volume problem or the 3D iterative refinement procedure. In particular, we show that reconstructions performed at coarse scale are more robust to angular errors and permit gains in computational speed. A key component of the proposed iterative scheme is its fast implementation. The costly step of reconstruction, which was previously hindering the use of advanced iterative methods in SPA, is formulated as a discrete convolution with a cost that does not depend on the number of projection directions. The inclusion of the contrast transfer function inside the imaging matrix is also done at no extra computational cost. By permitting full 3D regularization, the framework is by itself a robust alternative to direct methods for performing reconstruction in adverse imaging conditions (e.g., heavy noise, large angular misassignments, low number of projections). We present reconstructions obtained at different scales from a dataset of the 2015/2016 EMDataBank Map Challenge. The algorithm has been implemented in the Scipion package.

1. Introduction

Single-particle electron microscopy aims at characterizing the three-dimensional (3D) structure of proteins at the atomic level (Dubochet et al., 1988, Frank, 2006, Orlova and Saibil, 2011, Milne et al., 2013, Cheng et al., 2015, Fernandez-Leiro and Scheres, 2016). It takes advantage of cryo-electron microscopy (cryo-EM) to image, with nearly parallel electron rays and at cryogenic temperatures, the projection of numerous replicates of a macromolecule, each with its unknown orientation and position. After data acquisition, one produces a high-resolution 3D reconstruction by processing the large set of projection measurements with advanced algorithms, available from a variety of single-particle analysis (SPA) packages (Frank et al., 1981, Sorzano et al., 2004, Tang et al., 2007, Grigorieff, 2007, Hohn et al., 2007, Scheres, 2012, de la Rosa-Trevín et al., 2016, Punjani et al., 2017).

The reconstruction necessitates the estimation of the unknown pose of the particle replicates, which is challenging because the acquired measurements are typically extremely noisy and blurred by microscopy-related effects. Most packages perform the reconstruction task through a so-called 3D iterative-refinement procedure during which information is gradually added to a rough initial volume. In particular, the projection matching approaches refine an initial volume by alternating between reconstruction and estimation of the pose parameters (Penczek et al., 1994). The first rough estimate of the 3D structure is computed from high-SNR class averages—a complicated task in itself due to the potential conformational heterogeneity of the sample. Then, from this first reference volume, a collection of equally distributed projections is produced (reference projections) and used to estimate the projection direction of clusters of projection measurements by appropriate angular-assignment methods (Carazo et al., 2015). The process is then repeated with an increasing number of distinct projection classes until the optimization fulfills some convergence criterion.

Although multiple improvements have made 3D iterative-refinement procedures more reliable over the years (Scheres and Chen, 2012, Henderson, 2013), the overall algorithmic process remains non trivial. The presence of heavy noise on nearly identical projections, their low contrast, the conformational heterogeneity of molecular complexes, the unknown projection directions, the finite number of measurements, and the incomplete knowledge of the imaging process all cause the determination of the 3D structure to be a highly ill-posed inverse problem that may also suffer from overfitting. Moreover, the convergence of the global process depends heavily on the quality of the initial reconstruction (Sorzano et al., 2006, Henderson et al., 2012).

1.1. Reconstruction algorithms

In most instances, the reconstruction at every iteration of the refinement pipeline is carried out independently of the angular assignment. The reconstruction is usually performed with direct Fourier-reconstruction (DFR) methods based on the central-slice theorem (Penczek et al., 2004, Abrishami et al., 2015). Particularly popular are Fourier regridding methods, which use interpolation kernels in the Fourier domain to bring irregularly distributed samples onto a regular grid (Carazo et al., 2015, Abrishami et al., 2015). DFR methods have provided satisfactory results in a number of applications and their speed is a key advantage. Yet, their performance can be limited in certain adverse imaging situations.

A more sophisticated and robuster solution to the reconstruction task is to formulate it as a regularized inverse problem that is solved iteratively (Sorzano et al., 2017, Gordon et al., 1970, Marabini et al., 1998, Sigworth et al., 2010, Li et al., 2011). Some approaches also take into account the blurring of each projection by the contrast transfer function (CTF) of the microscope (Zhu et al., 1997, Penczek et al., 1997). Those iterative methods permit high-quality reconstruction but require very large computational resources, which strongly limits their applicability to SPA.

Most SPA reconstruction algorithms, whether direct or iterative, offer the possibility to adjust the resolution of the reconstructed volumes. However, to the best of our knowledge, no existing method permits reconstruction of volumes at different scales. This hinders the use of multiscale approaches that very effectively solve similar ill-posed inverse problems (Adelson et al., 1984). For the sake of clarity, we detail the benefits of multiscale approaches in a dedicated section.

1.2. Contributions

In this work, we propose a multiscale reconstruction framework for SPA. We represent 3D objects with scaled basis functions to reconstruct volumes at any desired scale in the real space. The controlled dilation of the basis functions gives us the possibility to adjust the scale of the representation to the quality of the measurements.

The reconstruction task itself is formulated as a regularized optimization that is solved iteratively. To make the use of such an iterative method finally feasible in SPA, we introduce a fast formulation for the costly step of the reconstruction. The cost of this operation does not depend on the number of projection directions, which results in a substantial speed up. Moreover, the inclusion of the CTF inside the reconstruction is done at no extra computational cost.

This multiscale reconstruction tool generates interesting opportunities for the stabilization of the initial volume problem or the 3D iterative refinement procedure. In particular, we show that reconstructions performed at coarse scale are robuster to angular errors and lead to gains in computational speed. We present reconstructions obtained at different scales from a dataset of the 2015/2016 EMDataBank Map Challenge.

The paper is structured as follows: The principles behind our multiscale framework and its relevance in the context of SPA are explained in Section 2. We detail the iterative reconstruction scheme implementing our fast multiscale framework in Section 3. The results are presented in Section 4 and discussed in Section 5.

1.3. Notations

Depending on the context, we write a continuous function $f,f(·)$ or $f\left(\mathit{x}\right)$ where $\mathit{x}=(x_1,\dots ,x_d)\in {\mathbb{R}}^d$. We shall either consider $d=3$ (objects in the spatial domain) or $d=2$ (objects in the projection domain). The spaceholder $(·)$ allows us to define mappings in a more compact way, e.g., $f(·/s)≔\mathit{x}\mapsto f(\mathit{x}/s)$. Sequences are denoted by c or $c\left[\mathit{k}\right]$ with $\mathit{k}=(k_1,\dots ,k_d)\in {\mathbb{Z}}^d$. Vectors are denoted by bold lowercase letters (e.g., $\mathbf{c}$) and matrices by bold uppercase letters (e.g., $\mathbf{H}$).

The p-norm of a vector $\mathbf{c}=({\mathrm{c}}_1,\dots ,{\mathrm{c}}_N)\in {\mathbb{R}}^N$ is defined as $\left|\right|\mathbf{c}|{|}_p={\left({\sum }_{n=1}^N|{\mathrm{c}}_n{|}^p\right)}^{\frac{1}{p}}$. In this work, we shall only consider $p=1$ and $p=2$. The spaces of finite energy sequences and functions are denoted by ${\ell }_2\left({\mathbb{Z}}^d\right)$ and $L_2\left({\mathbb{R}}^d\right)$, respectively. The continuous convolution is written as $(f\ast g)\left(\mathit{x}\right)={\int }_{{\mathbb{R}}^d}f\left(\mathit{\tau }\right)g(\mathit{x}-\mathit{\tau })\mathrm{d}\mathit{\tau }$. We make the distinction with the discrete convolution, denoted by $\left(c★d\right)\left[\mathit{k}\right]={\sum }_{\mathit{l}\in {\mathbb{Z}}^d}c\left[\mathit{l}\right]d[\mathit{k}-\mathit{l}]$. The Fourier transform of f is $\widehat{f}$. We denote the reflection of a function as $f^{\vee }\left(\mathit{x}\right)=f(-\mathit{x})$.

2. Multiscale framework

2.1. Multiscale for solving ill-posed problems

The idea behind multiscale processing is to process signals over a certain range of scales when executing multi-steps procedures. An advantage is that operations performed at coarse scale are usually robuster and permit gains in computational speed (Unser and Aldroubi, 1996). They come useful when (1) only incomplete and degraded information is available as input, and (2) a low-resolution output is acceptable for further processing. A benefit is that this robustness of the coarse initial process can positively impact the convergence of all subsequent steps in the procedure.

This observation is the motivation behind the so-called pyramid approaches (Adelson et al., 1984) that solve ill-posed optimizations iteratively using multiscale representations of volumes. Several works have successfully used pyramid processing for handling strongly ill-posed optimization problems with abundant local minima in biomedical imaging (Unser and Aldroubi, 1996, Thévenaz et al., 1998, Dengler, 1989, Desco et al., 2001). This approach has also been favorably used in alternate minimization frameworks, for example in blind deconvolution works (Fergus et al., 2006, Ruiz et al., 2015).

Multiscale-based approaches have already been used to improve angular-assignment procedures (Saad et al., 2000, Sorzano et al., 2004). In Sorzano et al. (2004) used a coarse-to-fine discrete wavelet transform to compute the correlation between the measurements and the reference projections. Indeed, it is sufficient to indicate that the projections come from different orientations when they do not match at coarse scale. If, however, they do match at coarse scale, then the analysis is pursued at finer scale. This multiscale wavelet-space matching algorithm provided a gain both in speed and in robustness for the angular-assignment procedure.

Several conditions specific to SPA further advocate for the use of pyramid-like approaches for the reconstruction itself. For example, it has been shown that the alignment of cryo-EM data can be done accurately with mere low-frequency data (Henderson et al., 2011) and that the determination of the pose parameters essentially depends on low-resolution information (Scheres and Chen, 2012). Thus, a coarse representation of volumes is more desirable at early stages of the global iterative reconstruction process. Indeed, its resolution proves sufficient for further processing while it remains robust to the incomplete information (i.e., few blurred class averages with unknown projection directions).

2.2. Proposed multiscale representation for SPA

We now describe how we represent a scaled 3D object within a generalized sampling framework (Unser, 2000). In this scheme, the discrete representation of a continuous object $f\left(\mathit{x}\right),\mathit{x}\in {\mathbb{R}}^3$, can be interpreted as the coefficients of some appropriately shifted basis functions that specify a particular reconstruction space. Simply put, the generalized sampling framework tells us how to properly characterize a continuous function with a sequence of discrete coefficients.

The important aspect in our case is to consider the scaled reconstruction space

$$V_s\left(\phi \right)=\left\{f_s\left(\mathit{x}\right)={\displaystyle \sum _{\mathit{k}\in {\mathbb{Z}}^3}}{c}_s\left[\mathit{k}\right]{\phi }_s(\mathit{x}-s\mathit{k}):c_s\in {\ell }_2\left({\mathbb{Z}}^3\right)\right\}$$ (1)

specified by the scaled basis function

$${\phi }_s\left(\mathit{x}\right)=\phi (\mathit{x}/s)\in L_2\left({\mathbb{R}}^3\right),$$ (2)

where $s>0$ denotes the scaling parameter.

The coefficient sequence $c_s$ corresponds to the discrete s-scaled representation of the object $f_s$ in the space $V_s\left(\phi \right)$. In practice, this sequence is restricted to a finite number of coefficients as the object $f_s$ is compactly supported. We write this vector of coefficients as ${\mathbf{c}}_s={\left(c_s\left[\mathit{k}\right]\right)}_{\mathit{k}\in {\mathrm{\Omega }}_{3\mathrm{D}}^s}$. Here, the set ${\mathrm{\Omega }}_{3\mathrm{D}}^s$ corresponds to the support of the coefficients required to represent the object $f_s$.

These coefficients ${\mathbf{c}}_s$ are the ones used in practice for the reconstruction procedure (see Section 3.1). Once the optimization is performed, the obtained coefficients are then re-expanded in the space $V_s\left(\phi \right)$ through (1) to obtain the scaled representation of the reconstruction $f_s$. To the best of our knowledge, such a multiscale reconstruction scheme based on generalized sampling theory has not been proposed in SPA so far.

A suitable choice for the basis function $\phi$ in (2) is the optimized Kaiser-Bessel window function (KBWF) (Lewitt, 1990, Nilchian et al., 2015) defined as

$$\phi \left(\mathit{x}\right)=\left\{\begin{array}{cc}\hfill \frac{{\left(1-{\left(\frac{\left|\right|\mathit{x}\left|\right|}{a}\right)}^2\right)}^{\frac{m}{2}}{I}_m\left(\alpha \sqrt{1-{\left(\frac{\left|\right|\mathit{x}\left|\right|}{a}\right)}^2}\right)}{I_m\left(\alpha \right)}\hfill & \hfill \mathrm{if}0⩽\left|\right|\mathit{x}\left|\right|⩽a,\hfill \\ \hfill 0\hfill & \hfill \mathrm{otherwise}\text{.}\hfill \end{array}\right)$$ (3)

The KBWF depends on three parameters: (1) the order m of the modified Bessel function $I_m$, (2) the window taper $\alpha$ which determines the shape of the KBWF, and (3) the support radius a which controls the smoothness of the KBWF. It was shown in Nilchian et al. (2015) that a KBWF represents functions very effectively when using specific parameter values (e.g., $m=2,\alpha =10.83,a=2$). Moreover, its isotropic property allows for a significant reduction in computational costs, as we shall later illustrate in Section 3.

The scaling parameter s in (2) is the central element of our multiscale representation. It dilates the basis function ${\phi }_s$ and thus controls the coarseness of the representation of f. A large value of s enforces a coarse (“low-resolution”) volume, while a small value imposes a fine (“high-resolution”) representation. We illustrate this in the Fourier-shell correlation (FSC) curves of Fig. 1, where the reconstructions become coarser as the scaling parameter increases. For the simulation, numerous (2000) noiseless projections with equi-distributed directions were produced, as to focus solely on the effect of the scaling parameter on the coarseness of the representations. Increasing the scale by factors of two gradually restricts the information by halves into the low-frequency region.

Fig. 1.

Impact of the controlled dilatation of the basis function by the scaling parameter $s>0$ on the reconstructed volume. The scale is increased from left ($s=1$) to right ($s=8$), with intermediate values $s=2$ and $s=4$. (a-d) Optimized isotropic KBWF ${\phi }_s$ (with $a=2,\alpha =10.8,m=2$) are dilated by increasing s. (e-h) Central orthogonal slices of the $(256\times 256\times 256)$ beta-galactosidase reconstructed with the proposed approach at different scales. Volumes are re-expanded on a fine grid through Eq. (1) to allow for a comparison. (i-l) Corresponding FSC curves.

Finally, the scale s also influences the size of the set of coefficients ${\mathbf{c}}_s$. More precisely, the number of coefficients decreases as the scale increases. Therefore, the scaling also strongly impacts the speed of the reconstruction algorithm as the procedure is then performed on significantly less samples.

3. Fast iterative reconstruction

We detail here the iterative reconstruction scheme implementing our fast multiscale framework.

3.1. Imaging model with multiscale representation

Let ${\mathcal{P}}_{{\mathit{\theta }}_p}\left\{f\right\}\left(\mathit{y}\right)$ with $\mathit{y}\in {\mathbb{R}}^2$ denote the X-ray transform of the atomic density f for the pth 3D particle with orientation ${\mathit{\theta }}_p$ (Euler angles) (Natterer, 2001). In cryo-EM, this entity is typically blurred by a point-spread function (PSF) $w^p$. We model the noiseless 2D cryo-EM measurements ${\tilde{b}}^p\left(\mathit{y}\right)$ of the pth particle as

$${\tilde{b}}^p\left(\mathit{y}\right)=\left({\mathcal{P}}_{{\mathit{\theta }}_p}\right\{f\}\ast w^p)\left(\mathit{y}\right)\text{.}$$ (4)

We use the generalized sampling framework (1) to discretize this forward model. We thus consider the s-scaled representation $f_s$ of the atomic density of interest f. Using the linearity and pseudo-shift-invariant properties of the X-ray transform (Natterer, 2001), we rewrite (4) as

$${\tilde{b}}^p\left(\mathit{y}\right)=\left({\mathcal{P}}_{{\mathit{\theta }}_p}\right\{f_s\}\ast w^p)\left(\mathit{y}\right)$$ (5)

$$={\displaystyle \sum _{\mathit{k}\in {\mathrm{\Omega }}_{3\mathrm{D}}^s}}{c}_s\left[\mathit{k}\right]\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p\right)(\mathit{y}-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})\text{.}$$ (6)

There, the hyperplane projection matrix ${\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\in {\mathbb{R}}^{2\times 3}$ has rows that specify the normal basis of the hyperplane perpendicular to the direction ${\mathit{\theta }}_p$ of integration.

The measurements ${\tilde{b}}^p\left(\mathit{y}\right)$ for the pth particle are assumed to be acquired at the sampled points ${\mathit{y}}_{\mathit{j}}=\mathit{j}\mathrm{\Delta }$ for $\mathit{j}\in {\mathrm{\Omega }}_{2\mathrm{D}}$. Here, the set ${\mathrm{\Omega }}_{2\mathrm{D}}$ denotes the support of the projection of $f_s$. For the sake of clarity, we consider $\mathrm{\Delta }=1$ and we note ${\tilde{\mathbf{b}}}^p$ the discrete noiseless measurement vector for the pth particle. This gives us the entries of the imaging matrix ${\mathbf{H}}_s^p$ as

$${\left[{\mathbf{H}}_s^p\right]}_{\mathit{j},\mathit{k}}=\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p\right)(\mathit{j}-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})$$ (7)

with $\mathit{j}\in {\mathrm{\Omega }}_{2\mathrm{D}}$ and $\mathit{k}\in {\mathrm{\Omega }}_{3\mathrm{D}}^s$.

In practice, each measurement ${\tilde{\mathbf{b}}}^p$ is corrupted by a substantial additive Gaussian noise ${\mathbf{n}}^p$ (Sorzano et al., 2004). With this degradation, we finally obtain the discrete formulation of the complete forward model with P particles as

$$\mathbf{b}={\mathbf{H}}_s{\mathbf{c}}_s+\mathbf{n},$$ (8)

where

$$\mathbf{b}=\left[\begin{array}{c}\hfill {\mathbf{b}}^1\hfill \\ \hfill {\mathbf{b}}^2\hfill \\ \hfill ⋮\hfill \\ \hfill {\mathbf{b}}^P\hfill \end{array}\right],{\mathbf{H}}_s=\left[\begin{array}{c}\hfill {\mathbf{H}}_s^1\hfill \\ \hfill {\mathbf{H}}_s^2\hfill \\ \hfill ⋮\hfill \\ \hfill {\mathbf{H}}_s^P\hfill \end{array}\right],\mathrm{and}\mathbf{n}=\left[\begin{array}{c}\hfill {\mathbf{n}}^1\hfill \\ \hfill {\mathbf{n}}^2\hfill \\ \hfill ⋮\hfill \\ \hfill {\mathbf{n}}^P\hfill \end{array}\right]\text{.}$$ (9)

3.2. Reconstruction scheme

The task at hand is to reconstruct the scaled 3D volume $f_s$ that best explains the complete vector $\mathbf{b}$ of 2D measurements in (8). We assume here that we have existing estimates (however inaccurate they may be) of the pose ${\mathit{\theta }}_p$ and the PSF $w^p$ for every particle $p=\{1,\dots ,P\}$. In our discrete formulation, the reconstruction consists in finding the coefficients ${\stackrel{\sim }{\mathbf{c}}}_s$ through a regularized optimization scheme (Candès et al., 2006),

$${\stackrel{\sim }{\mathbf{c}}}_s=\underset{{\mathbf{c}}_s\in \mathcal{C}}{\mathrm{argmin}}\left\{\frac{1}{2}\left|\right|{\mathbf{H}}_s{\mathbf{c}}_s-\mathbf{b}|{|}_2^2+\lambda \mathrm{\Psi }({\mathbf{c}}_s)\right\},$$ (10)

where $\mathrm{\Psi }$ is the regularization term used to inject prior knowledge into the reconstruction process, while $\mathcal{C}$ is a given convex constraint (e.g., positivity constraint).

The regularized formulation given by (10) is a classical and successful way of solving ill-posed inverse problems. It promotes the solution ${\stackrel{\sim }{\mathbf{c}}}_s$ that minimizes a combination of (1) the data-fidelity term $\left|\right|{\mathbf{H}}_s{\mathbf{c}}_s-\mathbf{b}|{|}_2^2$ that forces the solution to be consistent with the imaging model ${\mathbf{H}}_s$ and the measurements $\mathbf{b}$, and (2) the regularization term $\mathrm{\Psi }\left({\mathbf{c}}_s\right)$ that requires the solution to fulfill certain priors. The parameter $\lambda >0$ controls the strength of this regularization.

We consider here that $\mathrm{\Psi }\left({\mathbf{c}}_s\right)=\left|\right|{\mathbf{Lc}}_s|{|}_1$, where $\mathbf{L}$ is some regularization matrix. We shall assume henceforth that $\mathbf{L}$ is the gradient transform. This regularization is known as total variation (TV). It is a popular edge-preserving prior that is well suited to many applications (Rudin et al., 1992).

To minimize (10), we deploy an optimization routine called the alternating-direction method of multipliers (ADMM) (Boyd et al., 2010, Ramani and Fessler, 2012). For the sake of clarity, the detailed mathematical formulation of the algorithm is given in Appendix A. A key advantage of ADMM is that it splits the original problem into simpler optimization subproblems, which accelerates the convergence of the algorithm.

In our case, the ADMM sub-solvers of (10) are given by

$${\mathbf{c}}_s^{k+1}={\mathrm{argmin}}_{{\mathbf{c}}_s}\mathcal{L}({\mathbf{c}}_s,{\mathbf{u}}^k,{\mathit{\alpha }}^k)$$ (11)

$${\mathbf{u}}^{k+1}={\mathrm{argmin}}_{\mathbf{u}}\mathcal{L}({\mathbf{c}}_s^{k+1},\mathbf{u},{\mathit{\alpha }}^k)$$ (12)

$${\mathit{\alpha }}^{k+1}={\mathit{\alpha }}^k+\mu ({\mathbf{Lc}}_s^{k+1}-{\mathbf{u}}^{k+1}),$$ (13)

where $\mathbf{u}$ is an auxiliary variable, $\mathit{\alpha }$ is the vector of Lagrange multipliers, and the function $\mathcal{L}$ corresponds to the augmented Lagrangian of (10) given by

$$\mathcal{L}\left({\mathbf{c}}_s,\mathbf{u},\mathit{\alpha }\right)=\frac{1}{2}\left|\right|{\mathbf{H}}_s{\mathbf{c}}_s-\mathbf{b}|{|}_2^2+\lambda |\left|\mathbf{u}\right|{|}_1+{\mathit{\alpha }}^T({\mathbf{Lc}}_s-\mathbf{u})+\frac{\mu }{2}\left|\right|{\mathbf{Lc}}_s-\mathbf{u}|{|}_2^2\text{.}$$ (14)

The solution of (10) is therefore obtained by alternating between the minimizations (11), (12), (13) for a given number of iterations.

Eq. (12) admits a fast explicit solution through a soft-thresholding operation (Combettes et al., 2011):

$${\mathbf{u}}^{k+1}=\mathrm{sign}\left({\mathbf{Lc}}_s^{k+1}-{\mathit{\alpha }}^k\right)·\max \left(\mathbf{0},|{\mathbf{Lc}}_s^{k+1}-{\mathit{\alpha }}^k|-(\lambda /\mu )\right)\text{.}$$ (15)

Likewise, (13) is a simple parameter update. On the other hand, to compute the coefficient ${\mathbf{c}}_s^{k+1}$ in (11), we have to minimize the augmented Lagrangian $\mathcal{L}\left({\mathbf{c}}_s,\mathbf{u},\mathit{\alpha }\right)$, whose gradient with respect to ${\mathbf{c}}_s$ is

$${\nabla }_{{\mathbf{c}}_s}\mathcal{L}({\mathbf{c}}_s,{\mathbf{u}}^k,{\mathit{\alpha }}^k)=\underset{\mathbf{A}}{\underbrace{({\mathbf{H}}_s^T{\mathbf{H}}_s+\mu {\mathbf{L}}^T\mathbf{L})}}{\mathbf{c}}_s-\underset{\mathbf{d}}{\underbrace{\left({\mathbf{H}}_s^T\mathbf{b}+\mu {\mathbf{L}}^T({\mathbf{u}}^k-\frac{{\mathit{\alpha }}^k}{\mu })\right)}}\text{.}$$ (16)

We do this by solving

$${\nabla }_{{\mathbf{c}}_s}\mathcal{L}\left({\mathbf{c}}_s^{k+1},{\mathbf{u}}^k,{\mathit{\alpha }}^k\right)={\mathbf{Ac}}_s-\mathbf{d}=\mathbf{0},$$ (17)

iteratively using a conjugate-gradient (CG) method since the matrix $\mathbf{A}$ is badly conditioned (Boyd and Vandenberghe, 2004).

From a computational point-of-view, this minimization is the real bottleneck of the global optimization scheme. Indeed, it imposes the matrix-multiplication of ${\mathbf{H}}_s^T{\mathbf{H}}_s$ with the updated ${\mathbf{c}}_s$ vector at every ADMM iteration. If not carefully engineered, this operation comes at a heavy computational cost and makes the use of iterative algorithms out of practical reach for SPA.

3.3. Fast implementation of ${\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s$

In this section, we provide an alternate mathematical formulation of ${\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s$ that greatly reduces the computational cost of the operation. In our opinion, this contribution is by itself significant in the context of SPA, as it makes the use of advanced iterative algorithms conceivable in the field.

As a preliminary, let us define a condition on basis functions that is required for further calculations.

Definition 1 Radial Nyquist criterion —

The function $\phi \in L_2\left({\mathbb{R}}^3\right)$ satisfies the radial Nyquist criterion with respect to the grid ${\mathbb{Z}}^3$ if $\widehat{\phi }\left(\mathit{\omega }\right)=0$ for all $\left|\right|\mathit{\omega }\left|\right|⩾\pi$.

A function therefore satisfies the radial Nyquist criterion (RNC) if its Fourier transform is zero outside of a ball of radius $\pi$. Note that, if ${\phi }_{s_0}\left(\mathit{x}\right)=\phi (\mathit{x}/s_0)$ satisfies the RNC, then so does ${\phi }_s\left(\mathit{x}\right)=\phi (\mathit{x}/s)$ for all $s⩾s_0$.

We then have the result of Theorem 1 on the fast computation of ${\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s$ for objects that lie in the 3D space.

Theorem 1

Let ${\phi }_s\left(\mathit{x}\right)=\phi (\mathit{x}/s)$, with $\mathit{x}\in {\mathbb{R}}^3$ and $s>0$, be such that it satisfies the radial Nyquist criterion for all $s⩾s_0$. Moreover, let the imaging matrix ${\mathbf{H}}_s$ be as defined in (9) and $P\in {\mathbb{N}}^{\ast }$ denote the number of particles. Then, for all $s⩾s_0$, the discrete product ${\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s$ can be computed as the discrete convolution

$${\left[{\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s\right]}_{\mathit{k}}=\left(c_s★r_s\right)\left[\mathit{k}\right]$$ (18)

for $\mathit{k}\in {\mathrm{\Omega }}_{3\mathrm{D}}^s$, with kernel

$$r_s\left[\mathit{k}\right]=\left|s{|}^6{\displaystyle \sum _{p=1}^P}\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{\phi \right\}\ast {\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }^{\vee }\right\}\ast q_{1/s}^p\right)\right({\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})\text{.}$$ (19)

Here, the function $q_{1/s}^p\left(\mathit{y}\right)=(w^p\ast {\left(w^p\right)}^{\vee })\left(s\mathit{y}\right)$ with $\mathit{y}\in {\mathbb{R}}^2$ corresponds to the scaled auto-correlation function of the PSF $w^p\left(\mathit{y}\right)$.

The proof of Theorem 1 is given in Appendix B.

The benefit is that the costly step ${\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s$ can now be quickly computed as a pointwise multiplication in the Fourier domain, with a cost that does not depend on the number of projection directions.¹ Moreover, if the basis function ${\phi }_s$ is isotropic, then the costs are further reduced as the autocorrelation of ${\mathcal{P}}_{{\mathit{\theta }}_p}\left\{\phi \right\}$ needs to be computed only once.

In practice, the discrete convolution $\left(c_s★r_s\right)$ in (18) only needs to be computed for $\mathit{k}\in {\mathrm{\Omega }}_{3\mathrm{D}}^s$. We do this by convolving a padded ${\mathbf{c}}_s$ with a kernel of finite support. This finite kernel ${\mathbf{r}}_s$ is obtained by first convolving the autocorrelation function of ${\mathcal{P}}_{{\mathit{\theta }}_p}\left\{\phi \right\}$ with the scaled autocorrelation function of the PSF $w^p$, and then interpolating its value in the sampling points ${\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k}$.

Note that if no PSF is considered (i.e., $w^p$ is the Dirac distribution $\delta$ for all particles), the kernel reduces to

$$r_s\left[\mathit{k}\right]=\left|s{|}^4{\displaystyle \sum _{p=1}^P}\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{\phi \right\}\ast {\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }^{\vee }\right\}\right)\right({\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})\text{.}$$ (20)

The case when $w^p=\delta$ and $s=1$ (i.e., unscaled reconstruction) was used in X-ray tomography with fixed rotation axis by McCann et al. (2016).

3.4. Fast implementation of ${\mathbf{H}}_s^T\mathbf{b}$

Eq. (17) also requires the computation of the discrete product ${\mathbf{H}}_s^T\mathbf{b}$. Even though it only needs to be computed once during the whole reconstruction process, it can also be costly in its own rights. We present here its fast formulation.

Theorem 2

Let ${\phi }_s\left(\mathit{x}\right)=\phi (\mathit{x}/s)$ with $\mathit{x}\in {\mathbb{R}}^3$ and $s>0$ be such that it satisfies the radial Nyquist criterion for all $s⩾s_0$. Moreover, let the imaging matrix ${\mathbf{H}}_s^p$ be as defined in (7), the measurement vector $\mathbf{b}$ be as defined in (8), and $P\in {\mathbb{N}}^{\ast }$ denote the number of particles. Then, for all $s⩾s_0$, the matrix–vector product ${\mathbf{H}}_s^T\mathbf{b}$ can be computed as

$${\left[{\mathbf{H}}_s^T\mathbf{b}\right]}_{\mathit{k}}={\displaystyle \sum _{p=1}^P}\left(b^p\ast {\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s^{\vee }\right\}\ast {\left(w^p\right)}^{\vee }\right)\left(s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k}\right)$$ (21)

for $\mathit{k}\in {\mathrm{\Omega }}_{3\mathrm{D}}^s$.

The proof of Theorem 2 is given in Appendix C.

The product ${\mathbf{H}}_s^T\mathbf{b}$ can thus be obtained by computing the convolution in (21) on a fine grid, which significantly reduces the cost of the operation. Another benefit is that the computation of ${\mathbf{H}}_s^T\mathbf{b}$ can be easily parallelized over the set of particles.

3.5. Computational cost

We assume that the task is to reconstruct an image of size $N^3$ from a set of P measurements of size $M^2$. We also consider the support of our (unscaled) basis function to be $W^3$ and the scale of the basis function along each dimension to be s. For simplicity, we do not consider the cost of CTF inclusion here.

The computation of ${\mathbf{H}}_s^T\mathbf{b}$ through Theorem 2 requires a discrete convolution, which we perform via 2D fast Fourier transforms (FFT), with a cost in the order of $\mathcal{O}\left({\mathit{PM}}^2\log \right(M\left)\right)$. We store the result in a lookup table; then, each object point comes at a cost of $\mathcal{O}({\mathit{PN}}^3/s^3)$. The computation of ${\mathbf{H}}_s^T{\mathbf{H}}_s$ kernel through Theorem 1 requires the autocorrelation of the projection of a non-scaled basis function, which is precomputed and stored in a lookup table. Since we are using basis functions with isotropic properties, this lookup table is short. We then interpolate the kernel in the object domain, which implies a cost of $\mathcal{O}\left({\mathit{PW}}^{2}N\right)$. Finally, the convolution of the ${\mathbf{H}}_s^T{\mathbf{H}}_s$ with the current coefficient sequence only requires two FFT. This comes at a cost of $\mathcal{O}\left(N^3\log \right(N)/s^3)$ at every ADMM iteration.²

Whether the cost of our approach is comparable to that of DFR methods depends on the specific experimental conditions, especially on the scale desired for the reconstruction. In particular, the coarser the representation of the image, the quicker the reconstruction process. For example, when the reconstruction is performed at scale $s=4$, the cost of computing ${\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s$ is reduced by a factor 64 compared to fine-scale reconstruction with $s=1$. This is a massive computational gain.

3.6. Algorithm

A pseudo-code of the proposed reconstruction algorithm is provided in the Algorithm 1 box. The algorithm has been implemented in MATLAB® and is usable in Scipion. The algorithm can handle mirror projection images and correct for shifts. Symmetry of macromolecules is also taken into account in our implementation.

As inputs, the algorithm requires the projection measurements and the scale desired for the reconstruction. Optionally, the CTF information of each particle can be given as an input. This inclusion of the CTF is relevant only for reconstructions at the finest scale ($s=1$). Indeed, reconstructions at coarser scales ($s>1$) do not require high-frequency information correction (see Fig. 1, bottom line). Another (more practical) reason for this optional inclusion is that some SPA packages pre-correct the CTF effect on the measurements prior to reconstruction.

Our ADMM-based algorithm depends on three parameters: the regularization parameter $\lambda$, the number of ADMM iterations $n_A$ (i.e., the outer iterations) and the number of CG iterations $n_C$ (i.e., the inner iterations).

During our experiments, we have observed that the choice of these parameters tended to be very robust for coarse scales ($s>1$). A wide interval of $\lambda$ was thus yielding similar and satisfactory results; From those, we have selected a default value of $\lambda =100$. Note that this value may vary to some degree from a dataset to another. The number of inner and outer iterations also required little tuning. Throughout our experiments, we have used $n_A=30$ and $n_C=7$ for all scales.

Algorithm 1: Fast iterative reconstruction at scale s

At finest scale ($s=1$), an appropriate value of $\lambda$ can be found by running the thirty ADMM iterations with $\lambda$ spanning a range of powers of ten (e.g., five values from $\lambda ={10}^{-2}$ to $\lambda ={10}^2$). To do so, one does not need to recompute the kernel ${\mathbf{r}}_s$, nor the product ${\mathbf{H}}_s^T\mathbf{b}$, which makes the cost of this search acceptable. Finally, the penalty parameter $\mu$ in (14) is set proportional to $\lambda$ and thus need not be tuned.

4. Experiments

We have evaluated the performance of our multiscale algorithm on both simulated and real datasets. In particular, we have explored how the scaling was impacting the quality of the reconstruction and influencing its robustness to errors on angular misalignments.

The experiments performed with simulated data are described in Sections 4.1 and 4.2. We present the results obtained with real data from the 2015/16 EMDataBank Map Challenge in Section 4.3.

4.1. Simulation conditions

For our simulations, we used as ground-truth a $(128\times 128\times 128)$ $\beta$-galactosidase volume with $2.2\AA$ resolution (Bartesaghi et al., 2014). We started by computing 20,000 randomly equi-distributed projections of the ground-truth (Deserno, 2004). We used the imaging matrix given by (7) with a KBWF as basis function ($a=2,\alpha =10.8,m=2$) to compute these projections. Multiple experimental sets of data were generated by adding (1) Gaussian noise on the projection images such that their SNR is 1 and (2) angular error on their projection angles.

For every dataset, we clustered the projections in N distinct classes. The class representatives were N uniformly equi-distributed projections of the ground-truth (Deserno, 2004). Each projection was assigned to the class with the closest projection angles. Then, for every cluster, the projections were aligned to the reference image (by rigid transformation, using imregister in MATLAB®) and averaged. From these class averages, we reconstructed the 3D structures at different scales with our algorithm. Small cluster sizes (from 10 to 100) were considered, which mimics the processing conditions during the early refinement stages.

For the reconstruction, we used a different KBWF as basis function ($a=4,\alpha =19,m=2$) to reduce the risks of inverse crime.³ We applied TV regularization and performed our optimization with thirty outer iterations and seven inner iterations. The regularisation parameter was selected from five powers of 10 by picking the best output in terms of FSC.

We did not consider CTF correction in those simulations to permit direct comparison among reconstructions obtained at different scales (see also Section 3.6). Note that the beneficial impact of CTF inclusion for producing high-resolution reconstructions has been established in previous works (Zhu et al., 1997, Penczek et al., 1997).

The accuracy of each reconstruction was evaluated by computing the FSC metric with respect to the ground-truth, as is the standard in SPA. When necessary, we considered a common threshold value at $0.5$.

4.2. Robustness to angular misassignments

We have analyzed with simulated data how angular misassignments were influencing the quality of the reconstruction at scale $s=1,2,3,4$. The obtained results are presented in Fig. 2 and Fig. 3.

Fig. 2.

Effect of the scale on the robustness to angular misassignment of $\beta$-galactosidase reconstruction. The reported resolution is the FSC resolution estimate at the threshold value of $0.5$.

Fig. 3.

FSC curves of $\beta$-galactosidase reconstructed at scale 3 from 20,000 projections that were clustered in a varying number of classes (n = 11,31,59,99) before averaging. The variance of the error on the projection angles is $15.47$ degrees.

In Fig. 2, an increasing amount of error was added to the projection angles prior to clustering in 80 equi-distributed classes and averaging. Reconstruction was then performed at scale $s=1,2,3,4$ and the FSC at threshold value $0.5$ was returned. The results indicate that robustness to angular misassignments becomes stronger as the scale is coarsened. Indeed, although reconstructing at fine scale ($s=1$) performs effectively when the error level is low, its performance quickly degrades when the angular error increases. This behavior is much less pronounced at coarser scales ($s=2,3,4$), which show better stability against angular errors.

We have also explored how the choice of the scale was influencing the quality of the reconstruction when only a very limited amount of data (i.e., 11 projection classes) was available. The results are presented in Fig. 3. There, a fixed amount of error (with var=$15.47$ degrees) was added to the projection angles prior to clustering. The results show that, when performing reconstruction with very few data, all output volumes ($s=1,2,3,4$) have a roughly similar information content. This is not surprising: At all scales, there is only a limited resolution that can be achieved with such blurred and incomplete data.

Overall, those results suggest that, when the error on the estimated angles is significant and the number of projection classes is low, one would actually benefit from reconstructing volumes at a coarser scales. Then, the frequency content is preserved and the reconstruction is robuster to angular misassignements. Moreover, the gain in computational speed at such scales can be substantial (see Section 3.5).

4.3. Real data from the EMDataBank map challenge

We have used our multiscale algorithm to reconstruct a real target from the EMDataBank Map Challenge: the T20S Proteasome. A total of 22,884 projection images were extracted from 196 micrographs. The experimental details of the acquisition are given in Campbell et al. (2015). We classified these images into 32 classes using CL2D Sorzano et al., 2010 and constructed an initial volume assuming a D7 symmetry with the algorithm described in Sorzano et al. (2015). We then applied five iterations of the reconstruct_highres protocol in Scipion, using the default reconstruction method. The parameters used for running those five iterations are described in a paper in this same issue (Sorzano et al., 2018).

We then performed a final iteration of the reconstruct_highres in Scipion using our multiscale reconstruction method with scales $s=1,2,3,4$. For each scale, reconstruct_highres automatically separated the projection set in two halves and performed one iteration of angular assignment and reconstruction on each half.

The reconstruct_highres algorithm was run with its default parameters in Scipion, except for the Post-Processing options that were all disabled. For the ADMM-based reconstructions, we used a unique set of parameters for all scales: $\lambda =100,n_A=30,n_c=7$. We applied TV regularization and imposed a positivity constraint during reconstruction. We did not apply any post-processing operation (e.g., denoising) after reconstruction. The results are displayed in Fig. 4, Fig. 5, Fig. 6, Fig. 7.

Fig. 4.

Reconstructions at different scales ($s=1,2,3,4$) of the T20S Proteasome from the 2015/2016 EMDatabank Challenge.

Fig. 5.

FSC curves for the T20S Proteasome reconstructions at different scales ($s=1,2,3,4$). For each scale, the curve was obtained by comparing reconstructions from two distinct halves of the projection set.

Fig. 6.

Cross sections of T20S Proteasome reconstructions at different scales ($s=1,2,3,4$). Slices number from top row to bottom row: 75, 100, 125, 150, 175, 200. The yellow lines (top row) indicate the position where the intensity profiles displayed in Fig. 7 are measured.

Fig. 7.

Profile lines taken on a cross-section of the T20S Proteasome reconstructions at different scales ($s=1,2,3,4$). The position of the measured pixels is indicated by the yellow lines in Fig. 6.

Fig. 4 presents the reconstructions obtained at the different scales. Several cross sections from these reconstructions are displayed in Fig. 6. These qualitative results illustrate how the scaling influences the coarseness of the reconstructions while preserving their key structural features.

FSC curves of the reconstructed volumes are presented in Fig. 5. For each scale, the curve was obtained by comparing the reconstructions from the two distinct halves of the total projection set. Those FSC results testify to the considerable robustness achieved by coarse reconstructions whose curves are almost constantly equal to one.

Finally, profile lines taken on a cross-section of the T20S Proteasome reconstructions at different scales are given in Fig. 7.

5. Discussion

Those results obtained on simulated and real data confirm the increase in robustness brought by reconstructing volumes at coarser scales. This is consistent with observations made by other multiscale approaches in various biomedical applications (Unser and Aldroubi, 1996).

For SPA, the benefits of a multiscale tool become clear when considering the initial volume problem. This problem refers to the computation of the first estimation of the 3D structure required for progressive refinement. The task is a highly challenging one, as the lack of angular information for performing reconstruction is then at its peak. There are thus abundant local minima that algorithms can be trapped into. Yet the importance of a robust first estimation cannot be overstated. Several works have indeed demonstrated that the nature of this initial structure can considerably affect the final reconstruction (Henderson et al., 2012, Sorzano et al., 2006).

In that sense, our multiscale reconstruction scheme might provide novel ways of stabilizing this highly ill-posed optimization task. A judicious approach could be to start the process with rather coarse reconstructions, which are more robust to error on projections angles and yet contain all necessary information for angular estimation (Henderson et al., 2011, Scheres and Chen, 2012). Then, one could repeat the process by slowly increasing the scale as the angular assignment is refined. Work is currently underway to quantify the increase of robustness brought by multiscale reconstruction to this initial volume problem.

It is essential to realize that the use of the proposed iterative method in SPA was made entirely possible by the fast formulation of our algorithm. Without these novel mathematical contributions, the applicability of our framework in single-particle electron microscopy would have been rather limited, as is currently that of most iterative reconstruction techniques.

In addition to its multiscale aspect, a key feature of our framework is its ability to inject prior information into the optimization. Through its regularization term, our scheme can be a reliable alternative for handling these reconstructions for which direct methods might fail to yield satisfactory results.

Finally, the proposed scheme opens the door for several other developments in SPA, such as the inclusion of novel constraints, a different handling of specimens with M-fold symmetries and the use of promising learning-based approaches (Tosic and Frossard, 2011, Jin et al., 2017).

6. Conclusion

We have presented a novel multiscale reconstruction framework for single-particle electron microscopy. By appropriately representing three-dimensional (3D) objects with scaled basis functions, one can now reconstruct volumes at any desired scale in the real space. To make the use of this iterative reconstruction scheme in single-particle analysis (SPA) feasible, we have introduced a fast formulation for the iterative refinement step. The costly step of the reconstruction, which was previously hindering the use of advanced iterative methods in SPA, can now be computed as a discrete convolution with cost independent upon the number of projection directions. This multiscale reconstruction tool was evaluated on both simulated and real data. In both cases, results have highlighted the increase in robustness brought by reconstructing volumes at coarser scales.

Appendix A. ADMM-CG algorithm

We formulate our reconstruction task as

$${\stackrel{\sim }{\mathbf{c}}}_s=\underset{{\mathbf{c}}_s\in \mathcal{C}}{\mathrm{argmin}}\left\{\frac{1}{2}\left|\right|{\mathbf{H}}_s{\mathbf{c}}_s-\mathbf{b}|{|}_2^2+\lambda |\left|{\mathbf{Lc}}_s\right|{|}_1\right\},$$ (A.1)

where $\lambda$ controls the strength of the regularization. We define the auxiliary variable $\mathbf{u}=\mathbf{L}{\mathbf{c}}_s$ and rewrite the optimization problem as the constrained optimisation problem (Ramani and Fessler, 2012, Boyd et al., 2010),

$$\mathcal{J}\left({\mathbf{c}}_s\right)={\displaystyle \underset{{\mathbf{c}}_s\in {\mathbf{R}}^n,\mathbf{u}=\mathbf{L}{\mathbf{c}}_s}{\min }}\left\{\frac{1}{2}\left|\right|{\mathbf{H}}_s{\mathbf{c}}_s-\mathbf{b}|{|}_2^2+\lambda |\left|\mathbf{u}\right|{|}_1\right\}\text{.}$$ (A.2)

Its scaled augmented Lagrangian functional is given by

$${\mathcal{L}}_{\mu }\left({\mathbf{c}}_s,\mathbf{u},\mathit{\alpha }\right)=\frac{1}{2}\left|\right|{\mathbf{Hc}}_s-\mathbf{b}|{|}_2^2+\lambda |\left|\mathbf{u}\right|{|}_1+{\mathit{\alpha }}^T({\mathbf{Lc}}_s-\mathbf{u})+\frac{\mu }{2}\left|\right|{\mathbf{Lc}}_s-\mathbf{u}|{|}_2^2,$$ (A.3)

where $\mathit{\alpha }$ is the vector of Lagrange multipliers. ADMM is used to separate the optimization problem into simpler ones (Ramani and Fessler, 2012, Boyd et al., 2010),

$$\left\{\begin{array}{cc}\hfill {\mathbf{c}}_s^{k+1}=\underset{{\mathbf{c}}_s}{\text{argmin}}\mathcal{L}({\mathbf{c}}_s,{\mathbf{u}}^k,{\mathit{\alpha }}^k),\hfill & \hfill \left(a\right)\hfill \\ \hfill {\mathbf{u}}^{k+1}=\underset{\mathbf{u}}{\text{argmin}}\mathcal{L}({{\mathbf{c}}_s}^{k+1},\mathbf{u},{\mathit{\alpha }}^k),\hfill & \hfill \left(b\right)\hfill \\ \hfill {\mathit{\alpha }}^{k+1}={\mathit{\alpha }}^k+\mu ({\mathbf{Lc}}_s^{k+1}-{\mathbf{u}}^{k+1}),\hfill & \hfill \left(c\right)\text{.}\hfill \end{array}\right)$$ (A.4)

Eq. A.4(a) is a quadratic minimization with respect to ${\mathbf{c}}_s$ with gradient

$$\nabla {\mathcal{L}}_{{\mathbf{c}}_s}({\mathbf{c}}_s,{\mathbf{u}}^k,{\mathit{\alpha }}^k)=({\mathbf{H}}_s^T{\mathbf{H}}_s+\mu {\mathbf{L}}^T\mathbf{L}){\mathbf{c}}_s-({\mathbf{H}}_s^T\mathbf{b}+\mu {\mathbf{L}}^T({\mathbf{u}}^k-\frac{{\mathit{\alpha }}^k}{\mu }\left)\right)\text{.}$$ (A.5)

The critical element of the cost functional is the root of the gradient function, which we find by using a conjugate-gradient method.

Eq. A.4(b) admits the fast explicit solution

$${\mathbf{u}}^{k+1}={\mathrm{prox}}_{\frac{\lambda }{\mu }}({\mathbf{Lc}}_s^{k+1}-{\mathit{\alpha }}^k),$$ (A.6)

where the proximal operator corresponds to a soft-thresholding operation (Combettes et al., 2011). The solution of A.4(b) is thus obtained with

$${\mathbf{u}}^{k+1}=\mathrm{sign}\left({\mathbf{Lc}}_s^{k+1}-{\mathit{\alpha }}^k\right)·\max \left(\mathbf{0},|{\mathbf{Lc}}_s^{k+1}-{\mathit{\alpha }}^k-(\lambda /\mu )\right)\text{.}$$ (A.7)

Finally, A.4(c) corresponds to a simple update of the Lagrange parameter.

Appendix B. Proof of Theorem 1

As a preliminary, we provide in Proposition 3 a result on functions that satisfy the radial Nyquist criterion. This proposition will be needed in further proofs.

Proposition 3

For any pair of functions $(f,g)$ that satisfy the radial Nyquist criterion, it holds that

$${\displaystyle \sum _{\mathit{n}\in {\mathbb{Z}}^2}}f\left(\mathit{n}\right)g\left(\mathit{n}\right)={\int }_{{\mathbb{R}}^2}f\left(\mathit{x}\right)g\left(\mathit{x}\right)\mathrm{d}\mathit{x},$$ (B.1)

where $\mathit{n}=(n_1,n_2)$ and $\mathit{x}=(x_1,x_2)$.

proof. Since the functions f and g satisfy the radial Nyquist criterion, we can apply Shannon’s theorem and expand them using the $\mathrm{sinc}$ functions to obtain that

$$f\left(\mathit{x}\right)={\displaystyle \sum _{\mathit{k}}}f\left(\mathit{k}\right)\mathrm{sinc}(\mathit{x}-\mathit{k}),$$ (A.2)

$$g\left(\mathit{x}\right)={\displaystyle \sum _{\mathit{k}}}g\left(\mathit{k}\right)\mathrm{sinc}(\mathit{x}-\mathit{k}),$$ (A.3)

where $\mathrm{sinc}\left(\mathit{x}\right)=\mathrm{sinc}\left(x_1\right)\mathrm{sinc}\left(x_2\right)$. The orthonormality of the $\mathrm{sinc}$ function and its shifts yields the desired results.

We now provide the proof of Theorem 1 for objects that lie in a 3D space. We recall that the vector ${\mathbf{c}}_s$ is defined as ${\mathbf{c}}_s={\left(c_s\left[\mathit{k}\right]\right)}_{\mathit{k}\in {\mathrm{\Omega }}_{3\mathrm{D}}^s}$, where $c_s$ is the sequence of coefficients and the set ${\mathrm{\Omega }}_{3\mathrm{D}}^s$ corresponds to the support of the coefficients required to represent the object $f_s$.

proof We rewrite for $\mathit{k}\in {\mathrm{\Omega }}_{3\mathrm{D}}^s$ the entries of the discrete product ${\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s$ as

$$\begin{array}{cc}\hfill \hfill & \hfill {\left[{\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s\right]}_{\mathit{k}}={\left[{\displaystyle \sum _{p=1}^P}{\left({\mathbf{H}}_s^p\right)}^T{\mathbf{H}}_s^p{\mathbf{c}}_s\right]}_{\mathit{k}}\hfill \\ \hfill \hfill & \hfill \begin{array}{cc}\hfill {\displaystyle \underset{=}{\left(i\right)}}\hfill & \hfill {\sum }_{p=1}^P{\displaystyle \sum _{\mathit{l}\in {\mathbb{Z}}^3}}{c}_s\left[\mathit{l}\right]{\displaystyle \sum _{\mathit{j}\in {\mathbb{Z}}^2}}\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p\right)(\mathit{j}-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{l})\hfill \\ \hfill \hfill & \hfill \times \left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p\right)(\mathit{j}-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})\hfill \end{array}\hfill \\ \hfill \hfill & \hfill \begin{array}{cc}\hfill {\displaystyle \underset{=}{\left(\mathit{ii}\right)}}\hfill & \hfill {\sum }_{p=1}^P{\displaystyle \sum _{\mathit{l}\in {\mathbb{Z}}^3}}{c}_s\left[\mathit{l}\right]{\int }_{{\mathbb{R}}^2}\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p\right)(\mathit{y}-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})\hfill \\ \hfill \hfill & \hfill \times \left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p\right)(\mathit{y}-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{l})\mathrm{d}\mathit{y}\hfill \end{array}\hfill \\ \hfill \hfill & \hfill \begin{array}{cc}\hfill {\displaystyle \underset{=}{\left(\mathit{iii}\right)}}\hfill & \hfill {\sum }_{p=1}^P{\displaystyle \sum _{\mathit{l}\in {\mathbb{Z}}^3}}{c}_s\left[\mathit{l}\right]{\int }_{{\mathbb{R}}^2}\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p\right)\left(\stackrel{\sim }{\mathit{y}}\right)\hfill \\ \hfill \hfill & \hfill \times \left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p\right)(\stackrel{\sim }{\mathit{y}}-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}(\mathit{l}-\mathit{k}\left)\right)\mathrm{d}\stackrel{\sim }{\mathit{y}}\hfill \end{array}\hfill \\ \hfill \hfill & \hfill \begin{array}{cc}\hfill {\displaystyle \underset{=}{\left(\mathit{iv}\right)}}\hfill & \hfill {\sum }_{p=1}^P{\displaystyle \sum _{\mathit{l}\in {\mathbb{Z}}^3}}{c}_s\left[\mathit{l}\right]\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p\ast {\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p\right)}^{\vee }\right)\left(s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\right(\mathit{l}-\mathit{k}\left)\right)\hfill \end{array}\hfill \\ \hfill \hfill & \hfill \begin{array}{cc}\hfill {\displaystyle \underset{=}{\left(v\right)}}\hfill & \hfill {\sum }_{\mathit{l}\in {\mathbb{Z}}^3}{c}_s\left[\mathit{l}\right]{\displaystyle \sum _{p=1}^P}\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast {\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s^{\vee }\right\}\ast w^p\ast {\left(w^p\right)}^{\vee }\right)\left(s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\right(\mathit{l}-\mathit{k}\left)\right)\text{.}\hfill \end{array}\hfill \end{array}$$ (A.4)

We thus have that the discrete product ${\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s$ can be computed as the discrete convolution

$${\left[{\mathbf{H}}_s^T{\mathbf{H}}_s{\mathbf{c}}_s\right]}_{\mathit{k}}=(c_s\ast r_s)\left[\mathit{k}\right],$$ (A.5)

with kernel

$$r_s\left[\mathit{k}\right]={\displaystyle \sum _{p=1}^P}\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast {\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s^{\vee }\right\}\ast q^p\right)\left(s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k}\right),$$ (A.6)

where the function $q^p\left(\mathit{y}\right)=(w^p\ast {\left(w^p\right)}^{\vee })\left(\mathit{y}\right)$ with $\mathit{y}\in {\mathbb{R}}^2$ corresponds to the autocorrelation function of the PSF $w^p\left(\mathit{y}\right)$.

Equality (i) derives from the definition of ${\mathbf{H}}_s^p$ given by (7) and from the compact support of $c_s$ and ${\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}$. Equality (ii) results from Proposition 3, which can be invoked here as the function $\left({\mathcal{P}}_{{\mathit{\theta }}_p}\right\{{\phi }_s\}\ast w^p)\left(\mathit{n}\right)$ with $\mathit{n}\in {\mathbb{Z}}^2$ verifies the radial Nyquist criterion. Indeed, as the basis function ${\phi }_s$ verifies the radial Nyquist criterion by hypothesis, ${\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}$ also satisfies the radial Nyquist criterion through the central-slice theorem, and then so does ${\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast w^p$ through the convolution theorem. Equality (iii) is obtained through a simple change of variables, while Equalities (iv) and (v) use properties of the continuous convolution.

We then rewrite the kernel $r_s\left[\mathit{k}\right]$ given by (A.6) as:

$$\begin{array}{cc}\hfill \hfill & \hfill r_s\left[\mathit{k}\right]={\displaystyle \sum _{p=1}^P}\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}\ast {\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s^{\vee }\right\}\ast q^p\right)\left(s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k}\right)\hfill \\ \hfill \hfill & \hfill \begin{array}{cc}\hfill {\displaystyle \underset{=}{\left(\mathit{vi}\right)}}\hfill & \hfill \left|s{|}^2{\displaystyle \sum _{p=1}^P}\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{\phi \right\}(·/s)\ast {\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }^{\vee }\right\}(·/s)\ast q^p(·)\right)\right(s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})\hfill \end{array}\hfill \\ \hfill \hfill & \hfill \begin{array}{cc}\hfill {\displaystyle \underset{=}{\left(\mathit{vii}\right)}}\hfill & \hfill \left|s{|}^4{\displaystyle \sum _{p=1}^P}\left(\left({\mathcal{P}}_{{\mathit{\theta }}_p}\right\{\phi \}\ast {\mathcal{P}}_{{\mathit{\theta }}_p}\{{\phi }^{\vee }\left\}\right)(·/s)\ast q_{1/s}^p(·/s)\right)\right(s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})\hfill \end{array}\hfill \\ \hfill \hfill & \hfill \begin{array}{cc}\hfill {\displaystyle \underset{=}{\left(\mathit{viii}\right)}}\hfill & \hfill \left|s{|}^6{\displaystyle \sum _{p=1}^P}\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{\phi \right\}\ast {\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }^{\vee }\right\}\ast q_{1/s}^p\right)\right({\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k}),\hfill \end{array}\hfill \end{array}$$ (A.7)

where $q_{1/s}^p\left(\mathit{y}\right)=q\left(s\mathit{y}\right)$. Equality (vi) is obtained after applying twice the scale-invariance property of the X-ray transform ($s\ne 0$)

$${\mathcal{P}}_{\mathit{\theta }}\left\{f\right(·/s\left)\right\}\left(\mathit{x}\right)=\left|s\right|·{\mathcal{P}}_{\mathit{\theta }}\left\{f\right(·\left)\right\}(\mathit{x}/s)\text{.}$$ (A.8)

Equalities (vii) and (viii) are both derived by using a well-known result on the convolution of two scaled functions ${\mathcal{P}}_{\mathit{\theta }}\left\{f\right\},{\mathcal{P}}_{\mathit{\theta }}\left\{g\right\}\in {\mathbb{R}}^2$:

$$\left({\mathcal{P}}_{\mathit{\theta }}\left\{f\right\}(·/s)\ast {\mathcal{P}}_{\mathit{\theta }}\left\{g\right\}(·/s)\right)\left(\mathit{x}\right)=\left|s\right|\left({\mathcal{P}}_{\mathit{\theta }}\left\{f\right\}\ast {\mathcal{P}}_{\mathit{\theta }}\left\{g\right\}\right)(\mathit{x}/s)\text{.}$$ (A.9)

Appendix C. Proof of Theorem 2

We provide the proof for objects that lie in a 3D space.

proof We rewrite for $\mathit{k}\in {\mathrm{\Omega }}_{3\mathrm{D}}^s$ the entries of the product ${\mathbf{H}}_s^T\mathbf{b}$ as:

$$\begin{array}{cc}\hfill \hfill & \hfill {\left[{\mathbf{H}}_s^T\mathbf{b}\right]}_{\mathit{k}}={\left[{\displaystyle \sum _{p=1}^P}{\left({\mathbf{H}}_s^p\right)}^T{\mathbf{b}}^p\right]}_{\mathit{k}}\hfill \\ \hfill \hfill & \hfill {\displaystyle \underset{=}{\left(i\right)}}{\displaystyle \sum _{p=1}^P}{\displaystyle \sum _{\mathit{j}\in {\mathrm{\Omega }}_{2\mathrm{D}}}}{\mathbf{b}}^p\left[\mathit{j}\right]\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}(·-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})\ast w^p\right)\left(\mathit{j}\right)\hfill \\ \hfill \hfill & \hfill {\displaystyle \underset{=}{\left(\mathit{ii}\right)}}{\displaystyle \sum _{p=1}^P}{\int }_{{\mathbb{R}}^2}{b}^p\left(\mathit{y}\right)\left({\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}(·-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})\ast w^p\right)\left(\mathit{y}\right)\mathrm{d}\mathit{y}\hfill \\ \hfill \hfill & \hfill {\displaystyle \underset{=}{\left(\mathit{iii}\right)}}{\displaystyle \sum _{p=1}^P}{\int }_{{\mathbb{R}}^2}{b}^p\left(\mathit{y}\right){\int }_{{\mathbb{R}}^2}{\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}(\stackrel{\sim }{\mathbf{y}}-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})w^p(\mathit{y}-\stackrel{\sim }{\mathbf{y}})\mathrm{d}\stackrel{\sim }{\mathbf{y}}\mathrm{d}\mathit{y}\hfill \\ \hfill \hfill & \hfill {\displaystyle \underset{=}{\left(\mathit{iv}\right)}}{\displaystyle \sum _{p=1}^P}{\int }_{{\mathbb{R}}^2}{\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}(\stackrel{\sim }{\mathbf{y}}-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k}){\int }_{{\mathbb{R}}^2}{b}^p\left(\mathit{y}\right)w^p(\mathit{y}-\stackrel{\sim }{\mathbf{y}})\mathrm{d}\mathit{y}\mathrm{d}\stackrel{\sim }{\mathbf{y}}\hfill \\ \hfill \hfill & \hfill {\displaystyle \underset{=}{\left(v\right)}}{\displaystyle \sum _{p=1}^P}{\int }_{{\mathbb{R}}^2}{\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s\right\}(\stackrel{\sim }{\mathbf{y}}-s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k})\ast \left(b^p\ast {\left(w^p\right)}^{\vee }\right)\left(\stackrel{\sim }{\mathbf{y}}\right)\mathrm{d}\stackrel{\sim }{\mathbf{y}}\hfill \\ \hfill \hfill & \hfill {\displaystyle \underset{=}{\left(\mathit{vi}\right)}}{\displaystyle \sum _{p=1}^P}\left(b^p\ast {\mathcal{P}}_{{\mathit{\theta }}_p}\left\{{\phi }_s^{\vee }\right\}\ast {\left(w^p\right)}^{\vee }\right)\left(s{\mathbf{M}}_{{\mathit{\theta }}_p^{\perp }}\mathit{k}\right)\text{.}\hfill \end{array}$$ (A.10)

Equality (i) is obtained by applying the definition of ${\mathbf{H}}_s^p$ given by (7). Equality (ii) results from Proposition 3 (see also Proof of Theorem 1). Equalities (iii), (v), and (vi) are obtained by using the definition of continuous convolutions. Equality (iv) is a simple rearrangement of both integrals.