Orientation Determination of Cryo-EM Images Using Least Unsquared Deviations

Abstract

A major challenge in single particle reconstruction from cryo-electron microscopy is to establish a reliable ab initio three-dimensional model using two-dimensional projection images with unknown orientations. Common-lines–based methods estimate the orientations without additional geometric information. However, such methods fail when the detection rate of common-lines is too low due to the high level of noise in the images. An approximation to the least squares global self-consistency error was obtained in [A. Singer and Y. Shkolnisky, SIAM J. Imaging Sci., 4 (2011), pp. 543–572] using convex relaxation by semidefinite programming. In this paper we introduce a more robust global self-consistency error and show that the corresponding optimization problem can be solved via semidefinite relaxation. In order to prevent artificial clustering of the estimated viewing directions, we further introduce a spectral norm term that is added as a constraint or as a regularization term to the relaxed minimization problem. The resulting problems are solved using either the alternating direction method of multipliers or an iteratively reweighted least squares procedure. Numerical experiments with both simulated and real images demonstrate that the proposed methods significantly reduce the orientation estimation error when the detection rate of common-lines is low.

1. Introduction

Single particle reconstruction (SPR) from cryo-electron microscopy (cryo-EM) [12, 57] is an emerging technique for determining the three-dimensional (3D) structure of macromolecules. One of the main challenges in SPR is to attain a resolution of 4Å or better, thereby allowing interpretation of atomic coordinates of macromolecular maps [14, 60]. Although X-ray crystallography and NMR spectroscopy can achieve higher resolution levels (~ 1Å by X-ray crystallography and 2–5Å by NMR spectroscopy), these traditional methods are often limited to relatively small molecules. In contrast, cryo-EM is typically applied to large molecules or assemblies with size ranging from 10 to 150 nm, such as ribosomes [13], protein complexes, and viruses.

Cryo-EM is used to acquire 2D projection images of thousands of individual, identical frozen-hydrated macromolecules at random unknown orientations and positions. The collected images are extremely noisy due to the limited electron dose used for imaging to avoid excessive beam damage. A 3D electron density map can be reconstructed computationally from the 2D projection images. However, a broad computational framework is required for the reconstruction.

1.1. The algorithmic pipeline of SPR

The main steps of a full SPR procedure can be summarized as follows:

1. Particle selection: The individual molecules are detected and extracted from the micrograph by image segmentation.

2. Class averaging: Images with similar projection directions are averaged together to improve their signal-to-noise ratio (SNR).

3. Orientation estimation: This is done via the random-conical tilt technique, or common-lines–based approaches, or reprojection. This step may also include a shift estimation.

4. Reconstruction: A 3D volume is generated by a tomographic inversion algorithm.

5. Iterative refinement: The whole procedure is repeated from step 3, using the outcome of the most recent reconstruction until convergence.

Numerous technical and computational advances have contributed to the different processing steps in SPR. First, the particles can be automatically selected from micrographs using algorithms based on template matching, computer vision, neural network, or other approaches [67]. Then multivariate statistical data compression [22, 56] and classification techniques [54, 34, 49] can be used to sort and partition the large set of particle images by their viewing directions, producing “class averages” of enhanced SNR. After class averaging, an ab initio estimation of the pose parameters can be obtained using the random-conical tilt technique [40] or common-lines–based approaches [55, 47, 48]. Using the ab initio estimation of the pose parameters, a preliminary 3D map is reconstructed from the projection images by a tomographic inversion algorithm. The tomographic inversion is computed either by applying a weighted adjoint operator [40, 39, 19, 35] or by using an iterative approach for inversion [17, 28, 15]. We refer to the preliminary 3D map as the ab initio model or simply initial model. The initial model is then iteratively refined [33] in order to obtain a higher-resolution 3D reconstruction. In each iteration of the refinement process, the current 3D model is projected at several pre-chosen viewing directions, and the resulting reprojection images are matched with the particle images, giving rise to new estimates of their pose parameters. The new pose parameters are then used to produce a refined 3D model using a 3D reconstruction algorithm. This process is repeated for several iterations until convergence is achieved.

In this paper, we address the problem of orientation estimation using common-lines between cryo-EM images in order to obtain an improved initial model. This is a major challenge in SPR due to the low SNR of the raw images, especially for small particles.

1.2. Orientation determination using common-lines between images

The Fourier projection-slice theorem (see, e.g., [31]) plays a fundamental role in common-lines–based reconstruction methods. The theorem states that restricting the 3D Fourier transform of the volume to a planar central slice yields the Fourier transform of a 2D projection of the volume in a direction perpendicular to the slice (Figure 1). Thus, any two projections imaged from nonparallel viewing directions intersect at a line in Fourier space, which is called the common-line between the two images. The common-lines between any three images with linearly independent projection directions determine their relative orientation up to handedness. This is the basis of the “angular reconstitution” technique of van Heel [55], which was also developed independently by Vainshtein and Goncharov [53]. In this technique, the orientations of additional projections are determined in a sequential manner. Farrow and Ottensmeyer [10] used quaternions to obtain the relative orientation of a new projection in a least squares sense. The main problem with such techniques is that they are sensitive to false detection of common-lines, which leads to the accumulation of errors. Penczek, Zhu, and Frank [37] tried to obtain the rotations corresponding to all projections simultaneously by minimizing a global energy functional, which requires a brute force search in an exponentially large parametric space of all possible orientations for all projections. Mallick et al. [26] and Singer et al. [47] applied Bayesian approaches to use common-lines information from different groups of projections. Recently, Singer and Shkolnisky [48] developed two algorithms based on eigenvectors and semidefinite programming for estimating the orientations of all images. These two algorithms correspond to convex relaxations of the global self-consistency error minimization and can accurately estimate all orientations at relatively low common-line detection rates.

Figure 1.

Fourier projection-slice theorem. In the middle, P̂_i is the 2D Fourier transform of projection P_i (left) computed on a polar grid. c⃗_ij is a unit vector representing the direction of the common-line (indicated in red) between P̂_i and P̂_j on P̂_i. On the right, the two transformed images P̂_i and P̂_j intersect with each other at the common-line after rotations R_i and R_j, yielding (3.3), where R_i and R_j represent the orientations of the projections P_i and P_j, respectively.

When the SNR of the images is significantly low, the detected common-lines consist of a modest number of noisy inliers, which are explained well by the image orientations, along with a large number of outliers, which have no structure. The standard common-lines–based methods, including those using least squares (LS) [10, 48], are sensitive to these outliers. In this paper we estimate the orientations using a different, more robust self-consistency error, which is the sum of unsquared residuals [32, 51] rather than the sum of squared residuals of the LS formulation. Convex relaxations of least unsquared deviations (LUD) have recently been proposed for other applications, such as robust principal component analysis [23, 64] and robust synchronization of orthogonal transformations [61]. Under certain noise models for the distribution of the outliers (e.g., the haystack model of [23, 64]), such convex relaxations enjoy proven guarantees for exact and stable recovery with high probability. Such theoretical and empirical improvements that LUD brings compared to LS serve as the main motivation for considering in this paper the application of LUD to the problem of orientation estimation from common-lines in SPR.

The LUD minimization problem is solved here via semidefinite relaxation. When the detection rate of common-lines is extremely low, the estimated viewing directions of the projection images are observed to cluster together (see Figure 4). This artificial clustering can be explained by the fact that images that share the same viewing direction also share more than one common-line (see more details in section 5). In order to mitigate this spurious clustering of the estimated viewing directions, we add to the minimization formulation a spectral norm term, either as a constraint or as a regularization term. The resulting minimization problem is solved by the alternating direction method of multipliers (ADMM). We also consider the application of the iteratively reweighted least squares (IRLS) procedure, which is not guaranteed to converge to the global minimizer but performs well in our numerical experiments. We demonstrate that the ab initio models resulting from our new methods are more accurate and require fewer refinement iterations compared to LS-based methods.

Figure 4.

The dependency of the spectral norm of G (denoted as αK here) on the distribution of orientations of the images. Each red point denotes the viewing direction of a projection. Five different distributions of viewing directions of K = 500 projections are plotted on spheres. The larger the spectral norm αK is, the more clustered the viewing directions are.

The paper is organized as follows. In section 2 we review the detection procedure of common-lines between images. Section 3 presents the LS and LUD global self-consistency cost functions. Section 4 introduces the semidefinite relaxation and rounding procedure for the LUD formulation. The additional spectral norm constraint is considered in section 5. The ADMM method for obtaining the global minimizer is detailed in section 6, and the IRLS procedure is described in section 7. Numerical results for both simulated and real data are provided in section 8. Finally, section 9 is a summary with discussion.

2. Detection of common-lines between images

Typically, the first step for detecting common-lines is to compute the 2D Fourier transform of each image on a polar grid using, e.g., the nonuniform fast Fourier transform (NUFFT) [9, 11, 18]. The transformed images have resolution n_r in the radial direction and resolution n_θ in the angular direction; that is, the radial resolution n_r is the number of equi-spaced samples along each ray in the radial direction, and the angular resolution n_θ is the number of angularly evenly distributed Fourier rays computed for each image (Figure 1). For simplicity, we let n_θ be an even number. The transformed images are denoted by ( ${\overrightarrow{l}}_0^k,{\overrightarrow{l}}_1^k,\dots ,{\overrightarrow{l}}_{n_{\theta }-1}^k$), where ${\overrightarrow{l}}_m^k=(l_{m,1}^k,l_{m,2}^k,\dots ,l_{m,n_r}^k)$ is an n_r-dimensional vector, m ∈ {0, 1, …, n_θ − 1} is the index of a ray, k ∈ {1, 2, …, K} is the index of an image, and K is the number of images. The zero frequency term is shared by all lines independently of the image and is therefore excluded from comparison. To determine the common-line between two images P_i and P_j, the similarity between all n_θ radial lines ${\overrightarrow{l}}_0^i,{\overrightarrow{l}}_1^i,\dots ,{\overrightarrow{l}}_{n_{\theta }-1}^i$ from the first image and all n_θ radial lines ${\overrightarrow{l}}_0^j,{\overrightarrow{l}}_1^j,\dots ,{\overrightarrow{l}}_{n_{\theta }-1}^j$ from the second image is measured (overall, $n_{\theta }^2$ comparisons), and the pair of radial lines ${\overrightarrow{l}}_{m_{i,j}}^i$ and ${\overline{l}}_{m_{j,i}}^j$ with the highest similarity is declared as the common-line pair between the two images. However, as a radial line is the complex conjugate of its antipodal line, the similarity measure between ${\overrightarrow{l}}_{m_1}^i$ and ${\overrightarrow{l}}_{m_2}^j$ has the same value as that between their antipodal lines ${\overrightarrow{l}}_{m_1+n_{\theta }/2}^i$ and ${\overrightarrow{l}}_{m_2+n_{\theta }/2}^j$ (where addition of indices is taken modulo n_θ). Thus the number of distinct similarity measures that need to be computed is $n_{\theta }^2/2$, obtained by restricting the index m₁ to take values between 0 and n_θ/2 and letting m₂ take any of the n_θ possibilities (see also [55] and [36, p. 255]). Equivalently, it is possible to compare real-valued 1D line projections of the 2D projection images, instead of comparing radial Fourier lines that are complex-valued. According to the Fourier projection-slice theorem, each 1D projection is obtained by the inverse Fourier transform of the corresponding Fourier radial line ${\overrightarrow{l}}_m^k$ and its antipodal line ${\overrightarrow{l}}_{m+n_{\theta }/2}^k$, and is denoted as ${\overrightarrow{s}}_m^k$. The 1D projection lines of a cryo-EM image can be displayed as a 2D image known as a “sinogram” (see [55, 45]).

Traditionally, the pair of radial lines (or sinogram lines) that has the maximum normalized cross correlation is declared as the common line, that is,

$$(m_{i,j},m_{j,i})=\underset{0\le m_1<n_{\theta }/2,0\le m_2<n_{\theta }}{argmax}\frac{\langle {\overrightarrow{l}}_{m_1}^i,{\overrightarrow{l}}_{m_2}^j\rangle }{\Vert {\overrightarrow{l}}_{m_1}^i\Vert \Vert {\overrightarrow{l}}_{m_2}^j\Vert }\text{for}\text{all}i\ne j,$$ (2.1)

where m_i,j is a discrete estimate for where the jth image intersects with the ith image. In practice, a weighted correlation, which is equivalent to applying a combination of high-pass and low-pass filters, is used to determine proximity. As noted in [55], the normalization is performed so that the correlation coefficient becomes a more reliable measure of similarity between radial lines. Note that, even with clean images, this estimate will have a small deviation from its ground truth (unknown) value due to discretization errors. With noisy images, large deviations of the estimates from their true values (say, errors of more than 10°) are frequent, and their frequency increases with the level of noise. We refer to common lines whose m_i,j and m_j,i values were estimated accurately (up to the discretization error tolerance 10°) as “correctly detected” common lines, or “inliers,” and to the remaining common lines as “falsely detected” or “outliers.”

A simple inliers-outliers model was introduced and analyzed in [48, sections 5.1 and 7]. That model assumes that each common-line pair is an inlier with probability p, independent of the remaining common-lines, while the directions of the outliers are uniformly distributed. It was noted, however, already in [48, p. 560] that this model is too simplistic as it assumes independence while there must be dependency: The number of common-line pairs is (K²), and since there are only (Kn_θ) radial lines, there must be dependency among the pairs for sufficiently large K. It is therefore difficult to provide a theoretical analysis for the evaluation of the practical performance of the different common-lines–based orientation determination algorithms.

3. Weighted LS and LUD

We define the directions of detected common-lines between the Fourier transformed image P̂_i and the Fourier transformed image P̂_j as the following unit vectors (Figure 1):

$${\overrightarrow{c}}_{i,j}=(c_{ij}^1,c_{ij}^2)=(cos(2\pi m_{ij}/n_{\theta }),sin(2\pi m_{ij}/n_{\theta })),$$ (3.1)

$${\overrightarrow{c}}_{ji}=(c_{ji}^1,c_{ji}^2)=(cos(2\pi m_{ji}/n_{\theta }),sin(2\pi m_{ji}/n_{\theta })),$$ (3.2)

where c⃗_ij and c⃗_ji are on the Fourier transformed images P̂_i and P̂_j, respectively, and m_ij and m_ji are discrete estimates for the common lines’ positions using (2.1). Let the rotation matrices R_i ∈ SO(3), i = 1, …, K, represent the orientations of the K images. According to the Fourier projection-slice theorem (Figure 1), the common lines on every two images should be the same after the 2D Fourier transformed images are inserted in the 3D Fourier space using the corresponding rotation matrices; that is,

$$R_i\left(\begin{array}{c}{\overline{c}}_{ij}^T\\ 0\end{array}\right)=R_j\left(\begin{array}{c}{\overline{c}}_{ji}^T\\ 0\end{array}\right)\text{for}1\le i<j\le K.$$ (3.3)

These can be viewed as $\left(\begin{array}{c}K\\ 2\end{array}\right)$ linear equations for the 6K variables corresponding to the first two columns of the rotation matrices. (The third column of each rotation matrix does not contribute in (3.3) since it is multiplied by the zero third entries in (c⃗_ij, 0) and (c⃗_ji, 0).) The weighted LS approach for solving this system can be formulated as the minimization problem

$$\underset{R_1,\dots ,R_K\in \mathbf{SO}(3)}{min}\sum _{i\ne j}{w}_{ij}{\Vert R_i{({\overrightarrow{c}}_{ij},0)}^T-R_j{({\overrightarrow{c}}_{ji},0)}^T\Vert }^2,$$ (3.4)

where the weights w_ij indicate the confidence in the detections of common-lines between pairs of images. Since (c⃗_ij, 0)^T and (c⃗_ji, 0)^T are 3D unit vectors, their rotations are also unit vectors; that is, ||R_i(c⃗_ij, 0)^T|| = ||R_j(c⃗_ji, 0)^T|| = 1. It follows that the minimization problem (3.4) is equivalent to the maximization problem of the sum of dot products:

$$\underset{R_1,\dots ,R_K\in \mathbf{SO}(3)}{max}\sum _{i\ne j}{w}_{ij}\langle R_i{({\overrightarrow{c}}_{ij},0)}^T,R_j{({\overrightarrow{c}}_{ji},0)}^T\rangle .$$ (3.5)

When the weight w_ij = 1 for each pair i ≠ j, (3.5) is equivalent to the LS problem that was considered in [37], and more recently in [48] using convex relaxation of the nonconvex constraint set. The solution to the LS problem may not be optimal due to the typically large proportion of outliers (Figure 2).

Figure 2.

The histogram plots of errors in the detected common-lines c⃗_ij for all i and j, i.e., ||R_i(c⃗_ij, 0)^T − R_j(c⃗_ji, 0)^T||, where R_i is the true rotation matrix for all i. The fat tail in (c) indicates that the detected common-lines contain a large number of outliers. The cryo-EM images used for common-line detection are shown in Figure 3.

To guard the estimation of the orientations from outliers, we replace the sum of weighted squared residuals in (3.4) with the more robust sum of unsquared residuals and obtain

$$\underset{R_1,\dots ,R_K\in \mathbf{SO}(3)}{min}\sum _{i\ne j}\Vert R_i{({\overrightarrow{c}}_{ij},0)}^T-R_j{({\overrightarrow{c}}_{ji},0)}^T\Vert ,$$ (3.6)

or equivalently,

$$\underset{R_1,\dots ,R_K\in \mathbf{SO}(3)}{min}\sum _{i\ne j}\Vert {({\overrightarrow{c}}_{ij},0)}^T-R_i^T{R}_j{({\overrightarrow{c}}_{ji},0)}^T\Vert .$$ (3.7)

We refer to the minimization problem (3.6) as the least unsquared deviation (LUD) problem. The self-consistency error given in (3.6) reduces the contribution from large residuals that may result from outliers. We note that it is also possible to consider the weighted version of (3.6), namely,

$$\underset{R_1,\dots ,R_K\in \mathbf{SO}(3)}{min}\sum _{i\ne j}{w}_{ij}\Vert R_i{({\overrightarrow{c}}_{ij},0)}^T-R_j{({\overrightarrow{c}}_{ji},0)}^T\Vert .$$

For simplicity, we focus here on the unweighted version.

4. Semidefinite programming relaxation (SDR) and the rounding procedure

Both the weighted LS problem (3.4) and the LUD problem (3.6) are nonconvex and therefore extremely difficult to solve if one requires the matrices R_i to be rotations, that is, when adding the constraints

$$R_i{R}_i^T=I_3,det(R_i)=1\text{for}i=1,\dots ,K,$$ (4.1)

where I₃ is the 3 × 3 identity matrix. A relaxation method that neglects the constraints (4.1) will simply collapse to the trivial solution R₁ = ··· = R_K = 0, which obviously does not satisfy the constraint (4.1).

The relaxation in [48] that uses semidefinite programming (SDP) can be modified in a straightforward manner in order to deal with nonunity weights w_ij in (3.5). We present this modification here for three reasons. First, the weighted version is required by the IRLS procedure (see section 7). Second, the rounding procedure after the SDP employed here is slightly different than the deterministic rounding procedure presented in [48] and is closer in spirit to the randomized rounding procedure of Goemans and Williamson for the MAX-CUT problem [16]. Finally, in the appendix we prove exact recovery of the rotations by the SDR procedure when the detected common-lines are all correct.

4.1. Constructing the Gram matrix G from the rotations R_i

We denote the columns of the rotation matrix R_i by $R_i^1,R_i^2$, and $R_i^3$, and write the rotation matrices as

$$R_i=\left(\begin{array}{ccc}\mid & \mid & \mid \\ R_i^1& R_i^2& R_i^3\\ \mid & \mid & \mid \end{array}\right),i=1,\dots ,K.$$

We define a 3 × 2K matrix R by concatenating the first two columns of all rotation matrices:

$$R=\left(\begin{array}{cccccccc}\mid & \mid & & \mid & \mid & & \mid & \mid \\ R_1^1& R_1^2& \cdots & R_k^1& R_k^2& \cdots & R_K^1& R_K^2\\ \mid & \mid & & \mid & \mid & & \mid & \mid \end{array}\right).$$ (4.2)

The Gram matrix G for the matrix R is a 2K × 2K matrix of inner products between the 3D column vectors of R; that is,

$$G=R^{T}R.$$ (4.3)

Clearly, G is a rank-3 semidefinite positive matrix (G ≽ 0), which can be conveniently written as a block matrix

$$G={(G_{ij})}_{i,j=1,\dots ,K},$$

where G_ij is the 2 × 2 upper left block of the rotation matrix $R_i^T{R}_j$, that is,

$$G_{ij}=\left(\begin{array}{c}{(R_i^1)}^T\\ {(R_i^2)}^T\end{array}\right)(\begin{array}{cc}{R}_i^1& R_i^2\end{array}).$$

In addition, the orthogonality of the rotation matrices ( $R_i^T{R}_i=I$) implies that

$$G_{ii}=I_2,i=1,2,\dots ,K,$$ (4.4)

where I₂ is the 2 × 2 identity matrix.

4.2. SDR for weighted LS

We first define two 2K × 2K matrices S = (S_ij)_i,j=1,…,K and W = (W_ij)_i,j=1,…,K, where the 2 × 2 subblocks S_ij and W_ij are given by

$$S_{ij}={\overrightarrow{c}}_{ji}^T{\overrightarrow{c}}_{ij}$$

and

$$W_{ij}=w_{ij}\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right).$$

Both matrices S and W are symmetric, and they store all available common-line information and weight information, respectively. It follows that the objective function (3.5) is the trace of the matrix (W ∘ S) G:

$$\sum _{i\ne j}{w}_{ij}\langle R_i{({\overrightarrow{c}}_{ij},0)}^T,R_j{({\overrightarrow{c}}_{ji},0)}^T\rangle =\text{trace}((W\circ S)G),$$ (4.5)

where the symbol ∘ denotes the Hadamard product between two matrices. A natural relaxation of the optimization problem (3.5) is thus given by the SDP problem

$$\underset{G\in {ℝ}^{2K\times 2K}}{max}\text{trace}((W\circ S)G)$$ (4.6)

$$\mathrm{s}.\mathrm{t}.G_{ii}=I_2,i=1,2,\dots ,K,$$ (4.7)

$$G\succcurlyeq 0.$$ (4.8)

The nonconvex rank-3 constraint on the Gram matrix G is missing from this SDR [25]. The problem (4.6)–(4.8) is an SDP that can be solved by standard SDP solvers. In particular, it can be well solved by the solver SDPLR [4], which takes advantage of the low-rank property of G. SDPLR is a first-order algorithm via low-rank factorization and hence can provide approximate solutions for large-scale problems. Moreover, the iterations of SDPLR are extremely fast.

4.3. SDR for LUD

Similar to defining the Gram matrix G in (4.3), we define a 3K × 3K matrix G̃ as G̃ = (G̃_ij)_i,j=1,…,K, where each G̃_ij is a 3 × 3 block defined as ${\stackrel{\sim }{G}}_{ij}=R_i^T{R}_j$. Then, a natural SDR for (3.7) is given by

$$\underset{\stackrel{\sim }{G}\succcurlyeq 0}{min}\sum _{i\ne j}\Vert {({\overrightarrow{c}}_{ij},0)}^T-{\stackrel{\sim }{G}}_{ij}{({\overrightarrow{c}}_{ji},0)}^T\Vert \mathrm{s}.\mathrm{t}.{\stackrel{\sim }{G}}_{ii}=I_3.$$ (4.9)

The constraints missing in this SDP formulation are the nonconvex rank-3 constraint and the determinant constraints det(G̃_ij) = 1 on the Gram matrix G̃. However, the solution G̃ to (4.9) is not unique. Note that if a set of rotation matrices {R_i} is a solution to (3.7), then the set of conjugated rotation matrices {JR_iJ} is also a solution to (3.7), where the matrix J is defined as

$$J=\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& -1\end{array}\right).$$

Thus, another solution to (4.9) is the Gram matrix ${\stackrel{\sim }{G}}^J={({\stackrel{\sim }{G}}_{ij}^J)}_{i,j=1,\dots ,K}$ with the 3 × 3 sub-blocks given by ${\stackrel{\sim }{G}}_{ij}^J=J{R}_i^{T}JJ{R}_{j}J=J{R}_i^T{R}_{j}J$. It can be verified that $\frac{1}{2}(\stackrel{\sim }{G}+{\stackrel{\sim }{G}}^J)$ is also a solution to (4.9). Using the fact that

$$\frac{1}{2}({\stackrel{\sim }{G}}_{ij}+{\stackrel{\sim }{G}}_{ij}^J)=\left(\begin{array}{cc}{G}_{ij}& \begin{array}{l}0\\ 0\end{array}\\ 00& 0\end{array}\right),$$

the problem (4.9) is reduced to

$$\underset{G\succcurlyeq 0}{min}\sum _{i\ne j}\Vert {\overrightarrow{c}}_{ij}^T-G_{ij}{\overrightarrow{c}}_{ji}^T\Vert \mathrm{s}.\mathrm{t}.G_{ii}=I_2.$$ (4.10)

This is a SDR for the LUD problem (3.6). The problem (4.10) can be solved using ADMM (see details in section 6.2).

4.4. The randomized rounding procedure

The matrix R is recovered from a random projection of the solution G of the SDP (4.6). We randomly draw a 2K ×3 matrix P from the Stiefel manifold V₃(ℝ^2K). The random matrix P is computed using the orthogonal matrix Q and the upper triangular matrix R from QR factorization of a random matrix with standard i.i.d. Gaussian entries; that is, P = Q sign (diag (R)), where sign stands for the entrywise sign function and diag(R) is a diagonal matrix whose diagonal entries are the same as those of the matrix R. The matrix P is shown to be drawn uniformly from the Stiefel manifold in [29]. We compute the Cholesky decomposition of G = LL^T and then project the lower triangular matrix L onto the subspace spanned by the columns of the 2K × 3 matrix LP.¹

The 2K × 3 matrix LP is a proxy for the matrix R^T (up to a global 3 × 3 orthogonal transformation). In other words, we can regard the 3 × 2K matrix (LP)^T as composed from K matrices of size 3 × 2, denoted A_i (i = 1, …, K), namely,

$${(LP)}^T=(\begin{array}{cccc}{A}_1& A_2& \cdots & A_K\end{array}).$$

The two columns of each A_i correspond to $R_i^1$ and $R_i^2$ (compare to (4.2)). We therefore estimate the matrix $R_i^{[1,2]}=(\begin{array}{cc}{R}_i^1& R_i^2\end{array})$ as the closest matrix to A_i on the Stiefel manifold V₂(ℝ³) in the Frobenius matrix norm. The closest matrix is given by (see, e.g., [1]) $R_i^{[1,2]}=U_i{V}_i^T$, where $A_i=U_i{\mathrm{\sum }}_i{V}_i^T$ is the singular value decomposition of A_i. We note that, except for the orthogonality constraint (4.7), the SDP (4.6)–(4.8) is identical to the Goemans–Williamson SDP for finding the maximum cut in a weighted graph [16], where the SDR and the randomized rounding procedure [50, 25] for maximum cut problem is proved to have a 0.87 performance guarantee. Currently we do not have a similar performance guarantee for the SDP relaxation of the LS problem (3.5). From the complexity point of view, SDP can be solved in polynomial time to any given precision. The idea of using SDP for determining image orientations in cryo-EM was originally proposed in [48].

5. The spectral norm constraint

In our numerical experiments (see section 8) we observed that in the presence of many “outliers” (i.e., a large proportion of misidentified common-lines) the estimated viewing directions² that are obtained by solving either (4.6)–(4.8) or (4.10) are highly clustered (Figure 4). This empirical behavior of the solutions can be explained by the fact that images whose viewing directions are parallel share many common-lines, which results from the overlapping Fourier slices. In other words, when the viewing directions of R_i and R_j are nearby, the fidelity term ||R_i(c⃗_ij, 0)^T − R_j(c⃗_ji, 0)^T|| (that appears in all cost functions) can become small (i.e., close to 0), even when the common-line pair (c⃗_ij, c⃗_ji) is misidentified. When the proportion of outliers in the detected common-lines is large enough, the solution to either the LS problem or the LUD problem can consist of many rotations with similar viewing directions, so that most of the fidelity terms have small values. In such cases, the Gram matrix G has just two dominant eigenvalues, instead of three.

In order to prevent the viewing directions from clustering and the Gram matrix G from collapsing to a rank-2 matrix, we add the following constraint on the spectral norm of the Gram matrix G to the optimization problem (4.6)–(4.8) or (4.10):

$$G\preccurlyeq \alpha K{I}_{2K},$$ (5.1)

where I_2K is the 2K × 2K identity matrix, or equivalently,

$${\Vert G\Vert }_2\le \alpha K,$$ (5.2)

where ||G||₂ is the spectral norm of the matrix G, and the parameter $\alpha \in [\frac{2}{3},1]$ controls the spread of the viewing directions. If the true imaging orientations are uniformly sampled from the rotation group SO(3), then by the law of large numbers and the symmetry of the distribution of orientations, the spectral norm of the true Gram matrix G_true is approximately $\frac{2}{3}K$. (To see this, notice that Tr(G) = Tr(KI₂) = 2K. Thus, the sum of eigenvalues of G is 2K. Recall that G is of rank 3, so if the rotations are uniformly distributed, then each of its three nontrivial eigenvalues equals $\frac{2K}{3}$.) On the other hand, if the true viewing directions are highly clustered, then the spectral norm of the true Gram matrix G_true is close to K. For a known distribution of orientations, we can compute the spectral norm of the true Gram matrix G_true accordingly, which can be verified to be a number between $\frac{2}{3}$ and 1. In practice, however, the distribution of the viewing directions is usually unknown a priori, and often it cannot be assumed to be uniform. To prevent a solution with clustered viewing directions, we fix the parameter α to some number satisfying $\frac{2}{3}\le \alpha <1$, and perhaps even try a few possible values for α and choose the best value by examining the resulting reconstructions. Another method for choosing the best value for α is to compare the likelihood scores. We defer the definition of the likelihood score to section 9.

6. The alternating direction method of multipliers (ADMM) for SDRs with spectral norm constraint

The application of ADMM to SDP problems was considered in [62]. Here we generalize the application of ADMM to the optimization problems considered in the previous sections. ADMM is a multiple-splitting algorithm that minimizes the augmented Lagrangian function in an alternating fashion such that in each step it minimizes over one block of the variables with all other blocks fixed and then updates the Lagrange multipliers. We apply ADMM to the dual problems since the linear constraint (6.2) satisfies = I, which simplifies the computation of subproblems. The strong duality theorem, which is known as Slater’s theorem, guarantees that in the presence of a strictly feasible solution, a primal problem can be solved by solving its dual problem. To obtain a strictly feasible solution to the primal problems with a positive semidefinite constraint, the linear constraint (6.2), and the spectral norm constraint (6.3), we can construct a Gram matrix G in (4.3) using rotations sampled from a uniform distribution over the rotation group. Therefore, strong duality holds for the primal problems, and the primal problems can be solved by applying ADMM to their corresponding dual problems.

6.1. The relaxed weighted LS problem

The weighted LS problem after SDR (4.6)–(4.8) can be efficiently solved using SDPLR [4]. However, SDPLR is not suitable for the problem once the spectral norm constraint on G (5.2) is added to (4.6)–(4.8). This is because the constraint (5.2) can be written as αKI − G ≽ 0, but the matrix αKI − G does not have a low-rank structure. Moreover, SDP solvers using polynomial-time primal-dual interior point methods are designed for small to medium-sized problems. Therefore, they are not suitable for our problem. Instead, here we devise a version of the ADMM which takes advantage of the low-rank property of G.

After the spectral norm constraint (5.2) is added, the problem (4.6)–(4.8) becomes

$$\underset{G\succcurlyeq 0}{min}-\langle C,G\rangle$$ (6.1)

$$\mathrm{s}.\mathrm{t}.\mathcal{A}(G)=\mathbf{b},$$ (6.2)

$${\Vert G\Vert }_2\le \alpha K,$$ (6.3)

where

$$\begin{array}{c}\mathcal{A}(G)={\left(\begin{array}{c}{G}_{ii}^{11}\\ G_{ii}^{22}\\ \frac{\sqrt{2}}{2}{G}_{ii}^{12}+\frac{\sqrt{2}}{2}{G}_{ii}^{21}\end{array}\right)}_{i=1,2,\dots ,K},\mathbf{b}={\left(\begin{array}{c}{b}_i^1\\ b_i^2\\ b_i^3\end{array}\right)}_{i=1,2,\dots ,K},\\ b_i^1=b_i^2=1,b_i^3=0\text{for}\text{all}i,\end{array}$$ (6.4)

$G_{ij}^{pq}$ denotes the (p, q)th element in the 2 × 2 subblock G_ij, C = W ∘ S is a symmetric matrix, and 〈C, G〉 = trace (CG). Following the equality 〈 (G), y〉 = 〈G, (y)〉 for arbitrary

$$\mathbf{y}={\left(\begin{array}{c}{y}_i^1\\ y_i^2\\ y_i^3\end{array}\right)}_{i=1,2,\dots ,K},$$

the adjoint of the operator is defined as

$${\mathcal{A}}^{\ast }(\mathbf{y})=Y=\left(\begin{array}{cc}{Y}^{11}& Y^{12}\\ Y^{21}& Y^{22}\end{array}\right),$$

where for i = 1, 2, …, K

$$Y_{ii}^{11}=y_i^1,Y_{ii}^{22}=y_i^2,\text{and}{Y}_{ii}^{12}=Y_{ii}^{21}=y_i^3/\sqrt{2}.$$

It can be verified that = I. The dual problem of problem (6.1)–(6.3) is

$$\underset{\mathbf{y},X\succcurlyeq 0}{max}\underset{{\Vert G\Vert }_2\le \alpha K}{min}-\langle C,G\rangle -\langle \mathbf{y},\mathcal{A}(G)-\mathbf{b}\rangle -\langle G,X\rangle .$$ (6.5)

By rearranging terms in (6.5), we obtain

$$\underset{\mathbf{y},X\succcurlyeq 0}{max}\underset{{\Vert G\Vert }_2\le \alpha K}{min}-\langle C+X+{\mathcal{A}}^{\ast }(\mathbf{y}),G\rangle +{\mathbf{y}}^T\mathbf{b}.$$ (6.6)

Using the fact that the dual norm of the spectral norm is the nuclear norm (Proposition 2.1 in [41]), we can obtain from (6.6) the dual problem

$$\underset{\mathbf{y},X\succcurlyeq 0}{max}{\mathbf{y}}^T\mathbf{b}-\alpha K\Vert C+X+{\mathcal{A}}^{\ast }{(\mathbf{y})\Vert }_{\ast },$$ (6.7)

where ||·||_* denotes the nuclear norm. By introducing a variable Z = C + X + (y), we obtain from (6.7) that

$$\underset{\mathbf{y},X\succcurlyeq 0}{min}-{\mathbf{y}}^T\mathbf{b}+\alpha K{\Vert Z\Vert }_{\ast }$$ (6.8)

$$\mathrm{s}.\mathrm{t}.Z=C+X+{\mathcal{A}}^{\ast }(\mathbf{y}).$$ (6.9)

Since Z is a symmetric matrix, ||Z||_* is the summation of the absolute values of the eigenvalues of Z. The augmented Lagrangian function of (6.8)–(6.9) is defined as

$$\mathcal{L}(\mathbf{y},Z,X,G)=-{\mathbf{y}}^T\mathbf{b}+\alpha K{\Vert Z\Vert }_{\ast }+\langle G,C+X+{\mathcal{A}}^{\ast }(\mathbf{y})-Z\rangle +\frac{\mu }{2}{\Vert C+X+{\mathcal{A}}^{\ast }(\mathbf{y})-Z\Vert }_F^2,$$ (6.10)

where μ > 0 is a penalty parameter. Using the augmented Lagrangian function (6.10), we devise an ADMM that minimizes (6.10) with respect to y, Z, X, and G in an alternating fashion; that is, given some initial guess, in each iteration the following three subproblems are solved sequentially:

$${\mathbf{y}}^{k+1}=arg\underset{\mathbf{y}}{min}\mathcal{L}\left(\mathbf{y},Z^k,X^k,G^k\right),$$ (6.11)

$$Z^{k+1}=arg\underset{Z}{min}\mathcal{L}\left({\mathbf{y}}^{k+1},Z,X^k,G^k\right),$$ (6.12)

$$X^{k+1}=arg\underset{X\succcurlyeq 0}{min}\mathcal{L}\left({\mathbf{y}}^{k+1},Z^{k+1},X,G^k\right),$$ (6.13)

and the Lagrange multiplier G is updated by

$$G^{k+1}=G^k+\gamma \mu \left(C+X^{k+1}+{\mathcal{A}}^{\ast }\left({\mathbf{y}}^{k+1}\right)-Z^{k+1}\right),$$ (6.14)

where $\gamma \in (0,\frac{1+\sqrt{5}}{2})$ is an appropriately chosen step length.

To solve the subproblem (6.11), we use the first-order optimality condition

$${\nabla }_{\mathbf{y}}\mathcal{L}\left(\mathbf{y},Z^k,X^k,G^k\right)=0$$

and the fact that = I, and we obtain

$${\mathbf{y}}^{k+1}=-\mathcal{A}\left(C+X^k-Z^k\right)-\frac{1}{\mu }(\mathcal{A}(G)-\mathbf{b}).$$

By rearranging the terms of (y^k+1, Z, X^k, G^k), it can be verified that the subproblem (6.12) is equivalent to

$$\underset{Z}{min}\frac{\alpha K}{\mu }{\Vert Z\Vert }_{\ast }+\frac{1}{2}{\Vert Z-B^k\Vert }_F^2,$$

where $B^k=C+X^k+{\mathcal{A}}^{\ast }({\mathbf{y}}^{k+1})+\frac{1}{\mu }{G}^k$. Let B^k = U Λ U^T be the spectral decomposition of the matrix B^k, where Λ = diag (λ) = diag (λ₁, …, λ_2K). Then Z^k+1 = U diag (ẑ) U^T, where ẑ is the optimal solution of the problem

$$\underset{\mathbf{z}}{min}\frac{\alpha K}{\mu }{\Vert \mathbf{z}\Vert }_1+\frac{1}{2}{\Vert \mathbf{z}-\mathit{\lambda }\Vert }_2^2.$$ (6.15)

It can be shown that the unique solution of (6.15) admits a closed-form solution called the soft-thresholding operator, following a terminology introduced by Donoho and Johnstone [8]; it can be written as

$${\widehat{z}}_i=\{\begin{array}{ll}0\hfill & \text{if}\mid {\lambda }_i\mid \le \frac{\alpha K}{\mu },\hfill \\ (1-\frac{\alpha }{\mu }/\mid {\lambda }_i\mid ){\lambda }_i\hfill & \text{otherwise}.\hfill \end{array}$$

The problem (6.13) can be shown to be equivalent to

$$\underset{X}{min}{\Vert X-H^k\Vert }_F^2\mathrm{s}.\mathrm{t}.X\succcurlyeq 0,$$

where $H^k=Z^{k+1}-C-{\mathcal{A}}^{\ast }({\mathbf{y}}^{k+1})-\frac{1}{\mu }{G}^k$. The solution $X^{k+1}=V_{+}{\mathrm{\sum }}_{+}{V}_{+}^T$ is the Euclidean projection of H^k onto the semidefinite cone (section 8.1.1 in [3]), where

$$V\mathrm{\sum }{V}^T=(\begin{array}{cc}{V}_{+}& V_{-}\end{array})\left(\begin{array}{cc}{\mathrm{\sum }}_{+}& 0\\ 0& {\mathrm{\sum }}_{-}\end{array}\right)\left(\begin{array}{c}{V}_{+}^T\\ V_{-}^T\end{array}\right)$$

is the spectral decomposition of the matrix H^k, and Σ₊ and Σ₋ are the positive and negative eigenvalues of H^k.

It follows from the update rule (6.14) that

$$\begin{array}{l}{G}^{k+1}=(1-\gamma )G^k+\gamma \mu \left(C+X^{k+1}+{\mathcal{A}}^{\ast }\left({\mathbf{y}}^{k+1}\right)-Z^{k+1}+\frac{1}{\mu }{G}^k\right)\\ =(1-\gamma )G^k+\gamma \mu \left(X^{k+1}-H^k\right).\end{array}$$

6.2. The relaxed LUD problem

Consider the LUD problem after SDR:

$$\underset{G\succcurlyeq 0}{min}\sum _{i<j}\Vert {\overrightarrow{c}}_{ij}^T-G_{ij}{\overrightarrow{c}}_{ji}^T\Vert \mathrm{s}.\mathrm{t}.\mathcal{A}(G)=\mathbf{b},$$ (6.16)

where G, , and b are defined in (4.3) and (6.4). The ADMM devised to solve (6.16) is similar to and simpler than the ADMM devised to solve the version with the spectral norm constraint. We focus on the more difficult problem with the spectral norm constraint. By introducing ${\mathbf{x}}_{ij}={\overrightarrow{c}}_{ij}^T-G_{ij}{\overrightarrow{c}}_{ji}^T$ and adding the spectral norm constraint ||G||₂ ≤ αK, we obtain

$$\underset{{\mathbf{x}}_{ij},G\succcurlyeq 0}{min}\sum _{i<j}\Vert {\mathbf{x}}_{ij}\Vert \mathrm{s}.\mathrm{t}.\mathcal{A}(G)=\mathbf{b},{\mathbf{x}}_{ij}={\overrightarrow{c}}_{ij}^T-G_{ij}{\overrightarrow{c}}_{ji}^T,{\Vert G\Vert }_2\le \alpha K.$$ (6.17)

The dual problem of problem (6.17) is

$$\underset{{\mathit{\theta }}_{ij},\mathbf{y},X\succcurlyeq 0}{max}\underset{{\mathbf{x}}_{ij},{\Vert G\Vert }_2\le \alpha K}{min}\sum _{i<j}(\Vert {\mathbf{x}}_{ij}\Vert -\langle {\mathit{\theta }}_{ij},{\mathbf{x}}_{ij}-{\overrightarrow{c}}_{ij}^T+G_{ij}{\overrightarrow{c}}_{ji}^T\rangle )-\langle \mathbf{y},\mathcal{A}(G)-\mathbf{b}\rangle -\langle G,X\rangle .$$ (6.18)

By rearranging terms in (6.18), we obtain

$$\underset{{\mathit{\theta }}_{ij},\mathbf{y},X\succcurlyeq 0}{max}\underset{{\mathbf{x}}_{ij},{\Vert G\Vert }_2\le \alpha K}{min}-\langle Q(\mathit{\theta })+X+{\mathcal{A}}^{\ast }(\mathbf{y}),G\rangle +{\mathbf{y}}^T\mathbf{b}+\sum _{i<j}(\Vert {\mathbf{x}}_{ij}\Vert -\langle {\mathit{\theta }}_{ij},{\mathbf{x}}_{ij}\rangle +\langle {\mathit{\theta }}_{ij},{\overrightarrow{c}}_{ij}^T\rangle ),$$ (6.19)

where θ = (θ_ij)_{i,j=1,…, K}, ${\mathit{\theta }}_{ij}={({\theta }_{ij}^1,{\theta }_{ij}^2)}^T,{\overrightarrow{c}}_{ij}=(c_{ij}^1,c_{ij}^2)$,

$$Q(\mathit{\theta })=\frac{1}{2}\left(\begin{array}{cc}{Q}^{11}(\mathit{\theta })& Q^{12}(\mathit{\theta })\\ Q^{21}(\mathit{\theta })& Q^{22}(\mathit{\theta })\end{array}\right)\text{and}{Q}^{pq}(\mathit{\theta })=\left(\begin{array}{cccc}0& {\theta }_{12}^p{c}_{21}^p& \cdots & {\theta }_{1K}^p{c}_{K1}^q\\ c_{21}^q{\theta }_{12}^p& 0& \cdots & {\theta }_{2K}^p{c}_{K2}^q\\ ⋮& ⋮& \ddots & ⋮\\ c_{K1}^q{\theta }_{1K}^p& c_{K2}^q{\theta }_{2K}^p& \cdots & 0\end{array}\right)$$

for p, q = 1, 2. It is easy to verify that for 1 ≤ i < j ≤ K

$$\underset{{\mathbf{x}}_{ij}}{min}(\Vert {\mathbf{x}}_{ij}\Vert -\langle {\mathit{\theta }}_{ij},{\mathbf{x}}_{ij}\rangle )=\{\begin{array}{ll}0\hfill & \text{if}\Vert {\mathit{\theta }}_{ij}\Vert \le 1,\hfill \\ -\infty \hfill & \text{otherwise}.\hfill \end{array}$$ (6.20)

In fact, (6.20) is obtained using the inequality

$$\begin{array}{l}\Vert {\mathbf{x}}_{ij}\Vert -\langle {\mathit{\theta }}_{ij},{\mathbf{x}}_{ij}\rangle =\Vert {\mathbf{x}}_{ij}\Vert -\Vert {\mathit{\theta }}_{ij}\Vert \Vert {\mathbf{x}}_{ij}\Vert \langle {\mathit{\theta }}_{ij}/\Vert {\mathit{\theta }}_{ij}\Vert ,{\mathbf{x}}_{ij}/\Vert {\mathit{x}}_{ij}\Vert \rangle \\ \ge \Vert {\mathbf{x}}_{ij}\Vert -\Vert {\mathit{\theta }}_{ij}\Vert \Vert {\mathit{x}}_{ij}\Vert =(1-\Vert {\mathit{\theta }}_{ij}\Vert )\Vert {\mathit{x}}_{ij}\Vert ,\end{array}$$ (6.21)

and the inequality in (6.21) holds when θ_ij and x_ij have the same direction. Using (6.20) and the fact that the dual norm of the spectral norm is the nuclear norm, we can obtain from (6.19) the dual problem

$$\underset{{\mathit{\theta }}_{ij},\mathbf{y},X\succcurlyeq 0}{min}-{\mathbf{y}}^T\mathbf{b}-\sum _{i<j}\langle {\mathit{\theta }}_{ij},{\overrightarrow{c}}_{ij}^T\rangle +\alpha K{\Vert Z\Vert }_{\ast }$$ (6.22)

$$\mathrm{s}.\mathrm{t}.Z=Q(\mathit{\theta })+X+{\mathcal{A}}^{\ast }(\mathbf{y})\text{and}\Vert {\mathit{\theta }}_{ij}\Vert \le 1.$$ (6.23)

The augmented Lagrangian function of problem (6.22)–(6.23) is defined as

$$\mathcal{L}(\mathbf{y},\mathit{\theta },Z,X,G)=-{\mathbf{y}}^T\mathbf{b}+\alpha K{\Vert Z\Vert }_{\ast }-\sum _{i<j}\langle {\mathit{\theta }}_{ij},{\overrightarrow{c}}_{ij}^T\rangle +\langle G,Q(\mathit{\theta })+X+{\mathcal{A}}^{\ast }(\mathbf{y})-Z\rangle +\frac{\mu }{2}{\Vert Q(\mathit{\theta })+X+{\mathcal{A}}^{\ast }(\mathbf{y})-Z\Vert }_F^2$$ (6.24)

for ||θ_ij|| ≤ 1, where μ > 0 is a penalty parameter. Similar to section 6.1, using the augmented Lagrangian function (6.24), ADMM is used to minimize (6.24) with respect to y, θ, Z, X, and G alternatively; that is, given some initial guess, in each iteration the following four subproblems are solved sequentially:

$${\mathbf{y}}^{k+1}=arg\underset{\mathbf{y}}{min}\mathcal{L}\left(\mathbf{y},{\mathit{\theta }}^k,Z^k,X^k,G^k\right),$$ (6.25)

$${\mathit{\theta }}_{ij}^{k+1}=arg\underset{\Vert {\mathit{\theta }}_{\mathbf{ij}}\Vert \le \mathbf{1}}{min}\mathcal{L}\left({\mathbf{y}}^{k+1},\mathit{\theta },Z^k,X^k,G^k\right),$$ (6.26)

$$Z^{k+1}=arg\underset{Z}{min}\mathcal{L}\left({\mathbf{y}}^{k+1},{\mathit{\theta }}^{k+1},Z,X^k,G^k\right),$$ (6.27)

$$X^{k+1}=arg\underset{X\succcurlyeq 0}{min}\mathcal{L}\left({\mathbf{y}}^{k+1},{\mathit{\theta }}^{k+1},Z^{k+1},X,G^k\right),$$ (6.28)

and the Lagrange multiplier G is updated by

$$G^{k+1}=G^k+\gamma \mu \left(Q\left({\mathit{\theta }}^{k+1}\right)+X^{k+1}+{\mathcal{A}}^{\ast }\left({\mathbf{y}}^{k+1}\right)-Z^{k+1}\right),$$ (6.29)

where $\gamma \in (0,\frac{1+\sqrt{5}}{2})$ is an appropriately chosen step length. The methods for solving subproblems (6.25), (6.27), and (6.28) are similar to those used in (6.11), (6.12), and (6.13). To solve subproblem (6.26), we rearrange the terms of (y^k+1, θ, Z^k, X^k, G^k) and obtain an equivalent problem,

$$\underset{{\mathit{\theta }}_{ij}}{min}-\langle {\mathit{\theta }}_{ij},{\overrightarrow{c}}_{ij}^T\rangle +\frac{\mu }{2}{\Vert {\mathit{\theta }}_{ij}{\overrightarrow{c}}_{ji}+{\mathbf{\Phi }}_{ij}\Vert }_F^2\mathrm{s}.\mathrm{t}.\Vert {\mathit{\theta }}_{ij}\Vert \le 1,$$ (6.30)

where $\mathrm{\Phi }=X^k+{\mathcal{A}}^{\ast }({\mathbf{y}}^{k+1})-Z^k+\frac{1}{\mu }{G}^k,\mathrm{\Phi }=\left(\begin{array}{cc}{\mathrm{\Phi }}^{11}& {\mathrm{\Phi }}^{12}\\ {\mathrm{\Phi }}^{21}& {\mathrm{\Phi }}^{22}\end{array}\right)$, and ${\mathrm{\Phi }}_{ij}=\left(\begin{array}{cc}{\mathrm{\Phi }}_{ij}^{11}& {\mathrm{\Phi }}_{ij}^{12}\\ {\mathrm{\Phi }}_{ij}^{21}& {\mathrm{\Phi }}_{ij}^{22}\end{array}\right)$. Problem (6.30) is further simplified as

$$\underset{{\mathit{\theta }}_{ij}}{min}\langle {\mathit{\theta }}_{ij},\mu {\mathrm{\Phi }}_{ij}{\overrightarrow{c}}_{ji}^T-{\overrightarrow{c}}_{ij}^T\rangle +\frac{\mu }{2}{\Vert {\mathit{\theta }}_{ij}\Vert }^2\mathrm{s}.\mathrm{t}.\Vert {\mathit{\theta }}_{ij}\Vert \le 1,$$

whose solution is

$${\mathit{\theta }}_{ij}=\{\begin{array}{ll}\frac{1}{\mu }{\overrightarrow{c}}_{ij}^T-{\mathrm{\Phi }}_{ij}{\overrightarrow{c}}_{ij}^T\hfill & \text{if}\Vert \frac{1}{\mu }{\overrightarrow{c}}_{ij}^T-{\mathrm{\Phi }}_{ij}{\overrightarrow{c}}_{ij}^T\Vert \le 1,\hfill \\ \frac{{\overrightarrow{c}}_{ij}^T-\mu {\mathrm{\Phi }}_{ij}{\overrightarrow{c}}_{ij}^T}{\Vert {\overrightarrow{c}}_{ij}^T-\mu {\mathrm{\Phi }}_{ij}{\overrightarrow{c}}_{ij}^T\Vert }\hfill & \text{otherwise}.\hfill \end{array}$$

The practical issues related to how to take advantage of low-rank assumptions of G in the eigenvalue decomposition performed at each iteration, strategies for adjusting the penalty parameter μ, the use of a step size γ for updating the primal variable X, and termination rules using the infeasibility measures are discussed in detail in [62]. The convergence analysis of ADMM using more than two blocks of variables can be found in [20]. However, there are some conditions of Assumption A in [20, p. 5] that cannot be satisfied for our problem. For example, Assumption A requires that the feasible set be polyhedral, whereas the SDP cone in our problem is not a polyhedral. To generalize the convergence analysis in [20] to our problem, we will need to show that the local error bounds (see [20, pp. 8–9]) hold for the SDP cone. Currently we do not have a rigorous convergence proof for ADMM for our problem. In practice, we observe that the primal infeasibility, the dual infeasibility,³ and the relative gap between the primal and dual problems converge simultaneously to zero, quickly during first hundreds of iterations, then the rate of convergence slows down as the iterates approach an optimal solution.

7. The iterative reweighted least squares (IRLS) procedure

Since c⃗_ij and c⃗_ji are unit vectors, it is tempting to replace the LUD problem (3.7) with the following SDR:

$$\underset{G\in {ℝ}^{2K\times 2K}}{min}F(G)=\sum _{i,j=1,2,\dots ,K}\sqrt{2-2\sum _{p,q=1,2}{G}_{ij}^{pq}{S}_{ij}^{pq}}$$ (7.1)

$$\mathrm{s}.\mathrm{t}.G_{ii}=I_2,i=1,2,\dots ,K,$$ (7.2)

$$G\succcurlyeq 0,$$ (7.3)

$${\Vert G\Vert }_2\le \alpha K(\text{optional}),$$ (7.4)

where α is a fixed number between $\frac{2}{3}$ and 1, and the spectral norm constraint on G (7.4) is added when the solution to the problem (7.1)–(7.3) is a set of highly clustered rotations. Notice that this relaxed problem is, however, not convex since the objective function (7.1) is concave. We propose to solve (7.1)–(7.3) (possibly with (7.4)) by a variant of the IRLS procedure [6, 23, 64, 5], which at best converges to a local minimizer. With a good initial guess for G it can be hoped that the global minimizer is obtained. Such an initial guess can be taken as the LS solution. In section 7.2, we prove that the sequence of IRLS converges to a stationary point. However, our convergence analysis cannot exclude the possibility of convergence to saddle points that are not stable.

7.1. Main idea

Before the rounding procedure, the IRLS procedure finds an approximate solution to the optimization problem (7.1)–(7.3) (possibly with (7.4)) by solving its smoothing version

$$\underset{G\in {ℝ}^{2K\times 2K}}{min}F(G,\epsilon )=\sum _{i,j=1,2,\dots ,K}\sqrt{2-2\sum _{p,q=1,2}{G}_{ij}^{pq}{S}_{ij}^{pq}+{\epsilon }^2}$$ (7.5)

$$\mathrm{s}.\mathrm{t}.G_{ii}=I_2,i=1,2,\dots ,K,$$ (7.6)

$$G\succcurlyeq 0,$$ (7.7)

$${\Vert G\Vert }_2\le \alpha K(\text{optinal}),$$ (7.8)

where ε > 0 is a small number. The solution to the smoothing version is close to the solution to the original problem. In fact, let $G_{\epsilon }^{\ast }=argminF(G,\epsilon )$ and G^* = arg min F(G); then we shall verify that

$$\mid F(G_{\epsilon }^{\ast })-F(G^{\ast })\mid \le 4K^2\epsilon .$$ (7.9)

Using the definition of F(G, ε) and F(G), we obtain that

$$0\le F(G,\epsilon )-F(G)<4K^2\epsilon ,$$

and thus

$$(F(G^{\ast },\epsilon )-F(G_{\epsilon }^{\ast },\epsilon ))+(F(G_{\epsilon }^{\ast })-F(G^{\ast }))=(F(G^{\ast },\epsilon )-F(G^{\ast }))-(F(G_{\epsilon }^{\ast },\epsilon )-F(G_{\epsilon }^{\ast }))\le 4K^2\epsilon .$$

Since $F(G^{\ast },\epsilon )-F(G_{\epsilon }^{\ast },\epsilon )\ge 0$ and $F(G_{\epsilon }^{\ast })-F(G^{\ast })\ge 0$, inequality (7.9) holds.

In each iteration, the IRLS procedure solves the problem

$$G^{k+1}=arg\underset{G\succcurlyeq 0}{min}\sum _{i\ne j}{w}_{ij}^k(2-2\langle G_{ij},S_{ij}\rangle +{\epsilon }^2)\mathrm{s}.\mathrm{t}.\mathcal{A}(G)=\mathbf{b},(\text{optional}:{\Vert G\Vert }_2\le \alpha K)$$ (7.10)

on the (k + 1)th iteration, where $w_{ij}^0=1$, and

$$w_{ij}^k=1/\sqrt{2-2\langle G_{ij}^k,S_{ij}\rangle +{\epsilon }^2}\text{for}\text{all}k>0.$$

In other words, in each iteration, more emphasis is given to detected common-lines that are better explained by the current estimate G^k of the Gram matrix. The inclusion of the regularization parameter ε ensures that no single detected common-line can gain undue influence when solving problem (7.10), which is equivalent to

$$G^{k+1}=arg\underset{G\succcurlyeq 0}{max}\langle W^k\circ S,G\rangle \mathrm{s}.\mathrm{t}.\mathcal{A}(G)=\mathbf{b}(\text{optional}:{\Vert G\Vert }_2\le \alpha K).$$ (7.11)

We repeat the process until the residual sequence {r^k} has converged or the maximum number of iterations has been reached.

7.2. Convergence analysis

We shall verify below that the value of the cost function is nonincreasing, and that every cluster point of the sequence of IRLS is a stationary point of (7.5)–(7.7) in the following lemma and theorem, for the problem without the spectral norm constraint on G. The arguments can be generalized to the case with the spectral norm constraint. The proof of Theorem 7.2 follows the method of proof for Theorem 3 in the paper [30] by Mohan and Fazel.

Lemma 7.1

The value of the cost function sequence is monotonically nonincreasing; i.e.,

$$F(G^{k+1},\epsilon )\le F(G^k,\epsilon ),$$ (7.12)

where {G^k} is the sequence generated by the IRLS procedure of Algorithm 1.

Algorithm 1.

(the IRLS procedure). Solve optimization problem (7.1)–(7.3) (with the spectral norm constraint on G (7.4) if the input parameter α satisfies $\frac{2}{3}\le \alpha <1$), and then recover the orientations by rounding.

Input: a 2K × 2K common-line matrix S, a regularization parameter ε, a parameter α, and the total number of iterations Niter.

$w_{ij}^0=1$ for all i, j = 1, …, K;

G0 = 0;

for k = 1 → Niter, step size = 1 do

update W by setting $w_{ij}=w_{ij}^{k-1}$;

if $\frac{2}{3}\le \alpha <1$, obtain Gk by solving the problem (6.1)–(6.3) using ADMM; otherwise, obtain Gk by solving (4.6)–(4.8) using SDPLR (with initial guess Gk−1);

$r_{ij}^k=\sqrt{2-2{\sum }_{p,q=1,2}{G}_{ij}^{pq}{S}_{ij}^{pq}+{\epsilon }^2}$;

$w_{ij}^k=1/r_{ij}^k$;

the residual $r^k={\sum }_{i,j=1}^K{r}_{ij}^k$;

end for

obtain estimated orientations from GNiter using the randomized rounding procedure in section 4.4.

Output: estimated orientations R̂1, …, R̂K .

Proof

Since G^k is the solution of (7.11), using the Karush–Kuhn–Tucker (KKT) conditions, there exist y^k ∈ ℝ^2K and X^k ∈ ℝ^2K×2K such that

$$-{\mathcal{A}}^{\ast }({\mathbf{y}}^k)+X^k+W^{k-1}\circ S=0,\mathcal{A}(G^k)-\mathbf{b}=0,$$ (7.13)

$$G^k\succcurlyeq 0,X^k\succcurlyeq 0,\langle G^k,X^k\rangle =0.$$ (7.14)

Hence we have

$$\begin{array}{l}0=-{({\mathbf{y}}^k)}^T(\mathcal{A}(G^k)-\mathbf{b})+{({\mathbf{y}}^{(k+1)})}^T(\mathcal{A}(G^{k+1})-\mathbf{b})\\ ={({\mathbf{y}}^{k+1}-{\mathbf{y}}^k)}^T(\mathcal{A}(G^k)-\mathbf{b})+\langle {\mathcal{A}}^{\ast }({\mathbf{y}}^{k+1}),G^{k+1}-G^k\rangle \\ =\langle X^{k+1}+W^k\circ S,G^{k+1}-G^k\rangle \\ =\langle X^{k+1},G^{k+1}\rangle -\langle X^{k+1},G^k\rangle +\langle W^k\circ S,G^{k+1}-G^k\rangle \\ \le \langle W^k\circ S,G^{k+1}-G^k\rangle \\ =\frac{1}{2}\sum _{i\ne j}\left(-w_{ij}^k\left(2-2\langle G_{ij}^{k+1},S_{ij}\rangle +{\epsilon }^2\right)+w_{ij}^k\left(2-2\langle G_{ij}^k,S_{ij}\rangle +{\epsilon }^2\right)\right)\end{array}$$ (7.15)

$$=\frac{1}{2}\sum _{i\ne j}\left(-\frac{2-2\langle G_{ij}^{k+1},S_{ij}\rangle +{\epsilon }^2}{\sqrt{2-2\langle G_{ij}^k,S_{ij}\rangle +{\epsilon }^2}}+\sqrt{2-2\langle G_{ij}^k,S_{ij}\rangle +{\epsilon }^2}\right),$$ (7.16)

where the first and third equalities use (7.13), and the inequality (7.15) uses (7.14). From (7.16) we obtain

$$\begin{array}{l}F{(G^k,\epsilon )}^2={\left(\sum _{i\ne j}\sqrt{2-2\langle G_{ij}^k,S_{ij}\rangle +{\epsilon }^2}\right)}^2\\ \ge \left(\sum _{i\ne j}\sqrt{2-2\langle G_{ij}^k,S_{ij}\rangle +{\epsilon }^2}\right)\left(\sum _{i\ne j}\frac{2-2\langle G_{ij}^{k+1},S_{ij}\rangle +{\epsilon }^2}{\sqrt{2-2\langle G_{ij}^k,S_{ij}\rangle +{\epsilon }^2}}\right)\\ \ge {\left(\sum _{i\ne j}\sqrt{2-2\langle G_{ij}^{k+1},S_{ij}\rangle +{\epsilon }^2}\right)}^2=F{(G^{k+1},\epsilon )}^2,\end{array}$$ (7.17)

where the last inequality uses the Cauchy–Schwarz inequality and the equality holds if and only if

$$\frac{\sqrt{2-2\langle G_{ij}^{k+1},S_{ij}\rangle +{\epsilon }^2}}{\sqrt{2-2\langle G_{ij}^k,S_{ij}\rangle +{\epsilon }^2}}=c\text{for}\text{all}i\ne j,$$ (7.18)

where c is a constant. Thus (7.12) is confirmed.

Theorem 7.2

The sequence of iterates {G^k} of IRLS is bounded, and every cluster point of the sequence is a stationary point of (7.5)–(7.7).

Proof

Since trace(G^k) = 2K and G^k ≽ 0, the sequence {G^k} is bounded. It follows that W^k and trace((W^k ∘ S)G^k+1) are bounded. Using the strong duality of SDP, we conclude that b^Ty^k+1 = trace((W^k ∘ S)G^k+1) is bounded. In addition, from the KKT conditions (7.13)–(7.14) we obtain – (y^k+1) + W^k ∘ S ≽ 0. Using the definition of and S, the property of semidefinite matrices, and the fact that W^k is bounded, it can be verified that y^k is bounded. Using (7.13) again, we obtain

$$\Vert X^k\Vert =\Vert {\mathcal{A}}^{\ast }({\mathbf{y}}^k)-W^{k-1}\circ S\Vert \le \Vert {\mathcal{A}}^{\ast }({\mathbf{y}}^k)\Vert +\Vert W^{k-1}\circ S\Vert ,$$

which implies that X^k is bounded.

We now show that every cluster point of {G^k} is a stationary point of (7.5)–(7.7). Suppose to the contrary, and let Ḡ be a cluster point of {G^k} that is not a stationary point. By the definition of cluster point, there exists a subsequence {G^k_i, W^k_i, X^k_i, y^k_i} of {G^k, W^k, X^k, y^k} converging to (Ḡ, W̄, X̄, ȳ). By passing to a further subsequence if necessary, we can assume that {G^k_i+1, W^k_i+1, X^k_i+1, y^k_i+1} is also convergent, and we denote its limit by (Ĝ, Ŵ, X̂, ŷ). G^k_i+1 is defined as (7.10) or (7.11) and satisfies the KKT conditions (7.13)–(7.14). Passing to limits, we see that

$$\begin{array}{c}-{\mathcal{A}}^{\ast }(\widehat{\mathbf{y}})+\widehat{X}+\overline{W}\circ S=0,\mathcal{A}(\widehat{G})-\mathbf{b}=0,\\ \widehat{G}\succcurlyeq 0,\widehat{X}\succcurlyeq 0,\langle \widehat{G},\widehat{X}\rangle =0.\end{array}$$

Thus we conclude that Ĝ is a maximizer of the following convex optimization problem:

$$\underset{G\succcurlyeq 0}{max}\langle \overline{W}\circ S,G\rangle \mathrm{s}.\mathrm{t}.\mathcal{A}(\widehat{G})=\mathbf{b}.$$

Next, by assumption, Ḡ is not a stationary point of (7.5)–(7.7). This implies that Ḡ is not a maximizer of the problem above, and thus 〈 W̄ ∘ S, Ĝ 〉 > 〈 W̄ ∘ S, Ḡ 〉. From this last relation and (7.17)–(7.18) it follows that

$$F(\widehat{G},\epsilon )<F(\overline{G},\epsilon ).$$ (7.19)

Otherwise if F(Ĝ, ε) = F (Ḡ, ε), then 〈 Ĝ_ij, S_ij 〉 = 〈 Ḡ_ij, S_ij 〉 due to (7.17)–(7.18), and thus we would obtain 〈 W̄ ∘ S, Ĝ 〉 = 〈 W̄ ∘ S, Ḡ 〉, which is a contradiction.

On the other hand, it follows from Lemma 7.1 that the sequence {F(G^k, ε)}converges. Thus we have that

$$limF(G^k,\epsilon )=limF(G^{k_i},\epsilon )=F(\overline{G},\epsilon )=limF(G^{k_i+1},\epsilon )=F(\widehat{G},\epsilon ),$$

which contradicts (7.19). Hence, every cluster point of the sequence is a stationary point of (7.5)–(7.7).

In addition, using Hölder’s inequality, the analysis can be generalized to the reweighted approach to solve

$$\underset{G\succcurlyeq 0}{min}\sum _{i\ne j}{(2-2\langle G_{ij},S_{ij}\rangle )}^{\frac{p}{2}}\mathrm{s}.\mathrm{t}.\mathcal{A}(G)=\mathbf{b}(\text{optional}:{\Vert G\Vert }_2\le \alpha K),$$ (7.20)

where 0 < p < 1. The convergence analysis of IRLS (p = 1) in [6, 23] can be extended to different applications with p < 1. The problem (7.20) is an SDR of the problem

$$\underset{R_1,\dots ,R_K\in \mathbf{SO}(3)}{min}\sum _{i\ne j}{\Vert R_i{({\overrightarrow{c}}_{ij},0)}^T-R_j{({\overrightarrow{c}}_{ji},0)}^T\Vert }^p.$$ (7.21)

The smaller p is, the more penalty the outliers in the detected common-lines receive.

8. Numerical results

All numerical experiments were performed on a machine with 2 Intel(R) Xeon(R) X5570 CPUs, each with 4 cores, running at 2.93 GHz. In all the experiments, the polar Fourier transform of images for common-line detection had radial resolution n_r = 100 and angular resolution n_θ = 360. The number of iterations was set to be N_iter = 10 in all IRLS procedures. Reconstruction from the images with estimated orientations used the Fourier-based 3D reconstruction package FIRM⁴ [59]. The reconstructed volumes are shown in Figures 5, 6, and 9 using the visualization system Chimera [38].

Figure 5.

The clean volume (top), the reconstructed volumes, and the MSEs (8.2) of the estimated rotations. No spectral norm constraint was used (i.e., α = N/A) for all algorithms. The result using the IRLS procedure without α is not available due to the highly clustered estimated projection directions. All the volumes have size 129 × 129 × 129 voxels.

Figure 6.

The reconstructed volumes from images with SNR = 1/64 and the MSEs (8.2) of the estimated rotations using spectral norm constraints (i.e., α = 0.90, 0.75, and 0.67) for all algorithms. All the volumes have size 129 × 129 × 129 voxels. The result from the IRLS procedure with α = 0.67 for the spectral norm constraint is best.

Figure 9.

Initial models and refined models. (a) The ab initio models estimated by merging two independent reconstructions, each obtained from 1000 class averages. The resolutions of the models are 17.2Å, 16.7Å, 16.7Å, 16.7Å, and 16.1Å (from top to bottom, left to right) using the FSC 0.143 resolution cutoff (Figure 10). The model using the IRLS procedure without the spectral norm constraint (i.e., α = N/A) is not available, since the estimated projection directions are highly clustered. (b) The refined models corresponding to the ab initio models in (a). The resolutions of the models are all 11.1Å.

To evaluate the accuracy or the resolution of the reconstructions, we used the 3D Fourier shell correlation (FSC) [43]. FSC measures the normalized cross correlation coefficient between two 3D volumes over corresponding spherical shells in Fourier space, i.e.,

$$\text{FSC}(i)=\frac{{\sum }_{\mathbf{j}\in \mathit{Shel}{l}_i}\mathcal{F}({\mathbf{V}}_1)(\mathbf{j})·\overline{\mathcal{F}({\mathbf{V}}_2)(\mathbf{j})}}{\sqrt{{\sum }_{\mathbf{j}\in \mathit{Shel}{l}_i}{\mid \mathcal{F}({\mathbf{V}}_1)(\mathbf{j})\mid }^2·{\sum }_{\mathbf{j}\in \mathit{Shel}{l}_i}{\mid \mathcal{F}({\mathbf{V}}_2)(\mathbf{j})\mid }^2}},$$ (8.1)

where (V₁) and (V₂) are the Fourier transforms of volumes V₁ and V₂, respectively; the spatial frequency i ranges from 1 to N/2 − 1 times the unit frequency 1/(N · pixel size); N is the size of a volume; and Shell_i := {j : 0.5 + (i −1) + ε ≤ ||j|| < 0.5 + i + ε}, where ε = 1e − 4. In this form, the FSC takes two 3D volumes and converts them into a 1D array. In section 8.2, we used the FSC 0.143 cutoff criterion [44, 2] to determine the resolutions of the ab initio models and the refined models.

8.1. Experiments on simulated images

We simulated 500 centered images of size 129 × 129 pixels with pixel size 2.4Å of the 50S ribosomal subunit (the top image in Figure 5), where the orientations of the images were sampled from the uniform distribution over SO(3). White Gaussian noise was added to the clean images to generate noisy images with SNR = 1/16, 1/32, and 1/64 (Figure 3). Common-line pairs that were detected with an error smaller than 10° were considered to be correct. The common-line detection rates were 64%, 44%, and 23% for images with SNR = 1/16, 1/32, and 1/64, respectively (Figure 2).

Figure 3.

The first column shows three clean images of size 129×129 pixels generated from a 50S ribosomal subunit volume with different orientations. The other three columns show three noisy images corresponding to those in the first column with SNR = 1/16, 1/32, and 1/64, respectively.

To measure the accuracy of the estimated orientations, we defined the mean squared error (MSE) of the estimated rotation matrices R̂₁, …, R̂_K to be

$$\text{MSE}=\frac{1}{K}\sum _{i=1}^K{\Vert R_i-\widehat{O}{\widehat{R}}_i\Vert }^2,$$ (8.2)

where Ô is the optimal solution to the registration problem between the two sets of rotations {R₁, …, R_K} and {R₁, …, R̂_K} in the sense of minimizing the MSE. As shown in [48], there is a simple procedure for obtaining both Ô and the MSE from the singular value decomposition of the matrix $\frac{1}{K}{\sum }_{i=1}^K{\widehat{R}}_i{R}_i^T$.

We applied the LS approach using SDP and ADMM, and the LUD approach using ADMM and IRLS to estimate the images’ orientations, then computed the MSEs of the estimated rotation matrices, and lastly reconstructed the volume (Figures 5 and 6). In order to measure the accuracy of the reconstructed volumes, we measured each volume’s FSC (8.1) (Figure 7) against the clean 50S ribosomal subunit volume; that is, in our measurement V₁ was the reconstructed volume, and V₂ was the “ground truth” volume.

Figure 7.

FSCs (8.1) of the reconstructed volumes against the clean volume in Figures 5 and 6. The plots of the correlations show that the LUD approach using ADMM and IRLS (denoted as the blue dashed lines and green solid lines) outweighed the LS approach using SDP or ADMM (denoted as the red dot-dashed lines). Note that all of the last four subfigures are results for images with SNR = 1/64, where the last three subfigures are results using different α for the spectral norm constraint. In the third subfigure (left to right, top to bottom), there is no green solid line for the LUD approach using IRLS, since the IRLS procedure without the spectral norm constraint converges to a solution where the estimated projection directions are highly clustered and no 3D reconstruction can be computed.

When SNR= 1/16 and 1/32, the common-line detection rate was relatively high (64% and 44%), and the algorithms without the spectral norm constraint on G were enough to make a good estimation. The LUD approach using ADMM and IRLS outweighed the LS approach in terms of accuracy measured by MSE and FSC (Figures 5–7). Note that the LS approach using SDP failed when SNR = 1/32, while the LUD approach using either ADMM or IRLS succeeded. When SNR = 1/64, the common-line detection rate was relatively low (23%), and most of the detected common-lines were outliers (Figure 2). As a consequence, we observed that the algorithms without spectral norm constraint ||G||₂ ≤ αK did not work. In particular, the viewing directions of images estimated by the IRLS procedures without ||G|| ≤ αK converged to two clusters around two antipodal directions, yielding no 3D reconstruction. The LUD approach using ADMM failed in this case; however, the IRLS procedure with an appropriate regularization on the spectral norm (i.e., α = 0.67 since the true rotations were uniformly sampled over SO(3)) gave the best reconstruction.

8.2. Experiments on a real dataset

A set of micrographs of E. coli 50S ribosomal subunits was provided by Dr. M. van Heel. These micrographs were acquired by a Philips CM20 at defocus values between 1.37 and 2.06μm, and they were scanned at 3.36 Å/pixel. The particles (particularly E. coli 50S ribosomal subunits) were picked using the automated particle picking algorithm in EMAN Boxer [24]. Then, using the IMAGIC software package [52, 58], the 27,121 particle images of size 90 × 90 pixels were phase-flipped to remove the phase-reversals in the contrast transfer function (CTF), bandpass filtered at 1/150 and 1/8.4 Å, normalized by their variances, and then translationally aligned with the rotationally averaged total sum. The particle images were randomly divided into two disjoint groups of equal number of images. The following steps were performed on each group separately.

The images were rotationally aligned and averaged to produce class averages of better quality, following the procedure detailed in [66]. For each group, the images were denoised and compressed using Fourier–Bessel-based steerable principal component analysis (FBSPCA) [65]. Then, triple products of Fourier–Bessel expansion coefficients obtained in FBSPCA were used to compute rotationally invariant features of the images, i.e, the bispectrums [42, 21, 27]. For each image, an initial set of neighboring images was computed using the normalized cross-correlation of the bispectrums, which was later refined using the method described in [49] to produce new sets of neighbors. Finally, we averaged each image with its ten nearest neighbors after alignment.⁵ Three examples of averaged images are shown in Figure 8.

Figure 8.

Noise reduction by image averaging. Three raw ribosomal images are shown in the first column. Their closest two neighbors (i.e., raw images having similar orientations after alignments) are shown in the second and third columns. The average images shown in the last column were obtained by averaging over 10 neighbors of each raw image.

One thousand class averages were randomly selected from each group. The LS and LUD approaches with and without the spectral norm constraint were applied. Two reconstructed volumes were obtained from the two groups of images. The two resulting volumes were aligned and averaged to obtain the ab initio model (Figure 9(a)). We observed that the LUD approach gives much more reasonable ab initio models compared to the LS approach. In addition, the FSC of the two volumes was computed to estimate the resolution of the ab initio model (Figure 10). Among all the ab initio models, the one obtained by LS is at the lowest resolution, 17.2Å, while the one obtained by LUD through IRLS procedure has the highest resolution, 16.1Å. Notice that the FSC measures the variance error but not the bias error of the ab initio model. We also notice that the viewing directions of images estimated by the IRLS procedures without the spectral norm constraint converged to two clusters around two antipodal directions, resulting in no 3D reconstruction. Moreover, for this dataset, adding the spectral norm constraint on G with α = 0.85 did not improve the accuracy of the result, although this helped with regularizing the convergence in the IRLS procedure.

Figure 10.

Convergence of the refinement process. In panels (a)–(e), the FSC plots show the convergence of the refinement iterations. The ab initio models (Figure 9(a) used in (a)–(e) were obtained by solving the LS/LUD problems using SDP/ADMM/IRLS. The numbers of refinement iterations performed in (a)–(e) are 7, 3, 5, 3, and 4, respectively. Panel (f) shows FSC plots of the refined models in (a), (b), (c), and (e) against the refined model in (d), which are measurements of similarities between the refined models in Figure 9(b).

The two resulting volumes were then iteratively refined using 10,000 raw images in each group. In each refinement iteration, 2,000 template images were generated by projecting the 3D model from the previous iteration, then the orientations of the raw images were estimated using reference-template matching, and finally a new 3D model was reconstructed from the 10,000 raw images with highest correlation with the reference images. Each refinement iteration took about four hours. Therefore, a good ab initio model should be able to accelerate the refinement process by reducing the total number of refinement iterations. The FSC plots in Figure 10(a)–10(e) show the convergence of the refinement process using different ab initio models. We observed that all the refined models are at the resolution of 11.1Å. However, the worst ab initio model obtained by LS needed 7 iterations (about 28 hours) for convergence (Figure 10(a)), while the best ab initio model obtained by LUD needed 3 iterations (about 12 hours) for convergence (Figure 10(b) and (d)). Figure 10(f) uses FSC plots to compare the refined models. We observed that the refined models in Figure 10(b)–10(e) were consistent with each other, while the refined model obtained by LS in Figure 10(a) was slightly different from the others.

The average CPU times for computing the ab initio models in these two subsections are shown in Table 1. It is not surprising to see that the LS approach was the fastest and that adding the spectral norm constraint slowed down the ADMM and IRLS procedures. The reason for this slowing is that a large portion of the CPU time in ADMM and IRLS is due to the projections onto the semidefinite cone. These steps are expected to be accelerated by the recent advance in eigenspace computation [63]. However, when using the LUD approach for the real dataset, the time saved in the refinement was about 16 hours, which is much more than the time cost for computing the ab initio models (about 0.5–1 hour when ADMM was used).

Table 1.

The average cost time using different algorithms on 500 and 1000 images in the two experimental subsections. The notation α = N/A means that no spectral norm constraint ||G||₂ ≤ αK is used.

K	α = N/A			$\frac{2}{3}\le \alpha \le 1$
	LS (SDP)	LUD		LS (ADMM)	LUD
		ADMM	IRLS		ADMM	IRLS
500	7s	266s	469s	78s	454s	3353s
1000	31s	1864s	3913s	619s	1928s	20918s

9. Discussion

To estimate image orientations, we introduced a robust self-consistency error and used the ADMM or IRLS procedure to solve the associated LUD problem after SDR. Numerical experiments demonstrate that the solution is less sensitive to outliers in the detected common-lines than is the LS method approach. In addition, when the common-line detection rate is low, the spectral norm constraint on the Gram matrix G can help to tighten the semidefinite relaxation, and thus improves the accuracy of the estimated rotations in some cases. Moreover, the numerical experiments using the real dataset (section 8.2) demonstrate that the ab initio models resulting from the LUD-based methods are more accurate than initial models that result from least squares–based methods. In particular, our initial models require fewer time-consuming refinement iterations. We note that it is also possible to consider other self-consistency errors involving the unsquared deviations raised to some power p (e.g., the cases p = 1, 2 correspond to LUD and LS, respectively). We observed that the accuracy of the estimated orientations can be improved by using p < 1, provided that the initial guess is “sufficiently good.” The LUD approach and the spectral norm constraint on G can be generalized to the synchronization approach to estimate the images’ orientations [46].

Both LS and LUD-based algorithms have weaknesses and strengths in applications. The LS method is much faster than the LUD method, while the LUD method is more robust to outliers in the detected common-lines than is the LS method. Moreover, adding a spectral norm constraint can tighten the convex relaxation and thus improve the accuracy of the result, while it slows down the whole procedure. Based on the observations in our numerical experiments, we recommend using the LS method, the LUD method, and both methods with spectral norm constraints, respectively, for images with SNR from high to low. However, it is hard to quantify the actual value of SNR, due to the unknown 3D structure, and to choose a method accordingly. We suggest trying the LS method without any spectral norm constraint first, then examine the top four or five eigenvalues of the Gram matrix G to see whether G has three dominate eigenvalues. If the spectrum of G is reasonable, we can examine the ab initio model that is reconstructed using the estimated rotations. If one of the two examinations fails, the LUD and the spectral norm constraint can be considered to improve the result. One can also experiment with different orientation determination algorithms and choose the result with the highest likelihood score. Once an ab initio estimation of orientations is given, it is possible to evaluate the likelihood score of the estimation in one of two ways: (1) determine the “common-lines” by the estimated rotations, compute the Fourier coefficients in the common-lines, and then compute the sum of squared distances between the implied common-lines in the Fourier space; or (2) reconstruct a 3D volume using the estimated rotations and compute the sum of deviations between the original input 2D images and the corresponding 2D projections of the 3D volume (i.e., the reprojection error).

In [61], the LUD approach is shown to be more robust than the LS approach for the synchronization problem over the rotation group SO(d). Given some relative rotations $R_i^T{R}_j$, the synchronization problem is to estimate the rotations R_i ∈ SO(d), i = 1, …, K, up to a global rotation. It is verified that under a specific model of the measurement noise and the measurement graph for $R_i^T{R}_j$, the rotations can be exactly and stably recovered using LUD, exhibiting a phase transition behavior in terms of the proportion of noisy measurements. The problem of orientation determination using common-lines between cryo-EM images is similar to the synchronization problem. The difference is that the pairwise information given by the relative rotation $R_i^T{R}_j$ is full, while that given by the common-lines ${\overrightarrow{c}}_{ji}^T{\overrightarrow{c}}_{ij}$ is partial. Moreover, the measurement noise of each detected common-line c⃗_ij depends on images i and j, and thus it cannot be simply modeled, which brings the difficulties in verifying the conditions for the exact and stable orientation determination that we observed.

Appendix. Exact recovery of the Gram matrix G from correct common-lines

Here we prove that if the detected common-lines c⃗_ji (defined in (3.1)) are all correct and at least three images have linearly independent projection directions (i.e., the viewing directions of the three images are not on the same great circle on the sphere shown in Figure 4), then the Gram matrix G obtained by solving the LS problem (4.6)–(4.8) or the LUD problem (4.10) is uniquely that defined in (4.3). To verify the uniqueness of the solution G, it is enough to show rank(G)= 3 due to the SDP solution uniqueness theorem (see [7, pp. 36–39], [68]). Without loss of generality, we consider the SDP for the LS approach when applied on three images (i.e., K = 3 and w_ij = 1 in the problem (4.6)–(4.8)):

$$\underset{G_{6\times 6}\succcurlyeq 0}{max}\sum _{i,j=1,2,3}\langle G_{ij},{\overrightarrow{c}}_{ji}^T{\overrightarrow{c}}_{ij}\rangle \mathrm{s}.\mathrm{t}.G_{ii}=I_2.$$

Since the solution G is positive semidefinite, we can decompose G as

$$G=\left(\begin{array}{c}{{\mathbf{u}}_1^1}^T\\ {{\mathbf{u}}_1^2}^T\\ {{\mathbf{u}}_2^1}^T\\ {{\mathbf{u}}_2^2}^T\\ {{\mathbf{u}}_3^1}^T\\ {{\mathbf{u}}_3^2}^T\end{array}\right)(\begin{array}{cccccc}{\mathbf{u}}_1^1& {\mathbf{u}}_1^2& {\mathbf{u}}_2^1& {\mathbf{u}}_2^2& {\mathbf{u}}_3^1& {\mathbf{u}}_3^2\end{array}),$$

where ${\mathbf{u}}_i^p$, p = 1, 2 and i = 1, 2, 3, are column vectors. We will show rank(G)= 3; i.e., any four vectors among { ${\mathbf{u}}_1^1,{\mathbf{u}}_1^2,{\mathbf{u}}_2^1,{\mathbf{u}}_2^2,{\mathbf{u}}_3^1,{\mathbf{u}}_3^2$} span a space with dimensionality at most 3.

Define arrays u_i as

$${\mathbf{u}}_i=({\mathbf{u}}_i^1,{\mathbf{u}}_i^2);$$

then the inner product

$$\begin{array}{l}\langle G_{ij},{\overrightarrow{c}}_{ji}^T{\overrightarrow{c}}_{ij}\rangle =\langle {\mathbf{u}}_i^T{\mathbf{u}}_j,{\overrightarrow{c}}_{ji}^T{\overrightarrow{c}}_{ij}\rangle \\ =\langle {\overrightarrow{c}}_{ji}{\mathbf{u}}_i,{\overrightarrow{c}}_{ij}{\mathbf{u}}_j\rangle \\ =\langle c_{ji}^1{\mathbf{u}}_i^1+c_{ji}^2{\mathbf{u}}_i^2,c_{ij}^1{\mathbf{u}}_j^1+c_{ij}^2{\mathbf{u}}_j^2\rangle \\ \le 1,\end{array}$$

where the last inequality follows from the Cauchy–Schwarz inequality and the facts that all c⃗_ij are unit vectors, ${\mathbf{u}}_i^1$ and ${\mathbf{u}}_i^2$ are unit vectors and orthogonal to each other due to the constraint G_ii = I₂, and thus all $c_{ij}^1{\mathbf{u}}_j^1+c_{ij}^2{\mathbf{u}}_j^2$ are unit vectors on the Fourier slices of the images. The equality holds if and only if

$$c_{ji}^1{\mathbf{u}}_i^1+c_{ji}^2{\mathbf{u}}_i^2=c_{ij}^1{\mathbf{u}}_j^1+c_{ij}^2{\mathbf{u}}_j^2.$$ (A.1)

Thus when the maximum is achieved, due to (A.1) and the fact that the projection directions of the images are linearly independent, $dim(\text{span}\{{\mathbf{u}}_i^1,{\mathbf{u}}_i^2\}\cap \text{span}\{{\mathbf{u}}_j^1,{\mathbf{u}}_j^2\})=1$, and thus $dim(\text{span}\{{\mathbf{u}}_i^1,{\mathbf{u}}_i^2,{\mathbf{u}}_j^1,{\mathbf{u}}_j^2\})=3$. Therefore, without loss of generality, we only have to show that $dim(\text{span}\{{\mathbf{u}}_1^1,{\mathbf{u}}_2^1,{\mathbf{u}}_3^1,{\mathbf{u}}_3^2\})\le 3$. Using (A.1), assume that $\text{span}\{{\mathbf{u}}_1^1,{\mathbf{u}}_1^2\}\cap \text{span}\{{\mathbf{u}}_3^1,{\mathbf{u}}_3^2\}=\text{span}({\mathbf{v}}_1)$ and $\text{span}\{{\mathbf{u}}_2^1,{\mathbf{u}}_2^2\}\cap \text{span}\{{\mathbf{u}}_3^1,{\mathbf{u}}_3^2\}=\text{span}\{{\mathbf{v}}_2\}$, where v₁ and v₂ are linearly independent vectors (otherwise all three projection directions are linearly dependent, and thus the three Fourier slices of the images intersect at the same line). Therefore we have $\text{span}\{{\mathbf{v}}_1,{\mathbf{v}}_2\}=\text{span}\{{\mathbf{u}}_3^1,{\mathbf{u}}_3^2\},\text{span}\{{\mathbf{v}}_1,{\mathbf{u}}_1^1\}\subseteq \text{span}\{{\mathbf{u}}_1^1,{\mathbf{u}}_1^2\}$, and $\text{span}\{{\mathbf{v}}_2,{\mathbf{u}}_2^1\}\subseteq \text{span}\{{\mathbf{u}}_2^1,{\mathbf{u}}_2^2\}$. Thus $dim(\text{span}\{{\mathbf{u}}_1^1,{\mathbf{u}}_2^1,{\mathbf{u}}_3^1,{\mathbf{u}}_3^2\})=dim(\text{span}\{{\mathbf{u}}_1^1,{\mathbf{u}}_2^1,{\mathbf{v}}_1,{\mathbf{v}}_2\})\le dim(\text{span}\{{\mathbf{u}}_1^1,{\mathbf{u}}_1^2,{\mathbf{u}}_2^1,{\mathbf{u}}_2^2\})\le 3$.