Fast Steerable Principal Component Analysis

Abstract

Cryo-electron microscopy nowadays often requires the analysis of hundreds of thousands of 2-D images as large as a few hundred pixels in each direction. Here, we introduce an algorithm that efficiently and accurately performs principal component analysis (PCA) for a large set of 2-D images, and, for each image, the set of its uniform rotations in the plane and their reflections. For a dataset consisting of n images of size L × L pixels, the computational complexity of our algorithm is O(nL³ + L⁴), while existing algorithms take O(nL⁴). The new algorithm computes the expansion coefficients of the images in a Fourier–Bessel basis efficiently using the nonuniform fast Fourier transform. We compare the accuracy and efficiency of the new algorithm with traditional PCA and existing algorithms for steerable PCA.

I. Introduction

Principal component analysis (PCA) is widely used in image analysis and pattern recognition for dimensionality reduction and denoising. In particular, PCA is often one of the first steps [1] in the algorithmic pipeline of cryo-electron microscopy (cryo-EM) single particle reconstruction (SPR) [2] to compress and denoise the acquired 2D projection images in order to eventually determine the 3D structure of a macromolecule. The high level of noise in those images drastically deteriorates the performance of single-image based denoising methods, such as non-local means [3] and wavelet thresholding [4], and so the latter are outperformed by PCA. As any planar rotation of any given projection image is equally likely to appear in the experiment, by either in-plane rotating the detector or the specimen, it makes sense to include all possible rotations of the projection images when performing PCA. The resulting decomposition, termed steerable PCA, consists of principal components which are tensor products of radial functions and angular Fourier modes [5], [6], [7], [8], [9]. Beyond cryo-EM, steerable PCA has many other applications in image analysis and computer vision [10].

The term “steerable PCA” comes from the fact that rotating the principal components is achieved by a simple phase shift of their angular part. The principal components are invariant to any in-plane rotation of the images, therefore finding steerable principal components is equivalent to finding in-plane rotationally invariant principal components.

In cryo-EM data processing, in addition to compression and denoising, steerable PCA is also useful in generating rotationally invariant image features (i.e. bispectrum-like features [11]). These are crucial for fast rotationally invariant nearest neighbors search used in efficient computation of class averages [11]. Rotational alignment between image pairs can also be computed more efficiently using the expansion coefficients in a steerable basis.

In this letter, we focus on the action of the group O(2) on digital images by in-plane rotating and possibly reflecting them. The idea of using group actions for constructing group invariant features and filters has been previously proposed in [12], [13]. This group theoretical framework has been applied to SO(3) and SU(1, 1) in [14], [15]. The representation of finite groups, such as the dihedral groups, has been used for computing the Karhunen-Loéve expansion of digital images in [16].

Various efficient algorithms for steerable PCA have been introduced [17], [8]. However, steerable PCA of modern cryo-EM datasets that contain hundreds of thousands of large images poses a computational challenge. Also, it is important to ensure that the steerable PCA algorithm is numerically accurate when the input images are noisy. In order to exploit the special separation of variables structure of the principal components in polar coordinates, most algorithms rely on resampling the images on a polar grid. However, the transformation from Cartesian to polar is non-unitary, and thus changes the statistics of the noise. In particular, resampling transforms uncorrelated white noise to colored noise that may lead to spurious principal components.

Recently, [9] addressed this issue by incorporating a sampling criterion into the steerable PCA framework and introduced an algorithm called Fourier-Bessel steerable PCA (FBsPCA). FBsPCA assumes that the underlying clean images (before being possibly contaminated with noise) are bandlimited and essentially compactly supported in a disk. This assumption holds, for example, for 2D projection images of a 3D molecule compactly supported in a ball. It also implies that the images can be expanded in an orthogonal basis for bandlimited functions, such as the Fourier-Bessel basis. In FBsPCA, the Fourier-Bessel expansion of each image is truncated into a finite series using a sampling criterion that was introduced by Klug and Crowther [18]. The sampling criterion ensures that the transformation from the Cartesian grid to the truncated Fourier-Bessel expansion is nearly unitary. Moreover, the covariance matrix built from the expansion coefficients of the images and all their possible rotations has a block diagonal structure where the block size decreases as a function of the angular frequency. The computational complexity of FBsPCA is O(nL⁴) operations for n images of size L × L. Notice that, when n > L², the computational complexity of traditional PCA is O(nL⁴ + L⁶), where the first term corresponds to forming the L² × L² covariance matrix and the second term corresponds to its eigen-decomposition. Although FBsPCA and PCA have a similar computational complexity, FBsPCA leads to better denoising as it takes into account all possible rotations and reflections. This makes FBsPCA more suitable than traditional PCA as a tool for 2D analysis of cryo-EM images [9]. With the enhancement of electron microscope detectors’ resolution, a typical image size of a single particle can easily be over 300 × 300 pixels. Thus, FBsPCA is still not efficient enough to analyze a large number of images of large size (i.e. large n and large L). The bottleneck for this algorithm is the first step that computes the Fourier-Bessel expansion coefficients, whose computational complexity is O(nL⁴).

In this letter we introduce a fast Fourier-Bessel steerable PCA (FFBsPCA) that reduces the computational complexity for FBsPCA from O(nL⁴) to O(nL³) by computing the Fourier-Bessel expansion coefficients more efficiently and accurately. This is achieved by first mapping the images from their Cartesian grid representation to a polar grid representation in the reciprocal (Fourier) domain using the non-uniform fast Fourier transform (NUFFT) [19], [20], [21], [22]. The polar grid representation enables to efficiently evaluate the Fourier-Bessel expansion coefficients of the images by 1D FFT on concentric circles followed by accurate evaluation of a radial integral with a Gaussian quadrature rule. The overall complexity of computing the Fourier-Bessel coefficients is reduced to O(nL³) operations. The increased accuracy and efficiency in evaluating the Fourier-Bessel expansion coefficients are the main contributions of this letter.

We note that the Fourier-Bessel expansion coefficients can be computed in O(nL² log L) operations using algorithms for rapid evaluation of special functions [23] or a fast analysis-based Fourier-Bessel expansion [24]. However, such “fast” algorithms may only lead to a marginal improvement for two reasons. First, the break even point for them compared to the direct approach is for relatively large L such as L = 256 or larger. Second, forming the covariance matrix from the expansion coefficients still requires O(nL³) operations.

The letter is organized as follows: Section II contains the mathematical preliminaries of the Fourier-Bessel expansion, the sampling criterion, and the numerical evaluation of the expansion coefficients. The computation of the steerable principal components is described in Section III. We present the algorithm and give a detailed computational complexity analysis in Section IV. Various numerical examples concerning the computation time of FFBsPCA compared with FBsPCA and traditional PCA are presented in Section V. In the same section, we demonstrate the performance of FFBsPCA-based denoising using simulated cryo-EM projection images.

Reproducible research: The FFBsPCA is available in the SPR toolbox ASPIRE (http://spr.math.princeton.edu/). There are two main functions: FBCoeff computes the Fourier Bessel expansion coefficients and sPCA computes the steerable PCA basis and the associated expansion coefficients.

II. Fourier-Bessel Expansion of Bandlimited Images

We say that f has a band limit radius c if its Fourier transform

$$ℱ(f)({\xi }_1,{\xi }_2)={\int }_{{ℝ}^2}f(x,y)e^{-2\pi \iota (x{\xi }_1+y{\xi }_2)}dxdy$$ (1)

satisfies ℱ(f)(ξ₁, ξ₂) = 0, for ${\xi }_1^2+{\xi }_2^2>c^2$. In our setup, a digital image I is obtained by sampling a squared-integrable bandlimited function f on a Cartesian grid of size L × L, that is, I(i₁, i₂) = f(i₁Δ, i₂Δ), where $i_1,i_2=-\lceil \frac{L-1}{2}\rceil ,\dots ,\lfloor \frac{L-1}{2}\rfloor$, and Δ is the pixel size.

For pixel size Δ = 1, the Nyquist-Shannon sampling theorem implies that the Fourier transform of I is supported on the square [−1/2, 1/2) × [−1/2, 1/2). In many applications, the support size is effectively smaller due to other experimental considerations, for example, the exponentially decaying envelope of the contrast transfer function in electron microscopy. Thus, we assume that the band limit radius of all images is $0<c\le \frac{1}{2}$. The scaled Fourier-Bessel functions are the eigenfunctions of the Laplacian in a disk of radius c with Dirichlet boundary condition and they are given by

$${\psi }_c^{k,q}(\xi ,\theta )=\{\begin{array}{cc}\hfill N_{k,q}{J}_k\left(R_{k,q}\frac{\xi }{c}\right)e^{\iota k\theta },\hfill & \hfill \xi \le c,\hfill \\ \hfill 0,\hfill & \hfill \xi >c,\hfill \end{array}$$ (2)

where (ξ, θ) are polar coordinates in the Fourier domain (i.e., ξ₁ = ξ cos θ, ξ₂ = ξ sin θ, ξ ≥ 0, and θ ∈ [0, 2π)); $N_{k,q}={(c\sqrt{\pi }|J_{k+1}(R_{k,q})|)}^{-1}$ is the normalization factor; J_k is the Bessel function of the first kind of integer order k; and R_k,q is the q_th root of the Bessel function J_k. For a function f with band limit c that is also in L²(ℝ²) ∩ L¹(ℝ²),

$$ℱ(f)(\xi ,\theta )=\sum _{k=-\infty }^{\infty }\sum _{q=1}^{\infty }{a}_{k,q}{\psi }_c^{k,q}(\xi ,\theta ),$$ (3)

which converges pointwise. In Section II-A, we derive a finite truncation rule for the Fourier-Bessel expansion in Eq. (3).

A. Sampling Criterion

For digital implementations of Eq. (3), we must truncate it to a finite sum, namely to derive a sampling criterion for selecting k and q.

With the following convention for the 2D inverse polar Fourier transform of a function g(ξ, θ),

$${ℱ}^{-1}(g)(r,\varphi )={\int }_0^{2\pi }{\int }_0^{\infty }g(\xi ,\theta )e^{2\pi \iota r\xi \text{ cos}(\theta -\varphi )}\xi d\xi d\theta ,$$ (4)

the 2D inverse Fourier transform of the Fourier-Bessel functions, denoted ${ℱ}^{-1}({\psi }_c^{k,q})$, is given in polar coordinates as

$${ℱ}^{-1}({\psi }_c^{k,q})(r,\varphi )=\frac{2c\sqrt{\pi }{(-1)}^q{R}_{k,q}{J}_k(2\pi cr)}{{\iota }^k({(2\pi cr)}^2-R_{k,q}^2)}{e}^{\iota k\varphi }.$$ (5)

The maximum of $|{ℱ}^{-1}({\psi }_c^{k,q})(r,\varphi )|$ in (5) is obtained near the circle $r=\frac{R_{k,q}}{2\pi c}$ and ${ℱ}^{-1}({\psi }_c^{k,q})(r,\varphi )$ vanishes on concentric circles of radii $r=\frac{R_{k,q\prime }}{2\pi c}$ with q′ ≠ q. The smallest circle with vanishing ${ℱ}^{-1}({\psi }_c^{k,q})$ that encircles the maximum of $|{ℱ}^{-1}({\psi }_c^{k,q})|$ is of radius $r=\frac{R_{k,(q+1)}}{2\pi c}$.

We assume that the underlying clean images (before being possibly contaminated with noise) are essentially compactly supported in a disk of radius R. Therefore, we should rule out Fourier-Bessel functions for which the maximum of their inverse Fourier transform resides outside a disk of radius R. Otherwise, those functions introduce spurious information from noise. Notice that if the maximum is inside the disk, yet the zero after the maximum is outside the disk, then there is a significant spillover of energy outside the disk. We therefore require the more stringent criterion that the zero after the maximum is inside the disk, namely

$$\frac{R_{k,(q+1)}}{2\pi c}\le R.$$ (6)

This sampling argument gives a finite truncation rule for the Fourier-Bessel expansion in Eq. (3), that is

$$R_{k,(q+1)}\le 2\pi cR.$$ (7)

For each k, we denote by p_k the number of components satisfying Eq. (7). We also denote by $p={\sum }_{k=-k_{\text{max}}}^{k_{\text{max}}}{p}_k$ the total number of components, where k_max is the maximal possible value of k satisfying Eq. (7). The locations of Bessel zeros have been extensively studied, for example, in [25, p.517–521], [26, p.370], [27], [28], [29]. Several lower and upper bounds for Bessel zeros R_k,q were proven by Breen in [29], such as

$$R_{k,q}>k+\frac{2}{3}{|a_{q-1}|}^{3/2},$$ (8)

where a_q is the qth zero of the Airy function, shown to satisfy

$${[\frac{3}{8}\pi (4q-1.4)]}^{2/3}<|a_q|<{[\frac{3}{8}\pi (4q-0.965)]}^{2/3}.$$ (9)

Using the lower bound for |a_q| and the sampling criterion in Eq. (7), we have the following inequality for k and p_k,

$$2\pi cR\approx R_{k,(p_k+1)}>k+\pi p_k-\frac{1.4\pi }{4}.$$ (10)

Breen also obtained

$$R_{k,q}<(\frac{k}{2}+q-\frac{0.965}{4})\pi ,$$ (11)

so we get another inequality for k and p_k,

$$2\pi cR\approx R_{k,(p_k+1)}<(\frac{k}{2}+p_k+\frac{3.035}{4})\pi .$$ (12)

Combining Eqs. (10) and (12), we have the following lower and upper bounds for p_k,

$$2cR-\frac{k}{2}-\frac{3.035}{4}<p_k<2cR-\frac{k}{\pi }+\frac{1.4}{4}.$$ (13)

The bound for the highest angular frequency k_max is determined by setting p_k = 1 in Eq. (13), resulting in

$$4cR-3.517<k_{\text{max}}<2\pi cR-2.042.$$ (14)

Equation (13) implies that as the angular frequency k increases, the number of components p_k decreases. Moreover, using the lower and upper bounds for p_k and k_max in Eqs. (13) and (14), we derive that the total number of selected Fourier-Bessel basis functions is between 8(cR)² and 4π(cR)². When c is the largest possible band limit, i.e. $c=\frac{1}{2}$, the number of basis functions is between 2R² and πR², where the latter is approximately the number of pixels inside a disk of radius R. Also, whenever c = O(1) and R = O(L), we get that p = O(L²) and k_max = O(L).

Because the bandlimited function f is assumed to be essentially compactly supported, the infinite expansion in Eq. (3) is approximated by the finite expansion

$$P_{c,R}ℱ(f)(\xi ,\theta )=\sum _{k=-k_{\text{max}}}^{k_{\text{max}}}\sum _{q=1}^{p_k}{a}_{k,q}{\psi }_c^{k,q}(\xi ,\theta ),$$ (15)

where P_c,R is the orthogonal projection from L²(D_c) (the space of L² functions supported on a disk of radius c), to the space of functions spanned by a finite number of Fourier-Bessel functions that satisfy (7).

B. Numerical Evaluation of Fourier–Bessel Expansion Coefficients

Previously in [9], the evaluation of the expansion coefficients a_k,q of Eq. (15) was done by least squares. Let Ψ be the matrix whose entries are evaluations of the Fourier-Bessel functions at the Cartesian grid points, with rows indexed by the grid points and columns indexed by angular and radial frequencies. Finding the coefficient vector a as the solution to ${\text{min}}_a{\Vert \mathrm{\Psi }a-I\Vert }_2^2$ requires the computation of Ψ*I, which takes O(pL²) = O(L⁴) operations, because p = O(L²). In general a = (Ψ*Ψ)⁻¹Ψ*I, but here Ψ*Ψ is approximately the identity matrix, due to the orthogonality of the Fourier-Bessel functions.

We introduce here a method that computes the expansion coefficients a_k,q in O(L³) operations instead of O(L⁴). The expansion coefficients in Eq. (15) are given analytically by

$$a_{k,q}={\int }_0^{2\pi }{\int }_0^cℱ(f)(\xi ,\theta )\overline{{\psi }_c^{k,q}(\xi ,\theta )}\xi d\xi d\theta ={\int }_0^c{N}_{k,q}{J}_k\left(R_{k,q}\frac{\xi }{c}\right)\xi d\xi {\int }_0^{2\pi }ℱ(f)(\xi ,\theta )e^{-\iota k\theta }d\theta .$$ (16)

We evaluate the last integral numerically using a quadrature rule that consists of equally spaced points in the angular direction and a Gaussian quadrature rule in the radial direction, that is, using the nodes, ξ₁(j, l) = ξ_j cos(2πl/n_θ), ξ₂(j, l) = ξ_j sin(2πl/n_θ), j = 1, …, n_ξ, l = 0, …, n_θ − 1 (see Fig. 1). The values of n_ξ and n_θ depend on the compact support radius R and the band limit c and are derived later in the letter. To use this quadrature rule, we need to sample the Fourier transform of f at the quadrature nodes. This is approximated by the Fourier coefficients of the image I (consisting of samples of f on a Cartesian grid) at the given quadrature nodes, namely by the Fourier coefficients

$$F(I)({\xi }_1,{\xi }_2)=\frac{1}{{(2R)}^2}\sum _{i_1=-R}^{R-1}\sum _{i_2=-R}^{R-1}I(i_1,i_2)e^{-\iota 2\pi ({\xi }_1{i}_1+{\xi }_2{i}_2)},$$ (17)

which can be evaluated efficiently using the the nonuniform discrete Fourier transform. The angular integration in Eq. (16) is then sped up by 1D FFT on the concentric circles, followed by a numerical evaluation of the radial integral with a Gaussian quadrature rule.

Fig. 1.

Pictorial summary of the procedure for computing the Fourier-Bessel expansion coefficients. The original image (top left) is resampled on a polar Fourier grid (Eq. (17)) using NUFFT (top right and bottom right) followed by 1D FFT (Eq. (18)) on each concentric circle. The evaluation of the radial integral (Eq. (19)) gives the expansion coefficients a_k,q. The bow-tie phenomenon illustrated in bottom-left was discussed in [31].

As the samples on each concentric circle are equally-spaced, the natural quadrature weights for the angular integral are $\frac{2\pi }{n_{\theta }}$, with the nodes taken at ${\theta }_l=\frac{2\pi l}{n_{\theta }}$ for l = 0, …, n_θ − 1. The angular integration using one-dimensional FFT on each concentric circle thus yields

$$\widehat{F(I)}({\xi }_j,k)=\frac{2\pi }{n_{\theta }}\sum _{l=0}^{n_{\theta }-1}F(I)({\xi }_j,{\theta }_l)e^{-\iota \frac{2\pi kl}{n_{\theta }}}.$$ (18)

The radial integral is evaluated using the Gauss-Legendre quadrature rule [30, Chap. 4], which determines the locations of n_ξ points ${\{{\xi }_j\}}_{j=1}^{n_{\xi }}$ on the interval [0, c] and the associated weights w(ξ_j). The integral in Eq. (16) is thus approximated by

$$a_{k,q}\approx \sum _{j=1}^{n_{\xi }}{N}_{k,q}{J}_k\left(R_{k,q}\frac{{\xi }_j}{c}\right)\widehat{F(I)}({\xi }_j,k){\xi }_{j}w({\xi }_j).$$ (19)

Since I is real valued and J_−k(x) = (−1)^kJ_k(x), we get that $a_{-k,q}=a_{k,q}^{*}$ and thus we only need to evaluate coefficients with k ≥ 0.

The procedure for numerical evaluation of the Fourier-Bessel expansion coefficients is illustrated in Fig. 1. In practice, we have observed that using n_ξ = 4cR and n_θ = 16cR results in highly-accurate numerical evaluation of the integral in Eq. (16).

If our image can be expressed in terms of the truncated Fourier-Bessel expansion in Eq. (15), the approximation error in the radial integral comes from the numerical evaluation of the integrals

$$G(k,q_1,q_2)={\int }_0^c{J}_k\left(R_{k,q_1}\frac{\xi }{c}\right)J_k\left(R_{k,q_2}\frac{\xi }{c}\right)\xi d\xi ,$$ (20)

where the approximation error using n_ξ points is

$$E(k,q_1,q_2;n_{\xi })=\left|\sum _{j=1}^{n_{\xi }}{J}_k\right(R_{k,q_1}\frac{{\xi }_j}{c}\left)J_k\right(R_{k,q_2}\frac{{\xi }_j}{c}){\xi }_{j}w({\xi }_j)-G(k,q_1,q_2)|.$$ (21)

Asymptotically, a Bessel function behaves like a decaying cosine function with frequency $\frac{R_{k,q}}{2\pi }$ for $R_{k,q}r\gg |k^2-\frac{1}{4}|$ [26],

$$J_k(R_{k,q}r)~\sqrt{\frac{2}{\pi R_{k,q}r}}\text{cos}(R_{k,q}r-\frac{k\pi }{2}-\frac{\pi }{4}).$$ (22)

For a fixed n_ξ, the largest approximation error occurs when k = 0 and q₁ = q₂ = p₀, since $J_0\left(R_{0,p_0}\frac{\xi }{c}\right)$ is the most oscillatory function within the band limit. The Nyquist rate of $\xi J_0^2\left(R_{0,p_0}\frac{\xi }{c}\right)$ is $2\frac{2R_{0,p_0}}{2\pi }\approx 4cR$ and we need to sample at Nyquist rate, or higher. Therefore, we choose n_ξ = ⌈4cR⌉. Fig. 2a justifies this choice as the error decays dramatically to 10⁻¹⁷ before n_ξ = ⌈4cR⌉.

Fig. 2.

(a) Error, as a function of n_ξ, in the numerical evaluation of the integral G(0, p₀, p₀) in Eq. (20). (b) Error, as a function of n_θ, in the evaluation of the integral in Eq. (16).

To choose n_θ, we computed the root mean squared error (RMSE) in evaluating the expansion coefficients for simulated images composed of white Gaussian noise with various R and n_θ, while c = 1/2.We oversampled on the radial lines by n_ξ = ⌈10cR⌉ and the ground truth for the angular integral in Eq. (16) was computed by Eq. (18) via oversampling in the angular direction by n_θ = 60cR. We observe that when n_θ ≥ 16cR, the estimation error for the Fourier-Bessel expansion coefficients becomes negligible (see Fig. 2b). Notice that Eq. (14) implies that k_max < 2πcR. The corresponding Nyquist rate is bounded by 4πcR.We therefore sample at a slightly higher rate of n_θ = 16cR to ensure numerical accuracy.

Now that we are able to numerically evaluate a_k,q with high accuracy, we can study the spectral behavior of the finite Fourier-Bessel expansion of the images. We define a as the vector that contains the expansion coefficients a_k,q computed in Eq. (19) and denote by T* the transformation that maps an image I to its finite Fourier-Bessel expansion coefficients through Eqs. (17), (18) and (19), that is,

$$a=T^{*}I.$$ (23)

Ideally we would like T* to be a unitary transformation, that is T*T = I, so that the transformation from the images to the coefficients preserves the noise statistics. Numerically, we observe that the majority of the eigenvalues of T*T are 1 and the smallest eigenvalues are also close to 1 (see blue solid line in Fig. 3). The transformation T* is close to unitary because it is a numerical approximation of an expansion in an orthogonal basis (Fourier-Bessel), and the sampling criterion prevents aliasing. In Fig. 3, the eigenvalues of Ψ*Ψ are also plotted for comparison. It can be observed that T*T has fewer eigenvalues that deviate from 1. Although the Fourier-Bessel functions are orthogonal as continuous functions, their discrete sampled versions are not necessarily orthogonal, hence Ψ*Ψ deviates from the identity matrix. The fact that T*T is closer to the identity than Ψ*Ψ implies that the numerical evaluation of the expansion coefficient vector a as T*I is more accurate than estimating it as Ψ*I.We compare the numerical accuracy explicitly with an example. We choose a signal f that satisfies $ℱf(\xi ,\theta )={\psi }_c^{1,5}(\xi ,\theta )$ for c = 0.5 and R = 30, a_k,q = 1, for k = 1 and q = 5, and otherwise, a_k,q = 0. The evaluation method from [9] is applied here in Fourier space. It first evaluates discrete samples of ℱ(f) and the Fourier-Bessel basis on a Cartesian grid of size 2R × 2R, and then projects the discrete samples onto the basis. The root mean squared error (RMSE) is 7.2 × 10⁻⁵ and the maximum absolute error is 4.0 × 10⁻⁴. Using the numerical evaluation in Eq. (19), we get that RMSE = 1.2 × 10⁻¹⁶ and the maximum absolute error is 2.7 × 10⁻¹⁵.

Fig. 3.

Eigenvalues of T*T and Ψ*Ψ, where T* and Ψ* are the truncated Fourier-Bessel transforms using numerical integration and least squares respectively. These are also the spectra of the population covariance matrices of transformed white noise images. Most eigenvalues are close to 1, indicating that the truncated Fourier-Bessel transform is almost unitary. Thus white noise remains approximately white.

Computing the polar Fourier transform of an image of size L × L on a polar grid with n_ξ × n_θ points in Eq. (17) is implemented efficiently using NUFFT [19], [20], [21], [22], whose computational complexity is O(L² log L + n_ξn_θ). Since n_θ = 16cR = O(L) and n_ξ = 4cR = O(L), n_ξ × n_θ = O(L²) and the complexity of the discrete polar Fourier transform is O(L² log L). The complexity of the 1D FFTs in Eq. (18) is O(n_ξn_θ log n_θ), because there are n_ξ concentric circles with n_θ samples on each circle. Both n_ξ and n_θ are of O(L), so the total complexity of the 1D FFTs is also O(L² log L). Evaluating Eq. (19) (the quadrature rule for the radial integral in Eq. (16)) for all k and q requires a total of O(L³) operations using a direct method, because there are O(L²) basis functions to integrate, and each function is integrated using O(L) quadrature points. However, this complexity can be reduced to O(L² log L) using a fast Bessel transform [23], [24]. In summary, the computational complexity of computing the Fourier-Bessel expansion coefficients of an image of size L × L is O(L³) operations, or O(L² log L) using a “fast” transform.

III. Steerable Principal Components

Given a dataset of n images ${\{I_i\}}_{i=1}^n$, we denote by f_i the underlying bandlimited function that corresponds to the i’th image I_i. Under the action of the group O(2), the function f_i is transformed to $f_i^{\alpha ,\beta }$, where α ∈ [0, 2π) is the counter-clockwise rotation angle and β denotes reflection and takes values in {+, −}. More specifically, $f_i^{\alpha ,+}(r,\varphi )=f_i(r,\varphi -\alpha )$ and $f_i^{\alpha ,-}(r,\varphi )=f_i(r,\pi -(\varphi -\alpha ))$. The images $I_i^{\alpha ,+}$ and $I_i^{\alpha ,-}$ are obtained by sampling $f_i^{\alpha ,+}$ and $f_i^{\alpha ,-}$ respectively.

The Fourier transform of f_i commutes with the action of the group O(2), namely, $ℱ(f_i^{\alpha ,+})(\xi ,\theta )={(ℱ(f_i))}^{\alpha ,+}(\xi ,\theta )=ℱ(f_i)(\xi ,\theta -\alpha )$, and $ℱ(f_i^{\alpha ,-})(\xi ,\theta )={(ℱ(f_i))}^{\alpha ,-}(\xi ,\theta )=ℱ(f_i)(\xi ,\pi -(\theta -\alpha ))$. The transformation of the images under rotation and reflection can be represented by the transformation of their Fourier-Bessel expansion coefficients in Eq. (3). Under counter-clockwise rotation by an angle α, $ℱ(f_i^{\alpha ,+})$ is given by

$$ℱ(f_i^{\alpha ,+})(\xi ,\theta )=\sum _{k,q}{a}_{k,q}^i{\psi }_c^{k,q}(\xi ,\theta -\alpha )=\sum _{k,q}{a}_{k,q}^i{e}^{-\iota k\alpha }{\psi }_c^{k,q}(\xi ,\theta ).$$ (24)

Therefore a planar rotation introduces a phase shift in the expansion coefficients. Under rotation and reflection,

$$ℱ(f_i^{\alpha ,-})(\xi ,\theta )=\sum _{k,q}{a}_{k,q}^i{\psi }_c^{k,q}(\xi ,\pi -(\theta -\alpha ))=\sum _{k,q}{a}_{k,q}^i{N}_{k,q}{J}_k\left(R_{k,q}\frac{\xi }{c}\right)e^{\iota k(\pi -\theta +\alpha )}=\sum _{k,q}{a}_{k,q}^i{N}_{k,q}{(-1)}^k{J}_k\left(R_{k,q}\frac{\xi }{c}\right)e^{\iota (-k)\theta }{e}^{\iota k\alpha }=\sum _{k,q}{a}_{k,q}^i{e}^{\iota k\alpha }{\psi }_c^{-k,q}(\xi ,\theta )=\sum _{k,q}{a}_{-k,q}^i{e}^{-\iota k\alpha }{\psi }_c^{k,q}(\xi ,\theta ),$$ (25)

namely, the expansion coefficient $a_{k,q}^i$ changes to $a_{-k,q}^i{e}^{-ık\alpha }$.

If we augment the collection of bandlimited functions ${\{f_i\}}_{i=1}^n$ by all possible rotations and reflections, the Fourier transform of the sample mean of the augmented collection, denoted f_mean, becomes,

$$ℱ(f_{\text{mean}})(\xi ,\theta )=\frac{1}{2n}\sum _{i=1}^n\sum _{\beta \in \{+,-\}}\frac{1}{2\pi }{\int }_0^{2\pi }ℱ(f_i^{\alpha ,\beta })(\xi ,\theta )d\alpha .$$ (26)

Using the properties in Eqs. (24) and (25), we have

$$ℱ(f_{\text{mean}})(\xi ,\theta )=\frac{1}{2n}\sum _{i=1}^n\frac{1}{2\pi }\times {\int }_0^{2\pi }\sum _{k=-\infty }^{\infty }\sum _{q=1}^{\infty }[a_{k,q}^i+a_{-k,q}^i]e^{-\iota k\alpha }{\psi }_c^{k,q}(\xi ,\theta )d\alpha =\sum _{q=1}^{\infty }\left(\frac{1}{n}\sum _{i=1}^n{a}_{0,q}^i\right){\psi }_c^{0,q}(\xi ,\theta ).$$ (27)

As expected, the sample mean is radially symmetric, because ${\psi }_c^{0,q}$ is only a function of ξ but not of θ.

The rotationally invariant covariance kernel 𝒞((ξ, θ), (ξ′, θ′)) built from Fourier transformed functions with all their possible in-plane rotations and reflections is defined as

$$𝒞((\xi ,\theta ),(\xi \prime ,\theta \prime ))=\frac{1}{4\pi n}\times \sum _{i=1}^n\sum _{\beta \in \{+,-\}}{\int }_0^{2\pi }(ℱ(f_i^{\alpha ,\beta })(\xi ,\theta )-ℱ(f_{\text{mean}})(\xi ,\theta ))\times \overline{(ℱ(f_i^{\alpha ,\beta })(\xi \prime ,\theta \prime )-ℱ(f_{\text{mean}})(\xi \prime ,\theta \prime ))}d\alpha .$$ (28)

From Eq. (27) it follows that if we express ℱ(f_i) and ℱ(f_mean) in terms of the Fourier-Bessel basis and the associated expansion coefficients, subtracting the sample mean is equivalent to subtracting $\frac{1}{n}{\sum }_{j=1}^n{a}_{0,q}^j$ from the coefficients $a_{0,q}^i$, while keeping other coefficients unchanged. Therefore, we first update the zero angular frequency coefficients by $a_{0,q}^i\leftarrow a_{0,q}^i-\frac{1}{n}{\sum }_{j=1}^n{a}_{0,q}^j$, and then

$$𝒞((\xi ,\theta ),(\xi \prime ,\theta \prime ))=\frac{1}{4\pi n}\sum _{i=1}^n{\int }_0^{2\pi }\sum _{k=-\infty }^{\infty }\sum _{q=1}^{\infty }\sum _{k\prime =-\infty }^{\infty }\sum _{q\prime =1}^{\infty }(a_{k,q}^i{\psi }_c^{k,q}(\xi ,\theta )\overline{a_{k\prime ,q\prime }^i{\psi }_c^{k\prime ,q\prime }(\xi \prime ,\theta \prime )}+a_{-k,q}^i{\psi }_c^{k,q}(\xi ,\theta )\overline{a_{-k\prime ,q\prime }^i{\psi }_c^{k\prime ,q\prime }(\xi \prime ,\theta \prime )})e^{-\iota (k-k\prime )\alpha }d\alpha =\sum _{k=-\infty }^{\infty }\sum _{q=1}^{\infty }\sum _{k\prime =-\infty }^{\infty }\sum _{q\prime =1}^{\infty }{\psi }_c^{k,q}(\xi ,\theta )C_{(k,q),(k\prime ,q\prime )}\overline{{\psi }_c^{k\prime ,q\prime }(\xi \prime ,\theta \prime )},$$ (29)

where

$$C_{(k,q),(k\prime ,q\prime )}=\frac{1}{4\pi n}\sum _{i=1}^n{\int }_0^{2\pi }(a_{k,q}^i\overline{a_{k\prime ,q\prime }^i}+a_{-k,q}^i\overline{a_{-k\prime ,q\prime }^i})e^{-\iota (k-k\prime )\alpha }d\alpha ={\delta }_{k,k\prime }\frac{1}{n}\sum _{i=1}^n\text{Re}\left\{a_{k,q}^i\overline{a_{k\prime ,q\prime }^i}\right\}.$$ (30)

δ_k,k′ comes from the integral over α∈ [0, 2π). The covariance matrix in Eq. (30) is positive semi-definite and block diagonal because the non-zero entries of C correspond only to k = k′. Since the images are well approximated by the subspace spanned by a finite number of Fourier-Bessel basis functions (see Eq. (15)), C_{(k,q),(k′,q′)} are close to zero when (k, q) or (k′, q′) do not satisfy the sampling criterion in Eq. (7). Therefore, we have a finite matrix representation C of 𝒞. Moreover, it suffices to consider k ≥ 0, because C_{(k,q),(k,q′)} = C_{(−k,q),(−k,q′)}. Thus, the covariance matrix in Eq. (30) can be written as the direct sum $C={\oplus }_{k=0}^{k_{\text{max}}}{C}^{(k)}$, where C^(k) is by itself a sample covariance matrix of size p_k × p_k, given by,

$$C_{q,q\prime }^{(k)}=\frac{1}{n}\sum _{i=1}^n\text{Re}\left\{a_{k,q}^i\overline{a_{k,q\prime }^i}\right\}.$$ (31)

Let us denote by A^(k) the matrix of expansion coefficients, obtained by putting the coefficients $a_{k,q}^i$ for all q and all i into a matrix, where the columns are indexed by the image number i and the rows are indexed by the radial index q. The coefficient matrix A^(k) for k ≠ 0 is of size p_k × n and the covariance matrix for k ≠ 0 is,

$$C^{(k)}=\frac{1}{n}\text{Re}\left\{A^{(k)}{(A^{(k)})}^{*}\right\},$$ (32)

where A* is the conjugate transpose $(A_{ij}^{*}={Ā}_{ji})$. The case k = 0 is special because the expansion coefficients satisfy $a_{0,q}=\overline{a_{0,q}}$, and so A⁽⁰⁾ is a matrix of size p₀ × n and

$$C^{(0)}=\frac{1}{n}{A}^{(0)}{(A^{(0)})}^{*}.$$ (33)

We compute the eigenvalues ${\lambda }_1^{(k)}\ge {\lambda }_2^{(k)}\cdots \ge {\lambda }_{p_k}^{(k)}$ and eigenvectors $u_1^{(k)},u_2^{(k)},\dots ,u_{p_k}^{(k)}$ of the covariance matrices C^(k). Because 𝒞 and C are related through Eq. (29) and C is block diagonal as in Eq. (30), 𝒞((ξ, θ), (ξ′, θ′)) is well approximated by ${\sum }_{k=-k_{\text{max}}}^{k_{\text{max}}}{\mathrm{\Psi }}^{(k)}(\xi ,\theta )C^{(k)}{({\mathrm{\Psi }}^{(k)})}^{*}(\xi \prime ,\theta \prime )$, where Ψ^(k) contains Fourier-Bessel functions with angular frequency k. Equation (29) reveals that the eigenfunctions of 𝒞, which are the steerable principal components, can be expressed as linear combinations of the Fourier-Bessel functions with the coefficients given by the eigenvectors of the matrix C,

$$g^{k,l}(\xi ,\theta )=\sum _{q=1}^{p_k}{\psi }_c^{k,q}(\xi ,\theta )u_l^{(k)}(q)=\sum _{q=1}^{p_k}{N}_{k,q}{J}_k\left(R_{k,q}\frac{\xi }{c}\right)u_l^{(k)}(q)e^{\iota k\theta }.$$ (34)

Therefore the radial parts of the steerable principal components

$$f^{k,l}(\xi )=\sum _{q=1}^{p_k}{N}_{k,q}{J}_k\left(R_{k,q}\frac{\xi }{c}\right)u_l^{(k)}(q)$$ (35)

are linear combinations of the Bessel functions within the same angular frequency. The associated expansion coefficients for I_i are

$$c_{k,l}^i=\sum _{q=1}^{p_k}{a}_{k,q}^i{u}_l^{(k)}(q),\hspace{1em}\text{for }i=1,\dots ,n.$$ (36)

The computational complexity for forming the matrix C^(k) is $O(n{p}_k^2)$. The complexity for eigendecomposition of C^(k) is $O(p_k^3)$, since the size of the covariance matrix is p_k × p_k. Using the upper and lower bounds for p_k in Eq. (13) and assuming c = O(1) and R = O(L), we get ${\sum }_k{p}_k^2=O(L^3)$ and ${\sum }_k{p}_k^3=O(L^4)$. Therefore, the complexity for forming the covariance matrix C is $O(n{\sum }_k{p}_k^2)=O(n{L}^3)$ and the complexity of its full eigendecomposition is $O({\sum }_k{p}_k^3)=O(L^4)$. Equations (32) and (33) show that instead of constructing the covariance matrices C^(k) to compute the principal components, we can perform singular value decomposition (SVD) on the coefficient matrix A^(k) directly and take the left singular vectors as the principal components. The computational complexity for full rank SVD on A^(k) is $O(n{p}_k^2)$ and the total complexity of SVD of all coefficient matrices is $O(n{\sum }_k{p}_k^2)=O(n{L}^3)$.

IV. Algorithm and Computational Complexity

The new algorithm introduced in this letter is termed fast Fourier-Bessel steerable PCA (FFBsPCA). The algorithm is composed of two steps. In the first step, Fourier-Bessel expansion coefficients are computed according to Algorithm 1. The input to the algorithm includes an image dataset, the band limit c, and the compact support radius R. The second step (Algorithm 2) takes the Fourier-Bessel expansion coefficients from Algorithm 1 as input and computes the steerable PCA radial functions and the expansion coefficients of the images in the new steerable basis. Algorithm 2 is the same as the corresponding part of the algorithm in [9].

Algorithm 1.

Fast Fourier-Bessel Expansion

Require: n images I1, …, In sampled on a Cartesian grid of     size L × L with compact support radius R and band limit c.
1:	(Precomputation) Select (k, q)’s that satisfy the sampling criterion of Eq. (7). Fix nξ = ⌈4cR⌉ and nθ = ⌈16cR⌉.
2:	(Precomputation) Find nξ Gaussian quadrature points and weights on the interval [0, c] and evaluate $N_{k,q}{J}_k(R_{k,q}\frac{{\xi }_j}{c})$, j = 1, …, nξ, for all selected (k, q)’s.
3:	Compute F(Ii) (Eq. (17)) on a polar grid of size nξ × nθ by NUFFT for each i = 1, …, n.
4:	For each F(Ii), compute $a_{k,q}^i$ using Eqs. (18) and (19).
5:	return $a_{k,q}^i$ for all selected (k, q)’s.

The analysis of the computational complexity of FFBsPCA is as follows. The precomputation that generates all radial basis functions requires O(L³) operations because there are O(L²) basis functions, each of which is sampled over O(L) points. Computing the Fourier-Bessel expansion coefficients $a_{k,q}^i$ in Eq. (19) for all images takes O(nL³) operations (or O(nL² log L) with a fast Bessel transform) as discussed in Section II-B.

The complexity of constructing the covariance matrix C and computing its full eigendecomposition is O(nL³ + L⁴) as described in Section III. Another method for computing the principal components is by SVD of the coefficient matrices.

Algorithm 2.

Steerable PCA

Require: Fourier-Bessel expansion coefficients $a_{k,q}^i$ for n     images and the maximum angular frequency kmax.
1:	Compute the coefficient vector of the mean image $a_{0,q}^{\text{mean}}=\frac{1}{n}{\sum }_j{a}_{0,q}^j$. Then, set $a_{0,q}^i\leftarrow a_{0,q}^i-a_{0,q}^{\text{mean}}$.
2:	for k = 0, 1, …, kmax do
3:	Construct the coefficient matrix A(k).
4:	Compute the covariance matrix C(k), its eigenvalues       ${\lambda }_1^{(k)}\ge {\lambda }_2^{(k)}\cdots \ge {\lambda }_{p_k}^{(k)}$, and eigenvectors,       $u_1^{(k)},\dots ,u_{p_k}^{(k)}$; or perform SVD of A(k) and take the       left singular vectors $u_1^{(k)},\dots ,u_{p_k}^{(k)}$.
5:	Compute the radial eigenvectors fk,l(ξj) for j = 1,       …, nξ using Eq. (35).
6:	Compute the expansion coefficients of the images in       the new steerable basis $c_{k,l}^i$ using Eq. (36).
7:	end for
8:	return For all (k, l), $u_l^{(k)},{\lambda }_l^{(k)}$, fk,l, and $c_{k,l}^i$ i = 1, …, n.

Full rank SVD on all coefficient matrices requires O(nL³) floating point operations (see Section III).

To generate the new steerable basis, we take linear combinations of the Bessel functions as in line 5 of Algorithm 2, which takes O(L⁴) operations. Computing the steerable PCA expansion coefficients $c_{k,l}^i$ for i = 1 …, n (line 6 in Algorithm 2) requires O(nL³) operations by taking linear combinations of the Fourier-Bessel expansion coefficients as in Eq. (36). Therefore the total computational complexity of FFBsPCA is O(nL³ + L⁴).

The complexity of FBsPCA introduced in [9] is O(nL⁴). Thus, FFBsPCA is faster than FBsPCA. For PCA, when the number of images is smaller than the number of pixels in the compact support disk, we form X^TX and compute its eigendecomposition and the complexity is O(n²L² + n³). However, as the number of images grows, the complexity of PCA switches to O(nL⁴ + L⁶) since it becomes more efficient to compute the eigendecomposition of XX^T. Therefore the computational complexity of traditional PCA, without taking into account all rotations and reflections is O(min{n²L² + n³, nL⁴ + L⁶}). When n > O(L), FFBsPCA is more efficient than the traditional PCA.

FFBsPCA is easily adapted for parallel computation. The computation of Fourier-Bessel expansion coefficients in Algorithm 1 can run on multiple workers in parallel, where each worker is allocated with a subset of the images and Fourier-Bessel radial basis functions. In addition, in Algorithm 2, the radial eigenfunctions and the steerable PCA expansion coefficients can also be efficiently computed in parallel for each angular index k.

V. Numerical Experiments

We compare the running times of FFBsPCA, FBsPCA and traditional PCA, where the latter is computed without the images’ in-plane rotations and reflections. The algorithms are implemented in MATLAB on a machine with 60 cores, running at 2.3 GHz, with total RAM of 1.5TB.

We first simulated n = 24, 000 images with different radii of compact support R, while the band limit is fixed at c = 1/2. For small R, since FFBsPCA performs polar Fourier transformation, it appears slightly slower than FBsPCA. However when R increases, FFBsPCA is computationally more efficient (see Tab. I). We next fixed the size of the images while using R = 150 and c = 1/2, and varied the number of images n. Table II shows that the running time of FBsPCA and FFBsPCA grows linearly with n.

TABLE I.

Running Times (in seconds) as a Function of R for n = 2.4 × 10⁴, c = 1/2, and L = 2R

R	PCA	FBsPCA	FFBsPCA
30	8	7	51
60	214	50	87
90	1,636	168	148
120	1,640	413	234
150	1808	757	371
180	1,988	1,437	657
210	2,106	2,274	695
240	2,188	3,827	892

TABLE II.

Running Times (in Minutes) as a Function of n for Image Size 300 × 300 pixels (L = 300), with R = 150 and c = 1/2

n (×103)	PCA	FBsPCA	FFBsPCA
1	0.05	1.2	1.1
2	0.1	2.1	1.3
4	0.3	3.5	1.8
8	1.3	4.3	2.4
16	9.8	8.7	4.6
32	59.1	17.9	8.0
64	424.7	35.7	14.4
128	653.7	74.2	30.6

To show that our new algorithm can handle large datasets efficiently, we simulated a large dataset with 10⁵ images of size 300 × 300 pixels. The images consist entirely of Gaussian noise with mean 0 and variance 1. We assume that the compact support in the image domain is R = 150 and the band limit in Fourier domain is c = 1/2. In Table III, the total running time is divided into three parts: precomputation, Fourier-Bessel expansion (Algorithm 1), and steerable PCA (Algorithm 2). Fourier Bessel expansion took about 24 minutes, during which 91% of the time was spent on mapping images to polar Fourier grid, where we used the software package [22] downloaded from https://www-user.tu-chemnitz.de/potts/nfft/potts/nfft/. Numerical evaluation of the angular integration by 1D FFT and the radial integration by a direct method took 6.4% and 2.6% of the time respectively. Steerable PCA took 42 seconds.

TABLE III.

Timing for FFBsPCA on a Large Dataset With n = 10⁵ Images. Each Image is of Size 300 × 300 Pixels, R = 150 and c = 1/2. We Computed the Full Eigendecomposition in Algorithm 2

Steps	Time (sec)
Precomputation	7
NUFFT and Fourier-Bessel Expansion	1,438
Steerable PCA	42
Total	1487

In our third experiment, we simulated n = 10⁵ clean projection images from a reconstructed volume of a human mitochondrial large ribosomal subunit, downloaded from the electron microscopy data bank [32] (EMDB-2762). The original volume in the data bank is of size 320 × 320 × 320 voxels. We preprocessed the volume such that its center of mass is at the origin and cropped out a volume of size 240 × 240 × 240 voxels that contains the particle. Each projection image is of size 240 × 240 pixels. We simulated both the vanishing behavior of the CTF at low frequencies and the blurring effect due to the Gaussian envelope of the CTF. This was done by convolving the images with the inverse Fourier transform of

$$\text{min}(\pi \lambda z{f}^2+a,1)\text{ exp}(-B{f}^2),$$ (37)

where f is the frequency, λ is the wavelength of the electron beam, z is the defocus, and a is the phase of the CTF introduced by microscope. This stems from the analytic form of the CTF given by sin(πλzf² + a) exp (−Bf²). For the simulations we chose λ = 0.0197Å, z = 2.5 µm, a = 0.1rad, and B = 100Ų. Our clean images (see Fig. 4a) are the projection images filtered by the filter in Eq. (37) and they were then corrupted by additive white Gaussian noise at SNR= 1/30, corresponding to noise variance of σ² = 9 (see Fig. 4b).

Fig. 4.

Simulated projection images of the human mitochondrial large ribosomal subunit. Image size is 240 × 240 pixels.

We estimated the radius of compact support of the particle in real domain and the band limit in Fourier domain from the noisy images in the following way. We first subtracted the mean image of the dataset from each image. Then we computed the 2D variance map of the dataset averaged in the angular direction, to get the mean radial variance (see Fig. 5a). At large r, the mean radial variance levels off at 9, which corresponds to the noise variance. We subtracted the noise variance from the estimated mean radial variance and computed the cumulative variance by integrating the mean radial variance over r with a Jacobian weight rdr. The fraction of the cumulative variance reaches 99.9% at r = 98, and therefore R was chosen to be 98. In the Fourier domain, we computed the angular average of the mean 2D power spectrum. The curve in Fig. 5b also levels off at the noise variance when ξ is large. We used the same method as before to compute the cumulative radial power spectrum. The fraction reaches 99.9% at ξ = 0.196, therefore the band limit is chosen to be c = 0.196.

Fig. 5.

Estimating R and c from n = 10⁵ simulated noisy projection images of a human mitochondrial large ribosomal subunit. Each image is of size 240 × 240 pixels. (a) Mean radial variance of the images. The curve levels off at about σ² = 9 when r ≥ 98. The radius of compact support is chosen as R = 98. (b) Mean radial power spectrum. The curve levels off at σ² = 9 when ξ ≥ 0.196. The band limit is chosen as c = 0.196.

The radial functions of the top nine principal components are shown in Fig. 6. Each radial function is indexed by k and l, where k determines the angular Fourier mode and l is the order of the radial function within the same k. Taking the tensor product of the radial functions and their corresponding angular Fourier modes gives the two dimensional principal components in Fourier domain. It took about 9 minutes in total to get the steerable PCA radial components and the associated expansion coefficients. In particular, Fourier-Bessel expansion coefficients were computed in 9 minutes and the steerable PCA took 12 seconds.

Fig. 6.

FFBsPCA principal radial functions in Fourier domain. The dataset contains n = 10⁵ simulated human mitochondrial large ribosomal subunit projection images corrupted by additive white Gaussian noise with SNR= 1/30. Image size is 240 × 240 pixels, R = 98, c = 0.196. Each radial function is labeled with angular index k, radial order l, and eigenvalue λ.

We computed the traditional PCA and FBsPCA on the same dataset in real image domain (see Fig. 8 for PCA components), which took 60 minutes and 16 minutes respectively. In order to compare the principal components computed by FFBsPCA with those computed by traditional PCA, we take the inverse Fourier transform of the FFBsPCA components. We do not compute the inverse polar Fourier transform directly, since such a transform is ill-conditioned. Instead, since the FFBsPCA components are linear combinations of the Fourier-Bessel functions as in Eq. (34), we evaluate the steerable principal components on a Cartesian grid in real space using the linear combinations of ${ℱ}^{-1}({\psi }_c^{k,q})$, given by Eq. (5). Those principal components are shown in Fig. 7. Some of the top sixteen principal components computed from traditional PCA and FFBsPCA look very similar, for example, the first three and the last four principal components (see Fig. 7 and Fig. 8). Because the gap between the eigenvalues of the traditional PCA is very small for the components in the middle two rows of Fig. 8, those components become degenerate and therefore look different from the corresponding components in Fig. 7.

Fig. 8.

Traditional PCA principal components in real image domain for the same dataset used in Figures 6 and 7.

Fig. 7.

FFBsPCA principal components (eigenimages in real domain) corresponding to Figure 6.

In our simulation, each noisy projection image I is obtained by contaminating the clean image I_c with additive white Gaussian noise of variance σ² = 9. Given the noise level, we would like to automatically select the appropriate principal components to compress and denoise the noisy images. Since the transformation T* is nearly unitary, the coefficient matrices can be modeled approximately as $A^{(k)}=A_c^{(k)}+{\epsilon }^{(k)}$, where ε^(k) is white Gaussian noise with variance σ² and $A_c^{(k)}$ is the coefficient matrix for the clean images. In the case when there is no signal, that is $A_c^{(k)}=0$, all eigenvalues of the covariance matrix C^(k) from Eqs. (32) and (33) converge to σ² as n goes to infinity, while p_k is fixed. When $A_c^{(k)}\ne 0$, components with eigenvalues larger than σ² correspond to the underlying clean signal. In the non-asymptotic regime of a finite number of images, the eigenvalues of the sample covariance matrix from white Gaussian noise spread around σ². The empirical density of the eigenvalues can be approximated by the Marčenko-Pastur distribution with parameter γ_k, where ${\gamma }_0=\frac{p_0}{n}$ and ${\gamma }_k=\frac{p_k}{2n}$ for k > 0 and the eigenvalues of C^(k) are supported on $[{\lambda }_{-}^{(k)},{\lambda }_{+}^{(k)}]$, with ${\lambda }_{\pm }^{(k)}={\sigma }^2{(1\pm \sqrt{{\gamma }_k})}^2$. The principal components corresponding to eigenvalues larger than ${\lambda }_{+}^{(k)}$ correspond to signal information beyond noise level. Therefore, with the estimated noise variance σ̂², we denote by ${\lambda }_1^{(k)}\ge {\lambda }_2^{(k)}\ge \cdots \ge {\lambda }_{p_k}^{(k)}$ the eigenvalues of the covariance matrix C^(k), and select the components with eigenvalues

$${\lambda }_l^{(k)}>{\hat{\sigma }}^2{(1+\sqrt{{\gamma }_k})}^2,l=1,\dots ,p_k.$$ (38)

Various ways of selecting principal components from noisy data have been proposed. We refer to [33] for an automatic procedure for estimating the noise variance and the number of components beyond the noise level. For the simulated ribosomal subunit projections images, there are 966 steerable principal radial components above the threshold in Eq. (38), whereas considerably fewer principal components (391) with the traditional PCA were selected.

Moreover, we filter the expansion coefficients to get better denoising. To first order approximation, when n ≫ p_k, the noise simply shifts all eigenvalues upward by σ² and this calls for soft thresholding of the sample covariance eigenvalues: (λ − σ²)₊. To correct for the finite sample effect, we can apply more sophisticated shrinkage to the eigenvalues, such as the methods proposed in [34], [35]. Specifically, we applied the shrinkage method in [34] to the coefficients computed by FFBsPCA, FBsPCA, and PCA. Because we were able to use more principal components with FFBsPCA, we recovered finer details of the clean projection images, comparing Fig. 10c and Fig. 10f.

Fig. 10.

Enlarged view of 100 × 100 pixels box at the center of the images in Figure 9.

In addition to using data-adaptive bases, we also used a non-isotropic directional multiscale transform, i.e., Curvelet transform [36] with complex block thresholding and cycle spinning, to denoise the images. An example of a denoised image using PCA, Curvelet, FBsPCA, and FFBsPCA is shown in Fig. 9. The steerable PCA basis captures the variance of the clean dataset with fewer components than non-adaptive bases, such as Fourier-Bessel basis or Curvelets (see Fig. 11).

Fig. 9.

Denoising simulated projection images. (a) clean clean projection image, (b) noisy noisy projection image with SNR= 1/30, (c) denoised projection image using traditional PCA, (d) denoised projection image using Curvelet transform, complex block thresholding and cycle spinning, (e) denoised image using FBsPCA, and (f) denoised image using FFBsPCA.

Fig. 11.

Cumulative variance of FFBsPCA, Fourier-Bessel and Curvelet expansion coefficients of simulated clean ribosome projection images as in Fig. 4a.

We computed the mean squared error (MSE) and Peak SNR (PSNR) to quantify the denoising effects in Tab. IV and Tab. V. Comparing with the traditional PCA, FFBsPCA reduced the MSE by more than 25% and increased the PSNR by over 1.3 dB. When the images are of low SNR, Curvelets are unable to outperform data adaptive bases, such as PCA, FBsPCA and FFBsPCA (see Tab. IV and Tab. V). This experiment shows that FFBsPCA is an efficient and effective procedure for denoising large image datasets.

TABLE IV.

MSE of Denoised Images Using PCA, Curvelets, FBsPCA and FFBsPCA, all Computed Using Pixels Within R = 98

	MSE (10−5)
	Curvelet	PCA	FBsPCA	FFBsPCA
Image 1	1.38	1.10	0.77	0.77
Image 2	1.63	1.29	0.95	0.96
Image 3	1.58	1.17	0.85	0.85

TABLE V.

PSNR of Denoised Images Using PCA, Curvelets, FBSPCA and FFBSPCA, All Computed Using Pixels Within R = 98

	PSNR (dB)
	Curvelet	PCA	FBsPCA	FFBsPCA
Image 1	18.10	19.06	20.62	20.63
Image 2	17.90	18.93	20.26	20.23
Image 3	18.68	19.99	21.35	21.35

Finally, we show that steerable PCA denoising is robust to small shifts. We simulated clean data with random shifts in the ±x and ±y directions with maximum shifts equal to 0 (centered images), 5, 10, 15, and 20 pixels. The clean images are corrupted with additive white Gaussian noise of variance 9. As shown in Tab. VI, the denoising performance using FFBsPCA (measured in PSNR) is almost unaffected. The denoising results for centered images in Tab. V and Tab. VI are slightly different because we used different support sizes to evaluate PSNRs.

TABLE VI.

FFBsPCA Denoising of Images With Maximum Shifts of 0, 5, 10, 15, and 20 pixels. PSNRs Are Computed With Pixels Within R = 110. The Estimated Compact Support R Increases With Maximum Shift

		PSNR (dB)
max shifts (pixels)	R (pixels)	Image 1	Image 2	Image 3
0	98	21.61	21.16	22.34
5	99	21.53	21.26	22.32
10	102	21.42	21.13	22.11
15	107	21.59	21.31	22.18
20	110	21.31	21.30	22.07

VI. Conclusion

In this letter we presented a fast Fourier-Bessel steerable PCA method that reduces the computational complexity with respect to the size of the images so that it can handle larger images. The complexity of the new algorithm is O(nL³ + L⁴) compared with O(nL⁴) of the steerable PCA introduced in [9]. The key improvement is through mapping the images to a polar Fourier grid using NUFFT and evaluating the Fourier-Bessel expansion coefficients by angular 1D FFT and accurate radial integration.

This work has been mostly motivated by its application to cryo-EM single particle reconstruction. Besides compression and denoising of the experimental images required for 2D class averaging [11] and common-lines based 3D ab-initio modeling, FFBsPCA can also be applied in conjunction with Kam’s approach [37] that requires the covariance matrix of the 2D images [38]. The method developed here can also be extended to perform fast principal component analysis of a set of 3D volumes and their rotations. For this purpose, the Fourier-Bessel basis is replaced with the spherical-Bessel basis, and the expansion coefficients can be evaluated by performing the angular integration using a fast spherical harmonics transform [39] followed by radial integration.

Our numerical experiments show that an adaptive basis is necessary for denoising images with very low SNR. Steerable PCA is able to recover more signal components than PCA and achieves better denoising results. It is definitely possible to improve the denoising obtained by just using steerable PCA. For example, we can have more sophisticated dictionary denoising schemes, in which part of the dictionary is made of the steerable principal components and another part of the dictionary is made of wavelets. As these methods require the computation of steerable PCA, computing steerable PCA fast would be useful also for more advanced denoising schemes.

Finally, we remark that the Fourier-Bessel basis can be replaced in our framework with other suitable bases, for example, the 2D prolate spheroidal wave functions (PSWF) on a disk [40]. The 2D prolates also have a separation of variables form which makes them convenient for steerable PCA. A possible advantage of using 2D prolates is that they are optimal in terms of the size of their support.

Biographies

Zhizhen Zhao received the B.A. and M.Sc. degrees in physics from Trinity College, Cambridge University, Cambridge, U.K., the Ph.D. degree in physics from Princeton University, Princeton, NJ, USA, in 2008 and 2013, respectively. Since September 2013, she has been with the Courant Institute of Mathematical Sciences, New York University, New York, NY, USA. She is currently a Courant Instructor with NYU. Her research interests include applied and computational harmonic analysis, signal processing, data analysis and the applications in structural biology and atmospheric, and oceanic sciences.

Yoel Shkolnisky received the B.Sc. degree in mathematics and computer science, and the M.Sc. and Ph.D. degrees in computer science from Tel-Aviv University, Tel-Aviv, Israel, in 1996, 2001, and 2005, respectively. From July 2005 to July 2008, he was a Gibbs Assistant Professor in applied mathematics with the Department of Mathematics, Yale University, New Haven, CT, USA. Since October 2009, he has been with the Department of Applied Mathematics, School of Mathematical Sciences, Tel-Aviv University. His research interests include computational harmonic analysis, scientific computing, and data analysis.

Amit Singer received the B.Sc. degree in physics and mathematics and the Ph.D. degree in applied mathematics from Tel-Aviv University, Tel-Aviv, Israel, in 1997 and 2005, respectively. He is a Professor of Mathematics and a Member of the Executive Committee of the Program in Applied and Computational Mathematics (PACM), Princeton University, Princeton, NJ, USA. He joined Princeton University as an Assistant Professor in 2008. From 2005 to 2008, he was a Gibbs Assistant Professor in Applied Mathematics with the Department of Mathematics, Yale University, New Haven, CT, USA. His research interests include applied mathematics focuses on theoretical and computational aspects of data science, and on developing computational methods for structural biology. He served in the Israeli Defense Forces from 1997 to 2003. He was the recipient of the Moore Investigator in Data-Driven Discovery (2014), the Simons Investigator Award (2012), the Presidential Early Career Award for Scientists and Engineers (2010), the Alfred P. Sloan Research Fellowship (2010), and the Haim Nessyahu Prize for Best PhD in Mathematics in Israel (2007).