Fourier–Bessel rotational invariant eigenimages

Abstract

We present an efficient and accurate algorithm for principal component analysis (PCA) of a large set of two-dimensional images and, for each image, the set of its uniform rotations in the plane and its reflection. The algorithm starts by expanding each image, originally given on a Cartesian grid, in the Fourier–Bessel basis for the disk. Because the images are essentially band limited in the Fourier domain, we use a sampling criterion to truncate the Fourier–Bessel expansion such that the maximum amount of information is preserved without the effect of aliasing. The constructed covariance matrix is invariant to rotation and reflection and has a special block diagonal structure. PCA is efficiently done for each block separately. This Fourier–Bessel-based PCA detects more meaningful eigenimages and has improved denoising capability compared to traditional PCA for a finite number of noisy images.

1. INTRODUCTION

Principal component analysis (PCA) is a classical method for dimensionality reduction, compression, and denoising. The principal components are the eigenvectors of the sample covariance matrix. In image analysis, the principal components are often referred to as “eigenimages” and they form a basis adaptive to the image set. In some applications, including all planar rotations of the input images, PCA is advantageous. For example, in single particle reconstruction (SPR) using cryoelectron microscopy (cryo-EM) [1], the 3D structure of a molecule needs to be determined from many noisy 2D projection images taken at unknown viewing directions. PCA, known in this field as multivariate statistical analysis, is often a first step in SPR [2]. Inclusion of the rotated images for PCA is desirable because such images are just as likely to be obtained in the experiment by in-plane rotating either the specimen or the detector. When all rotated images are included for PCA, then the eigenimages have a special separation of variables form in polar coordinates in terms of radial functions and angular Fourier modes [3–5]. It is easy to steer the eigenimages by a simple phase shift, hence the name “steerable PCA.” Computing the steerable PCA efficiently and accurately is, however, challenging.

The first challenge is to mitigate the increased computational cost associated with replicating each image multiple times. Efficient algorithms for steerable PCA were introduced in [6,7] with computational complexity almost similar to that of traditional PCA on the original images without their rotations. Current efficient algorithms for steerable PCA first map the images from a Cartesian grid to a polar grid using, e.g., interpolation. Because the transformation from Cartesian to polar is not unitary, the eigenimages corresponding to images mapped to a polar grid are not equivalent to transforming the original eigenimages from Cartesian to polar.

The second challenge is associated with noise. The nonunitary transformation from Cartesian to polar changes the noise statistics. Interpolation transforms additive white noise to correlated (i.e., colored) noise. As a consequence, spurious eigenimages and eigenvalues corresponding to colored noise may arise in the eigen-analysis. The naïve algorithm for steerable PCA that replicates each image multiple times also mistreats the noise. While the realization of noise is independent between the original images, the realization of noise among duplicated images is dependent. This in turn can lead to noise-induced spurious eigenimages and to artifacts in the leading eigenimages.

We present a new efficient and accurate algorithm for computing the steerable PCA that meets these challenges. This is achieved by combining into the steerable PCA framework a sampling criterion similar to the criterion proposed by Klug and Crowther [8] in a different context of reconstructing a 2D image from its 1D projections. We represent the images in a truncated Fourier–Bessel basis in which the number of radial components is decreasing with the angular frequency. Our sampling criterion implies that the covariance matrix of the images has a block diagonal structure where the block size decreases as a function of the angular frequency. The block diagonal structure of the covariance matrix was observed and utilized in previous works on steerable PCA. However, while in all existing methods for steerable PCA the block size is constant, here the block size shrinks with the angular frequency. The incorporation of the sampling criterion into the steerable PCA framework is the main contribution of this paper.

2. SAMPLING CRITERION

We assume that the set of images correspond to spatially limited objects. By appropriate scaling of the pixel size, we can assume that the images vanish outside a disk of radius 1. The eigenfunctions of the Laplacian in the unit disk with vanishing Dirichlet boundary condition are the Fourier–Bessel functions. Hence, they form an orthogonal basis to the space of squared-integrable functions over the unit disk, and it is natural to expand the images in that basis. The Fourier–Bessel functions are given by

$${\psi }^{kq}(r,\theta )=\{\begin{array}{cc}{N}_{kq}{J}_k(R_{kq}r)e^{ik\theta },& r\le 1\\ 0& r>1,\end{array}$$ (1)

where N_kq is a normalization factor; J_k is the Bessel function of integer order k, and R_kq is the qth root of the equation

$$J_k(R_{kq})=0.$$ (2)

The functions ψ^kq are normalized to unity, that is

$${\int }_0^{2\pi }{\int }_0^1{\psi }^{kq}{({\psi }^{kq})}^{\ast }r\text{d}r\text{d}\theta =1.$$ (3)

The normalization factors are given by

$$N_{k,q}=\frac{1}{\sqrt{\pi }\mid J_{k+1}(R_{kq})\mid }.$$ (4)

We use here the following convention for the 2D Fourier transform of a function f in polar coordinates

$$\mathcal{F}(f)(k_0,{\phi }_0)={\int }_0^{2\pi }{\int }_0^{\infty }f(r,\theta )e^{-2\pi i{k}_{0}rcos(\theta -{\phi }_0)}r\text{d}r\text{d}\theta .$$ (5)

The 2D Fourier transform of the Fourier–Bessel functions denoted (ψ^kq), is given in polar coordinates as

$$\mathcal{F}({\psi }^{kq})(k_0,{\phi }_0)=2\sqrt{\pi }{(-1)}^q{(-i)}^k{R}_{kq}\frac{J_k(2\pi k_0)}{{(2\pi k_0)}^2-R_{kq}^2}{e}^{ik{\phi }_0}.$$ (6)

This result is typically derived using the Jacobi–Anger identity

$$e^{\mathit{iz}cos\theta }=\sum _{n=-\infty }^{\infty }{i}^n{J}_n(z)e^{in\theta }.$$ (7)

Notice that the Fourier transform (ψ^kq)(k₀, φ₀) vanishes on concentric rings of radii k₀ = (R_kq′/2π) with q′ ≠ q. The maximum of | (ψ^kq)(k₀, φ₀)| is obtained near the ring k₀ = (R_kq/2π), as can be verified from the asymptotic behavior of the Bessel functions [cf. Eq. (9)]. For images that are sampled on a squared Cartesian grid of size 2L × 2L pixels, with grid size 1/L (corresponding to the square [−1, 1] × [−1, 1]) the sampling rate is L and the corresponding Nyquist frequency (the band limit) is (L/2). Because of the Nyquist criterion, the Fourier–Bessel expansion requires components for which

$$\frac{R_{kq}}{2\pi }\le \frac{L}{2},$$ (8)

because other components represent features beyond the resolution and their inclusion would result in aliasing. The asymptotic behavior of the Bessel functions

$$J_k(R_{kq}r)~\sqrt{\frac{2}{\pi R_{kq}r}}cos\left(R_{kq}r-\frac{k\pi }{2}-\frac{\pi }{4}\right)$$ (9)

for R_kqr ≫ |k² − (1/4)|, suggests that the roots are asymptotically

$$R_{kq}~\frac{\pi }{2}\left(k+2q-\frac{1}{2}\right),$$ (10)

which is indeed the first term of the asymptotic expansion for the roots (see [9]). Equations (8) and (10) lead to the sampling criterion

$$k+2q\le 2L+\frac{1}{2}.$$ (11)

In practice, we do not rely on the asymptotic formula, Eq. (10). Instead, we find the roots of the Bessel functions numerically, and check directly which components satisfy the criterion R_kq ≤ πL. The number of components satisfying |k| + 2q ≤ 2L + (1/2) is approximately 2L², which is smaller than πL² (the number of pixels inside the unit disk) by a factor of 2/π. We remark that the cutoff criterion, Eq. (11), is similar to the criterion in [8] but is not identical to it. Specifically, the criterion in [8] has a different cutoff value for k + 2q. The difference stems from the fact that [8] studies a different problem, namely the 2D reconstruction problem of an image from its 1D line projections.

3. FOURIER–BESSEL EXPANSION OF IMAGES SAMPLED ON A CARTESIAN GRID

Suppose I₁, …, I_n are n images sampled on a Cartesian grid. We denote by Ĩ_i the continuous approximation of the ith image in terms of a truncated Fourier–Bessel expansion including only components satisfying the sampling criterion, namely

$${\stackrel{\sim }{I}}_i(r,\theta )=\sum _{k=-2L}^{2L}\sum _{q=1}^{p_k}{a}_{k,q}^i{\psi }^{kq}(r,\theta ),$$ (12)

where for each k, p_k denotes the number of components satisfying R_kq ≤ πL. We choose the expansion coefficients $a_{k,q}^i$ that minimize the squared distance between I_i and Ĩ_i, with the latter restricted to the grid points; that is, $a_{k,q}^i$ are the least squares solution to the overdetermined linear system obtained from Eq. (12) by evaluating the images and the Fourier–Bessel functions at the grid points. More specifically, let Ψ be the matrix whose entries are evaluations of the Fourier–Bessel functions at the grid points, with rows indexed by the grid points and columns indexed by angular and radial frequencies. Then, the coefficient vector aⁱ of the ith image is the solution of min_ai ||Ψaⁱ − I_i||², given by

$$a^i={({\mathrm{\Psi }}^{†}\mathrm{\Psi })}^{-1}{\mathrm{\Psi }}^{†}{I}_i.$$ (13)

The orthogonality of the Fourier–Bessel functions on the disk does not necessarily imply that their discretized counterparts are orthogonal. That is, the matrix Ψ^†Ψ may differ from the identity matrix. The columns of the matrix Ψ are guaranteed to approach orthogonality only as the grid is indefinitely refined (e.g., as L → ∞). In practice, we have numerically observed that the columns of Ψ are approximately orthogonal already for moderate values of L. In particular, white noise remains approximately white after the transformation, since the eigenvalues of (Ψ^†Ψ)⁻¹ are close to 1 (see Fig. 1). Indeed, for white noise images, the mean and covariance satisfy [I_i] = 0 and $\mathbb{E}[I_i{I}_i^{†}]={\sigma }^{2}I$, where σ² is the noise variance and I is the identity matrix. Therefore, Eq. (13) and the linearity of expectation yield [aⁱ] = 0 and [aⁱa^i†] = σ²(Ψ^†Ψ)⁻¹.

Fig. 1.

Eigenvalues of (Ψ^†Ψ)⁻¹, where Ψ is the truncated Fourier–Bessel transform for L = 6 and L = 60 pixels respectively. This is also the spectrum of transformed white noise images (see text). The eigenvalues are close to 1, indicating that the discretized Fourier–Bessel transform is approximately unitary and that white noise remains approximately white. For L = 60, among the eigenvalues, only 18 are above 1.05 and 34 are below 0.95. For L = 6, among the eigenvalues, there are 6 below 0.95.

In the previous work [7] on fast computation of steerable principal components for a large data set, images are re-sampled on a polar grid by the polar Fourier transform. The polar Fourier transform is nonunitary and therefore causes artifacts in the eigenimages. Suppose the linear transformation is described by the matrix A. The sample covariance matrix C_A of the transformed images is related to the sample covariance matrix C of the original images by

$$C_A={\mathit{ACA}}^{†}.$$ (14)

Unless A is unitary, there is no simple way to transform the eigenvectors of ACA^† to the eigenvectors of C. The advantage of using the Fourier–Bessel transform with the sampling criterion that adapts to the band limit of the images is that such transform is approximately unitary (Fig. 1).

4. SAMPLE COVARIANCE MATRIX

It is easy to “steer” the continuous approximation Ĩ_i of I_i [Eq. (12)]. If ${\stackrel{\sim }{I}}_i^{\alpha }$ denotes the rotation of the i′ th image by an angle α, then

$${\stackrel{\sim }{I}}_i^{\alpha }(r,\theta )={\stackrel{\sim }{I}}_i(r,\theta -\alpha )=\sum _{k,q}{a}_{k,q}^i{e}^{-ik\alpha }{\psi }^{kq}(r,\theta ).$$ (15)

Because J_−k(x) = (−1)^kJ_k(x), for real valued images $a_{-k,q}={(-1)}^k{a}_{k,q}^{\ast }$. Also, under reflection $a_{k,q}^i$ changes to $a_{-k,q}^i$, and the reflected image, denoted ${\stackrel{\sim }{I}}_i^r$, is given by

$${\stackrel{\sim }{I}}_i^r(r,\theta )={\stackrel{\sim }{I}}_i(r,\pi -\theta )=\sum _{k,q}{a}_{k,q}^i{\psi }^{kq}(r,\pi -\theta )$$ (16)

$$=\sum _{k,q}{a}_{-k,q}^i{\psi }^{kq}(r,\theta ).$$ (17)

The sample mean, denoted Ĩ_mean, is the continuous image obtained by averaging the continuous images and all their possible rotations and reflections:

$${\stackrel{\sim }{I}}_{\text{mean}}(r,\theta )=\frac{1}{2n}\sum _{i=1}^n\frac{1}{2\pi }{\int }_0^{2\pi }[{\stackrel{\sim }{I}}_i^{\alpha }(r,\theta )+{\stackrel{\sim }{I}}_i^{\alpha ,r}(r,\theta )]\text{d}\alpha .$$ (18)

Substituting Eqs. (15) and (17) into (18) we obtain

$${\stackrel{\sim }{I}}_{\text{mean}}(r,\theta )=\sum _{q=1}^{p_0}\left(\frac{1}{n}\sum _{i=1}^n{a}_{0,q}^i\right){\psi }^{0q}(r,\theta ).$$ (19)

As expected, the sample mean image is radially symmetric, because ψ^0q is only a function of r but not of θ.

The sample covariance matrix built from the continuous images and all possible rotations and reflections is formally

$$C=\frac{1}{2n}\sum _{i=1}^n\frac{1}{2\pi }{\int }_0^{2\pi }[({\stackrel{\sim }{I}}_i^{\alpha }-{\stackrel{\sim }{I}}_{\text{mean}}){({\stackrel{\sim }{I}}_i^{\alpha }-{\stackrel{\sim }{I}}_{\text{mean}})}^{†}+({\stackrel{\sim }{I}}_i^{\alpha ,r}-{\stackrel{\sim }{I}}_{\text{mean}}){({\stackrel{\sim }{I}}_i^{\alpha ,r}-{\stackrel{\sim }{I}}_{\text{mean}})}^{†}]\text{d}\alpha .$$ (20)

When written in terms of the Fourier–Bessel basis, the covariance matrix C can be directly computed from the expansion coefficients $a_{k,q}^i$. Subtracting the mean image is equivalent to subtracting the coefficients $a_{0,q}^i$ with $(1/n){\sum }_{i=1}^n{a}_{0,q}^i$, while keeping other coefficients unchanged. That is, we update

$$a_{0,q}^i\leftarrow a_{0,q}^i-\frac{1}{n}\sum _{i=1}^n{a}_{0,q}^i.$$ (21)

In the Fourier–Bessel basis the covariance matrix C is given by

$$C_{(k,q),(k^{\prime },q^{\prime })}$$ (22)

$$\begin{array}{l}=\frac{1}{4\pi n}\sum _{i=1}^n{\int }_0^{2\pi }[a_{k,q}^i{(a_{k^{\prime },q^{\prime }}^i)}^{\ast }+a_{-k,q}^i{(a_{-k^{\prime },q^{\prime }}^i)}^{\ast }]e^{-i(k-k^{\prime })\alpha }d\alpha \\ ={\delta }_{k,k^{\prime }}\frac{1}{n}\sum _{i=1}^n\mathfrak{R}\{a_{k,q}^i{(a_{k,q^{\prime }}^i)}^{\ast }\}.\end{array}$$ (23)

The nonzero entries of C correspond to k = k′, rendering its block diagonal structure (see Fig. 2). Also, it suffices to consider k ≥ 0, because C_{(k,q),(k,q′)} = C_{(−k,q),(−k,q′)}. Thus, the covariance matrix can be written as the direct sum

Fig. 2.

Illustration of the block diagonal structure of the rotational invariant covariance matrix. The block size shrinks as the angular frequency k increases.

$$C=\underset{k=0}{\overset{2L}{\oplus }}{C}^{(k)},$$ (24)

where $C_{q,q^{\prime }}^{(k)}=(1/n){\sum }_{i=1}^n\mathfrak{R}\{a_{k,q}^i{(a_{k,q^{\prime }}^i)}^{\ast }\}$ is by itself a sample covariance matrix of size p_k × p_k. The block size p_k decreases as the angular frequency k increases (see Fig. 2). We remark that the block structure of the covariance matrix in steerable PCA is well known; however, in previous works the block size is constant (i.e., independent of the angular frequency k). Our main observation is that the block size must reduce as the angular frequency increases in order to avoid aliasing. Moreover, if the images are corrupted by independent additive white (uncorrelated) noise, then each block C^(k) is also affected by independent additive white noise because the Fourier–Bessel transform is unitary (up to grid discretization).

5. ALGORITHM AND COMPUTATIONAL COMPLEXITY

We refer to the resulting algorithm as Fourier–Bessel steerable PCA (FBsPCA). The steps of FBsPCA are summarized in Algorithm 1. The computational complexity of FBsPCA (excluding pre-computation) is O(nL⁴ + L⁵), whereas the computational complexity of the traditional PCA (applied on the original images without their rotational copies) is O(nL⁴ + L⁶). The different steps of FBsPCA have the following computational cost. The cost for precomputing the Fourier–Bessel functions on the discrete Cartesian grid is O(L⁴), because the number of basis functions satisfying the sampling criterion is O(L²) and the number of grid points is also O(L²). The complexity of computing the pseudoinverse of Ψ^†Ψ is dominated by computing the singular value decomposition of Ψ in O(L⁶). Computing the pseudoinverse of Ψ^†Ψ is a precomputation that does not depend on the images. Alternatively, since the eigenvalues of Ψ^†Ψ are almost equal to one it can be approximated by the identity matrix. It takes O(nL⁴) to compute the expansion coefficients $a_{k,q}^i$. The computational complexity of constructing the block diagonal covariance matrix is O(nL³). Since the covariance matrix has a special block diagonal structure, its eigen decomposition takes O(L⁴). Constructing the steerable basis takes O(L⁵). Therefore, the total computational complexity for FBsPCA without the precomputation is O(nL⁴ + L⁵). For traditional PCA, computing the covariance matrix takes O(nL⁴) and its eigen decomposition takes O(L⁶). Overall, the computational cost of FBsPCA is lower than that of the traditional rotational variant PCA.

Algorithm 1.

Fourier–Bessel Steerable PCA (FBsPCA)

Require: n pre-whitened images I1, …, In sampled on a Cartesian grid of size 2L × 2L. (Precomputation) Compute ψkq for all Rkq ≤ πL. (Precomputation) Compute the Moore–Penrose pseudoinverse of Ψ. For each Ii compute $a_{k,q}^i$ using Eq. (13). Estimate the Ĩmean using Eq. (19). Subtract the mean image by changing $a_{0,q}^i$ to $a_{0,q}^i-(1/n){\sum }_i{a}_{0,q}^i$. For each k = 0, 1, …, 2L compute the covariance matrix C(k) of size pk × pk. For each k compute the eigenvalues of C(k), in descending order ${\lambda }_k^1\ge {\lambda }_k^2\cdots \ge {\lambda }_k^{p_k}$ and the associated eigenvectors $h_k^1,h_k^2,\dots ,h_k^{p_k}$. The eigenimages Vkl (0 ≤ k ≤ 2L, 1 ≤ l ≤ pk) are linear combinations of the Fourier–Bessel functions, i.e., $V^{kl}={\mathrm{\Psi }}^{(k)}{h}_k^l$, where Ψ(k) = [ψk1, …, ψkpk]. (optional) Use an algorithm such as [10] to estimate the number of components to choose for each C(k).

6. RESULTS

In the first experiment, we simulated n = 10⁴ clean projection images of E. coli 70S ribosome. The images are of size 129 × 129 pixels, but the molecule is confined to a disk of radius L = 55 pixels. We corrupted the clean images with additive white Gaussian noise at different levels of signal-to-noise ratio (SNR) (Fig. 3). The running time of traditional PCA was 695 s, compared to 85 s (including precomputation) for FBsPCA, both implemented in MATLAB on a machine with 2 Intel(R) Xeon(R) CPUs X5570, each with 4 cores, running at 2.93 GHz. The top 5 eigenimages for noisy images agree with the eigenimages from clean projection images [Figs. 4(a) and 4(d)]. The eigenimages generated by FBsPCA are much cleaner than the eigenimages from the traditional PCA [Figs. 4(d) and 4(e)]. We also see that the eigenimages generated by steerable PCA with polar transform (PTsPCA) are consistent with the eigenvectors from the traditional PCA [see Figs. 4(c) and 4(f)].

Fig. 3.

Simulated 70S ribosome projection images with different SNR.

Fig. 4.

Eigenimages for 10⁴ simulated 70S ribosome projection images. Clean images: (a) FBsPCA, (b) traditional PCA, and (c) PTsPCA. Noisy images with SNR = (1/50): (d) FBsPCA, (e) traditional PCA, and (f) PTsPCA. Image size is 129 × 129 pixels and L = 55 pixels.

In another experiment, we simulated images consisting entirely of white Gaussian noise with mean 0 and variance 1 and computed the Fourier–Bessel expansion coefficients. The distribution of the eigenvalues for the rotational invariant covariance matrix with different angular frequencies k are well predicted by the Marčenko–Pastur distribution (see Fig. 5). This property allows us to use the method described in [10] to estimate the noise variance and the number of principal components to choose.

Fig. 5.

Histogram of eigenvalues of C^(k) (with k = 0, 1, 20) for images consisting of white Gaussian noise with mean 0 and variance 1. The dashed lines correspond to the Marčenko–Pastur distribution. n = 10⁴, L = 55 pixels and σ² = 1.

We compared the eigenvalues in the previous experiment of the ribosome projection images with different SNRs (Fig. 6). As the noise variance (σ²) increases, the number of signal components that we are able to discriminate decreases. We use the method in [10] to automatically estimate the noise variance and the number of components beyond the noise level. With the estimated noise variance σ̂², components with eigenvalues ${\lambda }_k^i>{\widehat{\sigma }}^2{(1+\sqrt{1+{\gamma }_k^2})}^2$, where γ₀ = (p₀/n) and γ_k = (p_k/2n) for k > 0, are chosen (the case k = 0 is special because the expansion coefficients a_0,q are purely real, since $a_{k,q}={(-1)}^k{a}_{-k,q}^{\ast }$.

Fig. 6.

Eigenvalues for C⁽⁰⁾ and C⁽¹⁰⁾ for simulated projection images with different SNRs.

We compared the effects of denoising with FBsPCA, PTsPCA, and traditional PCA. The asymptotically optimal Wiener filters [11] designed with the eigenvalues and eigenimages from FBsPCA (PTsPCA, PCA, resp.) are applied to the noisy images. The number of signal components determined with PCA is 62, whereas the total number of eigenimages for FBsPCA is 430 for 10⁴ noisy images with SNR = (1/20) and L = 55 pixels. For PTsPCA, due to the nonunitary nature of the interpolation from Catersian to polar, we do not have a simple rule for choosing the number of eigenimages. Instead, we applied denoising multiple times corresponding to different number of eigenimages, and present here the denoising result with the smallest mean squared error (MSE). The optimal number of eigenimages for PTsPCA was 348. Of course, in practice the clean image is not available and such a procedure cannot be used. Still, the optimal denoising result by PTsPCA is inferior to that obtained by FBsPCA, where the number of components is chosen automatically. Even if we allow PTsPCA to use more components than FBsPCA, the denoising result does not improve. Figure 7 shows that FBsPCA gives the best denoised image. We computed the MSE, peak SNR (PSNR), and the mean structural similarity (MSSIM ) index [12] to show the effectiveness of the denoising effect using FBsPCA compared with traditional rotational variant PCA and PTsPCA (see Table 1).

Fig. 7.

Denoising by FBsPCA, PTsPCA, and PCA. n = 10⁴ and L = 55 pixels. (a) clean image. (b) noisy image (SNR = (1/20)). (c) FBsPCA denoised. (d) PTsPCA denoised. (e) PCA denoised.

Table 1.

Denoising Effect: FBsPCA, PTsPCA, and PCA

	MSE (10−5)	PSNR (dB)	1-MSSIM (10−6)
FBsPCA	6.6	20.2	4.4
PTsPCA	7.7	19.5	5.9
PCA	10.1	18.3	5.9

7. SUMMARY AND DISCUSSION

In this paper we adapted a sampling criterion that was originally proposed in [8] into the framework of steerable PCA. The Fourier–Bessel transform with the modified cutoff criterion R_kq ≤ πL is approximately unitary and keeps the statistics of white noise approximately unchanged. This sampling criterion obtains maximum information from a set of images and their rotated and reflected copies while preventing aliasing. Instead of constructing the invariant covariance matrix from the original images and their rotated and reflected copies, we compute the covariance matrix from the truncated Fourier–Bessel expansion coefficients. The covariance matrix has a special block diagonal structure that allows us to perform PCA on different angular frequencies separately. The block diagonal structure was also observed and utilized in previous algorithms for steerable PCA. However, we show here that the block size must shrink as the angular frequency increases in order to avoid aliasing and spurious eigenimages.

While steerable PCA has found applications in computer vision and pattern recognition, this work has been mostly motivated by its application to cryo-EM. Besides compression and denoising of the experimental images, it is worth mentioning that under the assumption that the viewing directions of the images are uniformly distributed over the sphere, Kam has previously demonstrated [13] that the covariance matrix of the images is related to the expansion coefficients of the molecule in spherical harmonics. Our FBsPCA can therefore be applied in conjunction to Kam’s approach.

When applying PCA to cryo-EM images, one has to take into account the fact that cryo-EM images are not perfectly centered. It is well known that the unknown translational shifts can be estimated experimentally using the double exposure technique. Specifically, the images that are analyzed by PCA are obtained from an initial low-dose exposure, while the centers of the images are determined from a second exposure with higher dose. Furthermore, image acquisition is typically followed by an iterative global translational alignment procedure in which images are translationally aligned with their sample mean, which was noted earlier to be radially symmetric. It has been observed that such translational alignment measures produce images whose centers are off by only a few pixels from their true centers. Such small translational misalignments mainly impact the high frequencies of the eigenimages. The general problem of performing PCA for a set of images and their rotations and translations, or other noncompact groups of transformations, is beyond the scope of this paper.

Another important consideration when applying PCA to cryo-EM images is the contrast transfer function (CTF) of the microscope. The images are not simply projections of the molecule, but are rather convolved with a CTF. The Fourier transform of the CTF is real valued and oscillates between positive and negative values. The frequencies at which the CTF vanishes are known as “zero crossings.” The images do not carry any information about the molecule at zero crossing frequencies. The CTF depends on the defocus value, and changing the defocus value changes the location of the zero crossings. Therefore, projection images from several different defocus values are acquired. When performing PCA for the images, one must take into account the fact that they belong to several defocus groups characterized by different CTFs. One possible way of circumventing this issue is to apply CTF correction to the images prior to PCA. A popular CTF correction procedure is phase flipping, which has the advantage of not altering the noise statistics. Alternatively, one can estimate the sample covariance matrix corresponding to each defocus group separately, and then combine these estimators using a least squares procedure in order to estimate the covariance matrix of projection images that are unaffected by CTF. The optimal procedure for performing PCA for images that belong to different defocus groups is a topic for future research.

Finally, we remark that the Fourier–Bessel basis can be replaced in our framework with other suitable bases. For example, the 2D prolate spheroidal wave functions (PSWFs) on a disk [14] enjoy from many properties that make them attractive for steerable PCA. In particular, among all band limited functions of a given band limit they are the mostly spatially concentrated in the disk. They also have a separation of variables form which makes them convenient for steerable PCA. However, an accurate numerical evaluation of the PSWF requires more sophistication compared to the Bessel functions, which is the reason why we have not applied PSWFs here.