Denoising and covariance estimation of single particle cryo-EM images

Abstract

The problem of image restoration in cryo-EM entails correcting for the effects of the Contrast Transfer Function (CTF) and noise. Popular methods for image restoration include ‘phase flipping’, which corrects only for the Fourier phases but not amplitudes, and Wiener filtering, which requires the spectral signal to noise ratio. We propose a new image restoration method which we call ‘Covariance Wiener Filtering (CWF). In CWF, the covariance matrix of the projection images is used within the classical Wiener filtering framework for solving the image restoration deconvolution problem. Our estimation procedure for the covariance matrix is new and successfully corrects for the CTF. We demonstrate the efficacy of CWF by applying it to restore both simulated and experimental cryo-EM images. Results with experimental datasets demonstrate that CWF provides a good way to evaluate the particle images and to see what the dataset contains even without 2D classification and averaging.

1. Introduction

Single particle reconstruction (SPR) using cryo-electron microscopy (cryo-EM) is a rapidly advancing technique for determining the structure of biological macromolecules at near-atomic resolution directly in their native state, without any need for crystallization (Bai et al., 2015; Milne et al., 2013; Nogales, 2016; Sigworth; Kühlbrandt, 2014). In SPR, 3D reconstructions are estimated by combining multiple noisy 2 D tomographic projections of macromolecules in different unknown orientations.

The acquired data consists of multiple micrographs from which particle images are extracted in the first step of the computational pipeline. Next, the images are grouped together by similarity in the 2D classification and averaging step (Zhao and Singer, 2014; Park and Chirikjian, 2014). Class averages can be used to inspect the underlying particles, and to estimate viewing angles and form a low resolution ab initio 3D model. Subsequently, this 3D model is refined to high resolution, and 3D classification might be performed as well.

In this paper we propose an image restoration method that provides a way for visualizing the particle images without performing any 2D classification. While noise reduction is achieved in 2D classification by averaging together different particle images, our method operates on each image separately, and performs Contrast Transfer Function (CTF) correction and denoising in a single step.

Existing image restoration techniques (for denoising and CTF correction) can be broadly categorized into two kinds of approaches (Penczek, 2010). The first is an approach known as ‘phase flipping’, which involves flipping the sign of the Fourier coefficients at frequencies for which the CTF is negative. Consequently, phase flipping restores the correct phases of the Fourier coefficients, but ignores the effect of the CTF on the amplitudes. Phase flipping preserves the noise statistics and is easy to implement, leading to its widespread usage in several cryo-EM software packages. However, it is suboptimal because it does not restore the correct Fourier amplitudes of the images. The second commonly used approach is Wiener filter based restoration, to which we refer here as traditional Wiener filtering (TWF). Wiener filtering takes into account both the phases and amplitudes of the Fourier coefficients, unlike phase flipping. However, calculation of the Wiener filter coefficients requires prior estimation of the spectral signal to noise ratio (SSNR) of the signal, which by itself is a challenging problem. It is therefore customary to either treat the SSNR as a precomputed constant as in the software package SPIDER (Frank et al., 1996), or to apply Wiener filtering only at later stages of the 3D reconstruction pipeline when the noise level is sufficiently low, such as in EMAN2 (Tang et al., 2007). It is also possible to use a combination of the two approaches, by first phase flipping the 2D images, and later correct only for the amplitudes in the 3D reconstruction step, as in IMAGIC (van Heel et al., 1996; van Heel et al., 2000). Despite its simplicity, there are several drawbacks to TWF. First, it cannot restore information at the zero crossings of the CTF. Second, it requires estimation of the SSNR. Third, it is restrictive to the Fourier basis which is a fixed basis not adaptive to the image dataset.

We refer to our proposed method as Covariance Wiener Filtering (CWF). CWF consists of first estimating the CTF-corrected covariance matrix of the underlying clean 2D projection images, followed by application of the Wiener filter to denoise the images. Unlike phase flipping, CWF takes into account both the phases and magnitudes of the images. Moreover, unlike TWF that always operates in the data-independent Fourier domain, CWF is performed in the data-dependent basis of principal components (i.e., eigenimages). Crucially, CWF can be applied at preliminary stages of data processing on raw 2D particle images. The resulting denoised images can be used for an early inspection of the dataset, to identify the associated symmetry, and to eliminate ‘bad’ particle images prior to 2D classification and 3D reconstruction. Additionally, the estimation of the 2D covariance matrix is itself of interest, for example, in Kam’s approach for 3D reconstruction (Kam, 1980; Bhamre et al., 2015).

The paper is organized as follows: Sections 2.1 and 2.2 detail the estimation of the covariance matrix for two different noise models, first for the simpler model of white noise, and second for the more realistic model of colored noise. In Section 2.3 we discuss the steerability property of the covariance matrix (Zhao et al., 2016). The associated deconvolution problem is solved to obtain denoised images using the estimated covariance matrix in Section 2.4. Finally in Section 3, we demonstrate CWF in a number of numerical experiments, with both simulated and experimental datasets. We obtain encouraging results for experimental datasets, in particular, those acquired with the modern direct electron detectors. Image features are clearly observed after CWF denoising. For reproducibility, the MATLAB code for CWF and its dependencies are available in the open source cryo-EM toolbox ASPIRE at www.spr.math.princeton.edu. The script cwf_script.m, calls the main function cwf.m

2. Methods

The first step of CWF is estimation of the covariance matrix of the underlying clean images, to which we refer as the population covariance. The second step of CWF is solving a deconvolution problem to recover the underlying clean images using the estimated covariance. In the rest of this section, we describe these steps in detail.

2.1. The model

The image formation model in cryo-EM under the linear, weak phase approximation (Frank, 1996) is given by

$$y_i=a_i*x_i+{ϵ}_i,i=\mathrm{1,2},\dots ,n,$$ (1)

where $n$ is the number of images, * denotes the convolution operation, $y_i$ is the noisy, CTF filtered ith image in real space, $x_i$ is the underlying clean projection image in real space, $a_i$ is the point spread function of the microscope that convolves with the clean image in real space, and ${ϵ}_i$ is additive Gaussian noise that corrupts the image, for each $i$. Taking the Fourier transform of Eq. (1) gives

$$Y_i=A_i{X}_i+{\xi }_i,i=\mathrm{1,2},\dots ,n,$$ (2)

where $Y_i,X_i$ and ${\xi }_i$ are now in Fourier space. $A_i$ is a diagonal operator, whose diagonal consists of the Fourier transform of the point spread function, and is also commonly known as the CTF. The CTF modulates the phases and the amplitudes of the Fourier coefficients of the image, and contains numerous zero crossings that correspond to frequencies at which no information is obtained. Any image restoration technique that aims to completely correct for the CTF must therefore correctly restore both the phases and the amplitudes. The zero crossings make CTF correction challenging since it cannot be trivially inverted. In experiments, different groups of images are acquired at different defocus values, in the hope that information that is lost from one group could be recovered from another group that has different zero crossings. In the experimental datasets used in this paper, the number of images per defocus group typically ranges from 50 to 1000.

In our statistical model, the Fourier transformed clean images $X_1,\dots ,X_n$ (viewed, for mathematical convenience, as vectors in ${\mathbb{C}}^p$, where $p$ is the number of pixels) are assumed to be independent, identically distributed (i.i.d.) samples from a distribution with mean $\mathbb{E}\left[\mathbf{X}\right]=\mu$ and covariance $\mathbb{E}\left[(\mathbf{X}-\mu )(\mathbf{X}-\mu {)}^T\right]=\Sigma$. Since the clean images are two-dimensional projections of the three-dimensional molecule in different orientations, the distribution of $\mathbf{X}$ in our model is determined by the three-dimensional structure, the distribution of orientations, the varying contrast due to changes in ice thickness, and structural variability, all of course unknown at this stage. The covariance matrix $\Sigma$ therefore represents the overall image variability due to these determinants. While these model assumptions do not necessarily hold in reality (Sorzano et al., 2007; Sorzano et al., 2015), they simplify the analysis and, as will be shown later lead to excellent denoising. Quoting George Box, “All models are wrong but some are useful” (Box, 1976).

Our denoising scheme requires $\mu$ and $\Sigma$. Since these quantities are not readily given, we estimate them from the noisy images themselves as follows. For simplicity, we first assume that the noise in our model is additive white Gaussian noise such that ${\xi }_i~𝒩\left(0,{\sigma }^2{I}_{p\times p}\right)$ in Eq. (2) are i.i.d. The white noise assumption is later replaced by that of the more realistic colored noise. First, notice from Eq. 2 it follows that

$$\mathbb{E}\left[{\mathbf{Y}}_i\right]=A_i\mathbb{E}\left[{\mathbf{X}}_i\right],i=\mathrm{1,2},\dots ,n.$$ (3)

So,

$$\begin{gathered} \mathbb{E}\left[\left({\mathbf{Y}}_i-\mathbb{E}\left[{\mathbf{Y}}_i\right]\right){\left({\mathbf{Y}}_i-\mathbb{E}\left[{\mathbf{Y}}_i\right]\right)}^T\right]=\mathbb{E}\left[A_i\left({\mathbf{X}}_i-\mu \right){\left({\mathbf{X}}_i-\mu \right)}^T{A}_i^T\right]+{\sigma }^{2}I \\ =A_i\Sigma A_i^T+{\sigma }^{2}I. \end{gathered}$$ (4)

Eq. (4) relates the second order statistics of the noisy images with the population covariance $\Sigma$ of the clean images, based on which we can estimate $\Sigma$.

Next, we construct estimators for the mean $\mu$ and population covariance $\Sigma$ using Eqs. (3) and (4). The mean $\mu$ of the dataset can be estimated as the solution to a least squares problem

$$\widehat{\mu }=\underset{\mu }{\text{arg}\text{min}}\sum _{i=1}^n{‖\left(Y_i-A_i\mu \right)‖}_2^2+\lambda \parallel \mu {\parallel }_2^2,$$ (5)

where $\lambda ⩾0$ is a regularization parameter. The solution to 5 is explicitly

$$\widehat{\mu }={\left(\sum _{i=1}^n{A}_i^T{A}_i+\lambda I\right)}^{-1}\left(\sum _{i=1}^n{A}_i^T{Y}_i\right).$$ (6)

The population covariance $\Sigma$ can be estimated as

$$\begin{gathered} \hat{\Sigma }=\underset{\Sigma }{\text{arg}\text{min}}\sum _{i=1}^n{‖\left(Y_i-\mathbb{E}\left[Y_i\right]\right){\left(Y_i-\mathbb{E}\left[Y_i\right]\right)}^T-\left(A_i\Sigma A_i^T+{\sigma }^{2}I\right)‖}_F^2 \\ =\underset{\Sigma }{\text{arg}\text{min}}\sum _{i=1}^n{‖A_i\Sigma A_i^T+{\sigma }^{2}I-C_i‖}_F^2, \end{gathered}$$ (7)

where $C_i=\left(Y_i-A_i\mu \right){\left(Y_i-A_i\mu \right)}^T$ and $\parallel \cdot {\parallel }_F$ is the Frobenius matrix norm. The estimators $\hat{\mu }$ and $\hat{\Sigma }$ can be shown to be consistent in the large sample limit $n\to \infty$, similar to the result in Appendix B of Katsevich et al. (2015).

To ensure that the estimated covariance is positive semidefinite (PSD), we project it onto the space of PSD matrices by computing its spectral decomposition and retaining only the non negative eigenvalues (and their corresponding eigenvectors). To solve Eq. (7), we differentiate the objective function with respect to $\Sigma$ and set the derivative to zero. This yields

$$\sum _{i=1}^n{A}_i^T{A}_i\hat{\Sigma }{A}_i^T{A}_i=\sum _{i=1}^n{A}_i^T{C}_{i}A-\sum _{i=1}^n{\sigma }^2{A}_i^T{A}_i.$$ (8)

Eq. (8) defines a system of linear equations for the elements of the matrix $\hat{\Sigma }$. However, direct inversion of this linear system is slow and computationally impractical for large image sizes. Notice that Eq. (8) can be written as

$$L\left(\hat{\Sigma }\right)=B,$$ (9)

where $L:{\mathbb{R}}^{p\times p}\to {\mathbb{R}}^{p\times p}$ is the linear operator acting on $\hat{\Sigma }$ defined by the left hand side of Eq. (8), and $B$ is the right hand side. Since applying $L$ only involves matrix multiplications, it can be computed fast, and the conjugate gradient method is employed to efficiently compute $\hat{\Sigma }$ instead of direct inversion, similar to how it is used in Andén et al. (2015).

Notice that $L\left(\hat{\Sigma }\right)$ is a PSD matrix whenever $\hat{\Sigma }$ is PSD (as a sum of PSD matrices), while $B$ may not necessarily be PSD due to finite sample fluctuations (i.e., $n$ is finite). It is therefore natural to project $B$ onto the cone of PSD matrices. This amounts to computing the spectral decomposition of $B$ and setting all negative eigenvalues to 0, which is an instance of eigenvalue thresholding.

We now describe an alternate eigenvalue thresholding procedure, better suited to cases in which the number of images $n$ is not exceedingly large. To that end, we first analyze the matrix $B$ when $X_i=0$ for all $i$, i.e., the input images are white noise images containing no signal. Let

$$M=\sum _{i=1}^n{A}_i^T{C}_i{A}_i=\sum _{i=1}^n{A}_i^T{Y}_i{Y}_i^T{A}_i.$$ (10)

Then, $\mathbb{E}\left[M\right]={\sigma }^2{\sum }_{i=1}^n{A}_i^T{A}_i$ and $B=M-\mathbb{E}\left[M\right]$. Let $S=\left(\mathbb{E}\right[M]{)}^{1/2}$, i.e. $S$ is PSD and $\mathbb{E}\left[M\right]=S^2$. Then multiplying both sides of Eq. (9) with $S^{-1}$ we get

$$S^{-1}L\left(\hat{\Sigma }\right)S^{-1}=S^{-1}\left(M-\mathbb{E}\left[M\right]\right)S^{-1}=S^{-1}M{S}^{-1}-I.$$ (11)

$S^{-1}M{S}^{-1}$ can be viewed as a sample covariance matrix of $n$ vectors in ${\mathbb{R}}^p$ whose population covariance is the identity matrix. When $p$ is fixed and $n$ goes to infinity, all eigenvalues of $S^{-1}M{S}^{-1}$ converge to 1. In practice, however, $n$ and $p$ are often comparable. In the limit $p,n\to \infty$ and $p/n\to \gamma$ with $0<\gamma <\infty$, the limiting spectral density of the eigenvalues converges to the Marčenko Pastur (MP) distribution (Marčenko and Pastur, 1967), given by

$$MP\left(x\right)=\frac{1}{2\pi }\frac{\sqrt{\left({\gamma }_{+}-x\right)\left(x-{\gamma }_{-}\right)}}{\gamma x}{1}_{\left[{\gamma }_{-},{\gamma }_{+}\right]},{\gamma }_{\pm }=(1\pm \sqrt{\gamma }{)}^2,$$ (12)

for $\gamma ⩽1$. It is therefore expected that $S^{-1}M{S}^{-1}$ would have eigenvalues (considerably) larger than 1, even in the pure white noise case. These large eigenvalues should not be mistakingly attributed to signal. In the case of images containing signal (plus noise), eigenvalues corresponding to the signal can only be detected if they reside outside of the support of the MP distribution. We use the method of Kritchman and Nadler (2008) to determine the number of eigenvalues corresponding to the signal. We then apply the operator norm eigenvalue shrinkage procedure (see Donoho et al.) to those eigenvalues, while setting all other eigenvalues to 0. We then use the conjugate gradient method¹ to solve Eq. (11) for $\hat{\Sigma }$, with the right hand side replaced with its shrinkage version. We observed in numerical simulations (see Fig. 3) that this procedure typically out-performs other shrinkage methods in terms of the accuracy of the estimated covariance matrix.

Fig. 3.

Relative MSE of the estimated covariance versus the number of images: The relative MSE of the estimated covariance $\hat{\Sigma }$, with and without using eigenvalue shrinkage, as a function of number of images, for synthetic data generated using EMDB-6454.

2.2. Covariance estimation with colored noise

So far, we assumed additive white Gaussian noise in the image formation process. In reality, the noise in experimental images is colored. That is, in the image formation model in Eq. (2), ${\xi }_i$ is additive colored Gaussian noise. We preprocess the images in order to “whiten” the noise. The noise power spectrum can be estimated, for example, using the pixels in the corners of the noisy projection images. To do this, we first estimate using correlograms the 2D autocorrelation of the corner pixels of the images which contain mostly noise and no signal. These corner pixels are used to estimate the 1 D autocorrelation, which is then extended to populate the 2D isotropic autocorrelation. We then calculate the Fourier transform of the 2D autocorrelation, which is the 2D power spectrum of noise. The noisy projection images in Fourier space are multiplied element-wise by the inverse of the estimated power spectral density, also called the whitening filter, so that the noise in the resulting images is approximately white. Let $W$ be the “whitening” filter, such that

$$W{Y}_i=W{A}_i{X}_i+W{\xi }_i,i=\mathrm{1,2},\dots ,n,$$ (13)

and $W{\xi }_i~𝒩\left(0,{\sigma }^{2}I\right)$.

Eq. (13) is reminiscent of Eq. (2). It is tempting to define a new “effective” CTF as $W{A}_i$ and estimate $\Sigma$ following the same procedure as in the case of white noise. However, the linear system akin to Eq. (8) for this case is ill-conditioned due to the product of $W$ with the CTF, and it takes a large number of iterations for conjugate gradient to converge to the desired solution. Instead, we seek an approach in which the linear system to solve is well conditioned as that in the case of white noise. Since the CTF’s $A_i,i=\mathrm{1,2},\dots n$ and the whitening filter $W$ are diagonal operators in the Fourier basis, they commute, and Eq. (13) becomes

$$W{Y}_i=A_{i}W{X}_i+W{\xi }_i,i=\mathrm{1,2},\dots ,n.$$ (14)

We therefore absorb $W$ into $X_i$, and estimate the matrix ${\Sigma }_W=W\Sigma W^T$ (the population covariance of $W\mathbf{X}$) using the same procedure as before. The population covariance $\Sigma$ is then estimated as

$$\hat{\Sigma }=W^{-1}{\hat{\Sigma }}_W{\left(W^T\right)}^{-1}.$$ (15)

2.3. Fourier-Bessel steerable PCA

The population covariance matrix $\Sigma$ must be invariant under inplane rotation of the projection images, therefore it is block diagonal in any steerable basis in which the basis elements are outer products of radial functions and angular Fourier modes. Following (Zhao et al., 2016), we choose to represent the images in a Fourier-Bessel basis and it suffices to estimate each diagonal block ${\Sigma }^{\left(k\right)}$, corresponding to the angular frequency $k$, separately. The Fourier-Bessel basis (Zhao et al., 2016) consists of $p_k$ basis functions (that satisfy the sampling criterion) for each angular frequency $k$, where $p_k$ decreases with increasing $k$. The matrix ${\Sigma }^{\left(k\right)}$ is thus of size $p_k\times p_k$.

An important property of the CTF’s $A_i$ and the whitening filter $W$ is that they are radially isotropic.² Therefore, the CTF’s and the whitening filter are also block diagonal in the Fourier Bessel basis. Eq. (8) (and its analog in the case of colored noise) is hence solved separately for each $k$ to estimate ${\Sigma }^{\left(k\right)}$.

2.4. Wiener filtering

The estimated covariance is further used to solve the associated deconvolution problem in Eq. (2) using Wiener filtering. The result is a denoised, CTF corrected image for each noisy, CTF affected measurement $Y_i$ for $i=\mathrm{1,2},\dots n$. We estimate $X_i$ in the white noise model using the Wiener filtering procedure as

$${\hat{X}}_i=\left(I-H_i{A}_i\right)\hat{\mu }+H_i{Y}_i,$$ (16)

where $H_i=\hat{\Sigma }{A}_i^T{\left(A_i\hat{\Sigma }{A}_i^T+{\sigma }^{2}I\right)}^{-1}$ is the linear Wiener filter (MacKay, 2004). In the case of colored noise,

$${\hat{X}}_i=\left(I-H_{i}W{A}_i\right)\hat{\mu }+H_i{Y}_i,$$ (17)

with $H_i=\hat{\Sigma }{A}_i^T{W}^T{\left(W{A}_i\hat{\Sigma }{A}_i^T{W}^T+{\sigma }^{2}I\right)}^{-1}$. Since the estimated covariance is block-diagonal in the Fourier Bessel basis, the Wiener filtering procedure is applied to the Fourier Bessel coefficients of the noisy images $Y_i$ for each angular frequency $k$ separately. The denoised Fourier Bessel expansion coefficients are used to reconstruct denoised images in Fourier space that are inverse Fourier transformed to acquire images in real space on a Cartesian grid.

2.5. Computational complexity

In practice, instead of each image being affected by a distinct CTF, all images within a given defocus group have the same CTF. So, given $D$ defocus groups with $d_i$ images in group $i$, one can equivalently minimize the objective function ${\sum }_{i=1}^D{d}_i\parallel \left(A_i\Sigma A_i^T+{\sigma }^{2}I\right)-{\sum }_{j=1}^{d_i}\frac{1}{d_i}\left(Y_{i_j}-\mathbb{E}\left[{\mathbf{Y}}_{ij}\right]\right){\left(Y_{i_j}-\mathbb{E}\left[{\mathbf{Y}}_{i_j}\right]\right)}^T{\parallel }_F^2$ in Eq. (7) (here $A_i$ denotes the CTF of the $i$’th defocus group, and $i_j$ index images in that group). As a result, the sums in Eq. (8) range from 1 to $D$ instead of from 1 to $n$, thereby reducing the computational cost of some operations. For images of size $L\times L$, estimating the mean using Eq. (6) takes $O\left(n{L}^2\right)$ (since $A_i$ is diagonal in the Fourier basis for each $i$). Computing the Fourier Bessel expansion coefficients takes $O\left(n{L}^3\right)$, as detailed in Zhao et al. (2016). When solving the linear system in Eq. (8) to estimate each ${\Sigma }^{\left(k\right)}$ separately, the matrices in Eq. (8) are of size $p_k\times p_k$. It is shown in Zhao et al. (2016) that ${\sum }_k{p}_k=O\left(L^2\right),{\sum }_k{p}_k^2=O\left(L^3\right)$, and ${\sum }_k{p}_k^3=O\left(L^4\right)$. While solving Eq. (9) using conjugate gradient for a given angular frequency, computing the action of the linear operation $L$ on ${\Sigma }^{\left(k\right)}$ takes $O\left(D{p}_k^3\right)$ per iteration, while computing $B$ takes $O\left(D{p}_k^3+n{p}_k^2\right)$. Thus, each iteration of conjugate gradient takes $O\left(D{\sum }_k{p}_k^3\right)$, that is, $O\left(D{L}^4\right)$ and there is also a one time computation of $O\left(n{L}^3\right)$. Wiener filtering the Fourier Bessel coefficients of an image for a given angular frequency $k$ takes $O\left(p_k^2\right)$. So the overall complexity for Wiener filtering the coefficients of all images is $O\left(n{L}^3\right)$. In summary, the overall complexity for CWF is $O\left(TD{L}^4+n{L}^3\right)$, where $T$ is the number of conjugate gradient iterations.

3. Results

In this section, we apply our algorithm to synthetic and experimental datasets to obtain denoised images. All algorithms are implemented in the UNIX environment, on a machine with 60 cores, running at 2.3 GHz, with total RAM of 1.5 TB. We perform numerical experiments with (i) a synthetic dataset with additive white and colored Gaussian noise and (ii) four experimental datasets, two of which were acquired with older detectors, and the other two with state-of-the-art direct electron detectors. For all the experimental datasets, the corresponding estimated CTF parameters were provided with the dataset. For all simulations, we use centered projection images. The algorithm does not require centered images. However, having non-centered images would result in an additional ‘blurring’ effect in the denoised images.

3.1. Simulated noisy dataset with white noise

For the first experiment with simulated data, we construct a synthetic dataset by modeling the image formation process in cryo-EM. The synthetic dataset is prepared from the 3D structure of the Plasmodium falciparum 80S ribosome bound to E-tRNA, available on the Electron Microscopy Data Bank (EMDB) as EMDB-6454. We first generate clean 2D projection images starting from a 3D volume, at directions sampled uniformly over the sphere, and then corrupt the generated clean projection images with different CTF’s and additive white Gaussian noise. The projection images are divided into 10 defocus groups, with the defocus value ranging from $1\mu \mathrm{m}$ to $4\mu \mathrm{m}$. The B-factor of the decay envelope was chosen as 10 Ų, the amplitude contrast as 7%, the voltage as 300 kV, and the spherical aberration as 2 mm. To ensure that the denoising quality of CWF is robust to the mean estimation of the dataset, the regularization parameter $\lambda$ in the least squares mean estimation in Eq. (6) was fixed at 1 for all the experiments described here.

Fig. 1 shows the results of denoising raw, CTF-affected noisy images with CWF and TWF at various levels of the SNR. We have used the EMAN2 (Tang et al., 2007) implementation of TWF (note that we perform phase flipping followed by TWF only on the raw images in EMAN2, and not on averages). The SNR used here is defined relative to the CTF affected images that constitute the clean signal, and is calculated as an average value for the entire dataset. Using 20 cores, calculating the Fourier Bessel coefficients took 79 s while covariance estimation and Wiener filtering together took 6 s in the experiment with SNR=1/60.

Fig. 1.

Synthetic white noise: A comparison of the denoising results of traditional Wiener filtering (TWF) and CWF for the synthetic dataset prepared from EMDB-6454, the P. falciparum 80S ribosome bound to E-tRNA. The dataset consists of 10,000 images of size 105 × 105, which are divided into 10 defocus groups, with the defocus value ranging from $1\mu \mathrm{m}$ to $4\mu \mathrm{m}$. The two rows in each subfigure correspond to two clean images belonging to different defocus groups; the first one belongs to the group with the smallest defocus value of $1\mu \mathrm{m}$, while the second image belongs to the group with the largest defocus value of $4\mu \mathrm{m}$.

It is seen that TWF works very well at high SNR (⩾ 1), but deteriorates at lower SNR’s as expected. Note that the denoising results of TWF depend strongly on the defocus value. The location of the zeros in the CTF is such that images corresponding to high defocus values preserve low frequency information, while images corresponding to low defocus values retain more high frequency information. With CWF, there is no such strong dependence on the defocus value, since the covariance matrix is estimated using information from all defocus groups.

Fig. 2a shows the relative MSE of denoised images as a function of the SNR of the dataset. The MSE (norm of the difference between the denoised image and the original, clean image) shown here corresponds to the same range of SNR’s (from 1/60 to 1) as in Fig. 1. Fig. 2b shows the relative MSE of the denoised images as a function of the number of images used to estimate the covariance in the experiment. The covariance estimation improves as the number of images in the dataset increases, and so the denoising is also expected to improve, as seen from Fig. 2b.

Fig. 2.

(a) Relative MSE versus the SNR, for a fixed number of images: The relative MSE of the denoised images as a function of the SNR, for synthetic data generated using EMDB-6454. The MSE reported here is averaged over all images. $n$ denotes the number of images used in the experiment. (b) Relative MSE versus the number of images, for a fixed SNR: The relative MSE of the denoised images as a function of the number of images, for synthetic data generated using EMDB-6454. The MSE reported here is averaged over all images.

The importance of the eigenvalue shrinkage procedure is elucidated in Fig. 3. Here, we compare the error in the estimated covariance with and without eigenvalue shrinkage, for varying number of images used in the experiment. The relative MSE of the estimated covariance $\hat{\Sigma }$ is defined as

$$MS{E}_{rel}=\frac{\parallel \Sigma -\hat{\Sigma }{\parallel }_F^2}{\parallel \Sigma {\parallel }_F^2}.$$ (18)

3.2. Simulated noisy dataset with colored noise

The noise that corrupts images in cryo-EM is not perfectly white, but often colored. To simulate this, we perform experiments with synthetic data generated from EMDB-6454 as described in 3.1, this time adding colored Gaussian noise with the noise response $f\left(\mathrm{k}\right)=\frac{1}{\sqrt{\left(1+{\mathrm{k}}^2\right)}}(\mathrm{k}$ is the radial frequency) to each clean, CTF-affected projection image. Fig. 4 shows the denoised images for this case.

Fig. 4.

Synthetic colored noise: Denoising results of CWF for the synthetic dataset with additive colored Gaussian noise, prepared from EMDB-6454, the P. falciparum 80S ribosome bound to E-tRNA, as detailed in the caption of Fig. 1.

3.3. Experimental dataset - TRPV1

We apply CWF to an experimental dataset of the TRPV1 ion channel, taken using a K2 direct electron detector. It is available on the public database Electron Microscope Pilot Image Archive (EMPIAR) as EMPIAR-10005, and the 3D reconstruction is available on EMDB as EMDB-5778, courtesy of Liao et al. (Liao et al., 2013). The dataset consists of 35,645 motion corrected, picked particle images of size 256 × 256 pixels with a pixel size of 1.2156 Å. Using 20 cores, calculating the Fourier Bessel coefficients took 312 s while covariance estimation and Wiener filtering together took 574 s. The result is shown in Fig. 5. CWF retains 384 eigenvalues of $\Sigma$.

Fig. 5.

Denoising an experimental dataset of TRPV1 (Liao et al., 2013): Here we show, for three images in the dataset, the raw image, the closest true projection image generated from the 3D reconstruction of the molecule (EMDB 5778), the denoised image obtained using TWF, and the denoised image obtained using CWF. In this experiment, 35,645 images of size 256 × 256 belonging to 935 defocus groups were used. The amplitude contrast is 10%, the spherical aberration is 2 mm, and the voltage is 300 kV.

3.4. Experimental dataset - 80S ribosome

We apply CWF to an experimental dataset of the Plasmodium falciparum 80S ribosome bound to the anti-protozoan drug emetine, taken using a FEI FALCON II 4k × 4k direct electron detector. The raw micrographs and picked particles are available on the public database EMPIAR as EMPIAR-10028, and the 3D reconstruction is available on EMDB as EMDB-2660, courtesy of Wong et al. (2014). The dataset we used was provided by Dr. Sjors Scheres, and consists of 105,247 motion corrected, picked particle images of size 360 × 360 with a pixel size of 1.34 Å. Using 20 cores, calculating the Fourier Bessel coefficients took 731 s while covariance estimation and Wiener filtering together took 385 s. The result is shown in Fig. 6. CWF retains 962 eigenvalues of $\Sigma$.

Fig. 6.

Denoising an experimental dataset of the 80S ribosome (Wong et al., 2014): Here we show, for three images in the dataset, the raw image, the closest true projection image generated from the 3D reconstruction of the molecule (EMDB 2660), the denoised image obtained using TWF, and the denoised image obtained using CWF. In this experiment, the first 30000 images out of the 105,247 images in the dataset were used for covariance estimation. The images are of size 360 × 360 and belong to 290 defocus groups. The amplitude contrast is 10%, the spherical aberration is 2mm, and the voltage is 300 kV.

3.5. Experimental dataset - $I{P}_{3}R1$

We apply CWF to an experimental dataset of the Inositol 1, 4, 5-triphosphate receptor 1(IP₃R1) provided by Dr. Irina Serysheva, obtained using the older Gatan 4k × 4k CCD camera (Ludtke et al., 2011). The 3D reconstruction obtained from this dataset is available on EMDB as EMDB-5278. The dataset consists of 37,382 images of size 256 × 256 pixels with a pixel size of 1.81 Å. Using 20 cores, calculating the Fourier Bessel coefficients took 429 s while covariance estimation and Wiener filtering together took 589 s. The result is shown in Fig. 7. CWF retains 290 eigenvalues of $\Sigma$.

Fig. 7.

Denoising an experimental dataset of IP₃R1 (Ludtke et al., 2011): Here we show, for three images in the dataset, the raw image, the closest true projection image generated from the 3D reconstruction of the molecule (EMDB 5278), the denoised image obtained using TWF, and the denoised image obtained using CWF. In this experiment, 37,382 images of size 256 × 256 belonging to 851 defocus groups were used. The amplitude contrast is 15%, the spherical aberration is 2 mm, and the voltage is 200 kV.

3.6. Experimental dataset −70 S ribosome

We apply CWF to an experimental dataset of the 70S ribosome provided by Dr. Joachim Frank’s group (Agirrezabala et al., 2012). This heterogeneous dataset consists of 216,517 images of size 250 × 250 pixels with a pixel size of 1.5 Å, obtained using the older TVIPS TEMCAM-F415 (4k × 4k) CCD detector. The 3D reconstruction obtained from this dataset is available on EMDB as EMDB5360. Using 20 cores, calculating the Fourier Bessel coefficients took 1174 s while covariance estimation and Wiener filtering together took 113 s. The result is shown in Fig. 8. CWF retains 219 eigenvalues of $\Sigma$.

Fig. 8.

Denoising an experimental dataset of 70S (Agirrezabala et al., 2012): Here we show, for three images in the dataset, the raw image, the closest true projection image generated from the 3D reconstruction of the molecule (EMDB 5360), the denoised image obtained using TWF, and the denoised image obtained using CWF. In this experiment, the first 99,979 images out of the 216,517 images in the dataset were used for covariance estimation. The images are of size 250 × 250 and belong to 38 defocus groups. The amplitude contrast is 10%, the spherical aberration is 2.26 mm, and the voltage is 300 kV.

3.7. Outlier detection

In the cryo-EM pipeline, a significant amount of time is spent on discarding outliers by visual inspection after the particle picking step. CWF provides an automatic way to classify picked particles into “good” particles and outliers. The classifier uses the contrast of a denoised image to determine if it is an outlier.

The specimen particles can be at various depths in the ice layer at the time of imaging, so the acquired projection images can have different contrasts. The contrast can be modeled as an additional scalar parameter $\alpha$ for each acquired noisy projection image as in Eq. (19), typically as a uniformly distributed random variable spread about its mean at 1.

$$Y_i={\alpha }_i{A}_i{X}_i+{\xi }_i,i=\mathrm{1,2},\dots ,n.$$ (19)

We absorb the contrast $\alpha$ into $\mathbf{X}$ and estimate ${\alpha }_i{X}_i$ in this case, using the same procedure as before. We perform an experiment with synthetic data generated using EMDB-6454 with additive colored Gaussian noise at SNR = 1/20, and $\alpha \in \left[\mathrm{0.75,1.5}\right].$ 10% of the projection images are replaced by “outliers”, that is, pure noise images containing no signal. Fig. 9c shows the estimated mean image $\mu$, and Fig. 9d shows the top 6 principal components of the estimated covariance $\hat{\Sigma }$, also known as eigenimages. Fig. 9a and b show a sample of raw and denoised images respectively. High contrast images enjoy a higher SNR and are thus of interest for subsequent steps of the pipeline. On the other hand, outlier images, which typically have low contrast after denoising, can be automatically detected by a linear classifier after CWF and discarded from the dataset. In the experiment shown in Fig. 9a and b, a classifier with a threshold of 0.95 for the contrast discards 95% of the outliers, while 3% of the inliers are also discarded in the process.

Fig. 9.

(a) Raw images: A sample of synthetic data generated using EMDB-6454 with additive colored Gaussian noise at SNR = 1/20.10% of the projection images are replaced by pure noise. The contrast parameter $\alpha$ ranges from 0.75 to 1.5. The outliers are shown in the last column. Inset in a yellow box is the contrast of each image. (b) Denoised images: The denoised images using CWF. Notice the low contrast outliers in the last column. (c) Estimated mean image. (d) Top 6 eigenimages: Inset in a yellow box is the corresponding eigenvalue.

One can also use a different classifier based on features like the relative energy of the image before and after denoising, etc. However, outliers that look like particles, for example, images belonging to a different class of a heterogeneous dataset which act as “contaminants”, are difficult to detect using this method.

4. Conclusion

In this paper we presented a new approach for image restoration of cryo-EM images, CWF, whose main algorithmic components are covariance estimation and deconvolution using Wiener filtering. CWF performs both CTF correction, by correcting the Fourier phases and amplitudes of the images, as well as denoising, by eliminating the noise thereby improving the SNR of the resulting images. In particular, since CWF applies Wiener filtering in the data-dependent basis of principal components (“eigenimages”), while TWF applies Wiener filtering in the data-independent Fourier basis, we see in numerical experiments that CWF performs better than TWF, and considerably better at high noise levels. We demonstrated the ability of CWF to restore images for several experimental datasets, acquired with both CCD detectors and the state-of-the-art direct electron detectors.

Due to the high noise level typical in cryo-EM images, 2D classification is performed before estimating a 3D ab initio model. Class averages enjoy a higher SNR and are used to estimate viewing angles and obtain an initial model. For future work, it remains to be seen whether the resulting denoised images from CWF can be directly used to estimate viewing angles, without performing classification and averaging. Another possible future direction is integration of CWF into existing 2D class averaging procedures in order to improve their performance.