Image-derived generative modeling of pseudo-macromolecular structures – towards the statistical assessment of Electron CryoTomography template matching

Abstract

Cellular Electron CryoTomography (CECT) is a 3D imaging technique that captures information about the structure and spatial organization of macromolecular complexes within single cells, in near-native state and at sub-molecular resolution. Although template matching is often used to locate macromolecules in a CECT image, it is insufficient as it only measures the relative structural similarity. Therefore, it is preferable to assess the statistical credibility of the decision through hypothesis testing, requiring many templates derived from a diverse population of macromolecular structures. Due to the very limited number of known structures, we need a generative model to efficiently and reliably sample pseudo-structures from the complex distribution of macromolecular structures. To address this challenge, we propose a novel image-derived approach for performing hypothesis testing for template matching by constructing generative models using the generative adversarial network (GAN). We conducted hypothesis testing experiments for template matching on both simulated and experimental subtomograms, allowing us to conclude the identity of subtomograms with high statistical credibility and significantly reducing false positives. Our general approach can be extended to assess other template matching tasks, such as face, iris, and fingerprint scans, where statistical tests are also very important.

1. Introduction

The cell is the basic structural and functional unit of all living organisms. Biochemical processes of the living cell are often catalyzed by tiny cellular machines called macromolecular complexes. To fully understand these cellular processes, it is extremely helpful to systematically extract the structure and spatial organization of macromolecular complexes in single cells. Cellular Electron Cryo Tomography (CECT) [2] is a powerful 3D imaging tool that enables the study of sub-cellular structures at near-native state and in sub-molecular resolution. However, the quality of the reconstructed CECT images (a.k.a. tomograms) suffer from many current imaging limitations, such as low signal-to-noise ratio (SNR), missing wedge and limited number of angular samples, to a point where interpretation by visual inspection is impractical. As such, locating instances of macromolecular complexes inside tomograms has remained an extremely challenging computer vision problem [4]. A popular method for this task has been template matching, which we describe now.

Let ${𝒮}_{known}$ denote the set of known macromolecular complexes, let $P$ denote a cubic subvolume of a tomogram containing a single macromolecule (a.k.a. subtomogram), and let $T\left(C\right)$ denote the template derived from a macromolecule $C$. Given $P$ and a known complex $KC\in {𝒮}_{known},$ template matching is a method for deciding whether $P$ contains an identical macromolecule as $KC$ by using the Pearson cross-correlation score $c(P,T(KC\left)\right)$ [6]. To calculate the score, $T\left(KC\right)$ is first rigidly aligned against $P$ using a fast alignment method[6, 21] and then the score is computed with missing wedge compensation [5].

In most cases, template matching makes the decision based on a single chosen threshold for the correlation score or the complex most similar to $P$ is declared a match by default from having the highest score from a batch of known macromolecules. However, as cross-correlation is only a relative measure of similarity, this approach is neither rigorous nor statistically meaningful. Instead, hypothesis testing is preferred for quantitatively assessing the statistical credibility of the decision. Performing an accurate hypothesis test is extremely challenging since the number of macromolecules with known structures is very limited. In addition, the structural distribution of macromolecules is highly complex. Across space and time, macromolecules typically adopt different conformations as part of their function and dynamically interact with other macromolecules [22]. Across different species and cell types, the majority of macromolecules are still unknown [9].

In this paper, we propose a statistically rigorous treatment of template matching using a novel, image-derived, Monte Carlo approach for performing hypothesis testing and calculating empirical p-values[10]. The Monte Carlo samples are pseudo-macromolecular complexes (denoted henceforth as C₀) sampled from the structure distribution $f_{structure}$ of macromolecules. We used a 3D-WGAN to learn $f_{structure}$ from a collection of known complexes (see Section 2.1). Our procedure is as follows:

(Step 1) Train a 3D-WGAN to learn the structural distribution $f_{structure}$.

(Step 2) Determine the macromolecule of interest $C_{Interest}:=\arg {\max }_{KC\in {𝒮}_{known}}c(P,T(KC\left)\right)$ to be the known macromolecule with the highest alignment score.

(Step 3) Using pseudo-complexes $C_0~f_{structure}$ generated by the 3D-WGAN and sampled away from $C_{Interest}$ (see Section 2.2), perform a hypothesis test for whether $P$ contains a macromolecule identical to $C_{Interest}$ and calculate the empirical p-value under the null hypothesis $H_0$ (see Section 2.3).

To formally establish the hypothesis testing procedure, we specify the null hypothesis $H_0$ and the alternative hypothesis $H_A$ as the following:

$$H_0:P\mathrm{does}\mathrm{not}\mathrm{contain}\mathrm{a}\mathrm{macromolecule}\mathrm{identical}\mathrm{to}{C}_{Interest}$$ (1)

$$H_A:P\mathrm{contains}\mathrm{a}\mathrm{macromolecule}\mathrm{identical}\mathrm{to}{C}_{Interest}$$ (2)

The statistical credibility is assessed by the p-value, which is the probability of obtaining $c\left(P,T\left(C_0\right)\right)$ at least as extreme as $c\left(P,T\left(C_{Interest}\right)\right)$, given that $H_0$ is true. The lower the p-value, the stronger the evidence against $H_0$, giving more statistical credibility that $H_A$ is true. To calculate an empirical p-value, we randomly sample pseudo-macromolecular complexes $C_0~f_{structure}$ to derive a Monte Carlo empirical distribution of the test statistics $c\left(P,T\left(C_0\right)\right)$ under the null.

Throughout this paper, let ${𝒮}_{pseudo}$ denote the set of pseudo-complexes (i.e. macromolecules generated from the 3D-WGAN). We emphasize that ${𝒮}_{pseudo}$ and ${𝒮}_{known}$ have important differences:

• The identity of $KC\in {𝒮}_{known}$ is known and $T\left(KC\right)$ is used to identify the macromolecule in a subtomogram through the method of template matching.

• The identity of $C_0\in {𝒮}_{pseudo}$ is unknown and $T\left(C_0\right)$ is only used for hypothesis testing.

As a recent advancement in unsupervised deep learning, the generative adversarial network (GAN) learns the distribution of training images on the image manifold and generates highly realistic images [7]. The training process of the GAN is akin to a minimax game between two neural network adversaries, the generator and the discriminator. The generator seeks to improve its output images by minimizing the discriminator’s classification accuracy while the discriminator seeks to maximize its accuracy. After training, the generator can produce diverse and realistic images from the distribution of the original training images. To our knowledge, no method exists for constructing such generative models for template search for CECT data.

Our contributions are summarized as follows:

1. A novel, Monte Carlo approach for statistical assessing template matching through hypothesis testing to calculate empirical p-values.

2. An approach for generating the density maps of pseudo-macromolecular complexes, using the 3D Deep Convolutional Wasserstein GAN (3D-WGAN). We showed that our model is able to capture the shape manifold of macromolecules, and sample realistic and diverse pseudo-complexes from the structural distribution of macromolecules.

2. Methods

2.1. Learning f_structure with a 3D-WGAN

As Step 1 of our procedure, we construct a generative model $G:{\mathbb{R}}^{100}\to {𝒮}_{pseudo}$ to learn the structural distribution of macromolecules f_structure. Our generative model combines the 3D-GAN [20] with the Improved Wasserstein GAN [8]. In our paper, we refer to our model as 3D Deep Convolutional Wasserstein GAN (3D-WGAN)¹. The network architecture of our 3D-WGAN is presented in Figure 1. Inspired by [20], our network generator and the discriminator are implemented each with four convolution layers of stride 2 and kernel size 4³, which we chose to be a factor of 64 to reduce the checkerboard artifact [16]. We found that using half as many filters as[20] was sufficient for the GAN to stabilize at producing good results. In the hidden layers, we used the LeakyReLU activation with $\alpha =0.2$. Following [1, 8], we only used Batch Normalization [11] in the generator.

Figure 1:

The network architecture for both the generator and discriminator of the 3D-WGAN. Each convolution layer is labeled in the format N − K × K × K − S, signifying N filters with kernel size K³ and stride S.

A common problem for GANs with especially limited data is mode collapse, which occurs when the generator only produces structures with very low diversity. We adopted the minibatch discrimination layer in the discriminator (see [18]) to reduce the collapse of the generator by penalizing low-entropy generators. This layer allows the discriminator to observe many samples at once, so that it can also take entropy of a batch of samples into account when deciding between real and pseudo images.

In our experiments, each macromolecular complex was represented as a 3D gray-scale image (a.k.a. density map) with 64³ voxels and 0.6nm pixel size. We constructed our dataset with 15 experimental macromolecular complexes that are diverse in shape and size (see third row of Figure 3). To prevent overfitting to a specific orientation or structure, we performed data augmentation and rotated each structure 600 times for a total of 9000 training structures. As the 3D convolution operation is not rotation invariant, this data augmentation improves the training of the 3D-WGAN. We trained the 3D-WGAN using a batch size of 64 and the Adam optimizer with ${\beta }_1=0.5,{\beta }_2=0.99$, and the learning rate as 0.0001, shown to be successful in previous works [17, 20]. Following [8], we trained the discriminator ten times as often as the generator, and we used a gradient penalty of 10.

Figure 3:

Let $C_{ij}$ denote the structure in the $i^{\text{th}}$ row and the $j^{\text{th}}$ column. (A) Each (red) $C_{1j}\in {𝒮}_{pseudo}$, and (blue) $C_{2j}\in {𝒮}_{known}$ is the most similar structure to $C_{1j}$ in ${𝒮}_{known}$. (B) Similarly, Each (red) $C_{3j}\in {𝒮}_{known}$ and is listed according to its PDB ID. Each $C_{4j}\in {𝒮}_{pseudo}$ is the most similar structure to $C_{3j}$ in ${𝒮}_{pseudo}$.

2.2. Sampling pseudo-complexes far away from $C_{Interest}$

Under the null, the instance of $C_{Interest}$ should be not be covered by the distribution of pseudo-complexes. This procedure also reduces the chance of sampling pseudo-complexes so similar to $C_{Interest}$ that they could be viewed as copies of $C_{Interest}$ in the hypothesis test. With the following three-step procedure, we could sample pseudo-complexes from $f_{structure}$ to be “far away” from $C_{Interest}$ in the latent representation of $f_{structure}$:

1. Regressor (a.k.a. Inverse Generator): We trained a regressor for the inverse map of the generator, using a 3D extension of the AutoEncoder GAN model (see [14]). Instead of using cross-entropy loss, we used the sum squared error of the reconstructed images since our images were not normalized to (0, 1) by a sigmoid. The network architecture of the regressor is an exact mirror image of the 3D-WGAN generator.

2. Kernel Density Estimator (KDE): We trained a KDE to learn a distribution ε of the latent representation of $C_{Interest}$, given by regressor’s output for 300 random rotations of $C_{Interest}$. The KDE’s bandwidth was determined using 3-fold cross validation.

3. Bayes Classifier: Let $\pi (\pi \ll 1)$ be the prior probability of pseudo-complexes, which can be estimated from the data. Denote $𝒩$ as the standard 100-dimensional multivariate Gaussian. The decision boundary to distinguish the distribution of pseudo-complexes and $C_{Interest}$ is where the probability of a complex being a pseudo-complex is the same as the probability of a complex being the known complex $C_{Interest}$.

According to Bayes rule, the rejection region can be written as:

$$ℛ=\left\{G\left(v\right):v\in {\mathbb{R}}^{100}\text{such}\text{that}𝒩\left(v\right)<\pi \cdot ℰ\left(v\right)\right\}$$

. When we sample $v$ from $𝒩$, we reject the members of $ℛ$.

2.3. Monte Carlo approach for evaluating the statistical credibility of template matching

After determining the complex of interest $C_{Interest}$ in Step 2, we perform a statistical assessment of template matching for Step 3 by calculating an empirical p-value using pseudo-macromolecular complexes. The true p-value $p$ is the probability of obtaining results at least as extreme as the observed $c\left(P,T\left(C_{Interest}\right)\right)$ given the null hypothesis $H_0$ is true. Since the distribution of $C_0$ under $f_{structure}$ is unknown, we use a Monte Carlo simulation to obtain an unbiased empirical p-value $\widehat{p}$ by ranking the observed test statistic $c\left(P,T\left(C_{Interest}\right)\right)$ amongst the alignment scores of complexes sampled from the learned distribution of the 3D-WGAN. By the strong law of large numbers, our empirical p-value converges almost surely to the true p-value as the number of Monte Carlo samples $B\to \mathrm{\infty }$.

$$p={\mathbb{E}}_{H_0;C_0~f_{structure}}\left[\mathbb{I}\left\{c\left(P,T\left(C_{Interest}\right)\right)\le c\left(P,T\left(C_0\right)\right)\right\}\mid C_0\notin ℛ\right]$$ (3)

$$=\mathrm{P}\mathrm{r}\left(c\left(P,T\left(C_{Interest}\right)\right)\le c\left(P,T\left(C_0\right)\right)\mid C_0\notin ℛ\right)$$ (4)

$$\widehat{p}=B^{-1}\sum _{b=1}^B\mathbb{I}[c\left(P,T\left(C_{Interest}\right)\right)\le c(P,T(C_0^{\left(b\right)}))]\underset{B\to \mathrm{\infty }}{\stackrel{\text{a.s.}}{⟶}}p$$ (5)

where ${\{C_0^{\left(b\right)}\}}_{1\le b\le B}$ are Monte Carlo samples from $f_{structure}$ and out of the rejection region $ℛ$, specified in Section 2.2.

The statistical credibility of the decision is measured by the p-value: the smaller the p-value, the more statistical credibility we have to reject the null and to support the alternative. Therefore, when $c\left(P,T\left(C_{Interest}\right)\right)$ is ranked in the highest 1% of the templates used for template matching (i.e. $\widehat{p}\le 0.01$), we could conclude that $P$ contains a macromolecule identical to $C_{Interest}$ with high confidence. We note here that in CECT template matching tasks, there is usually a large number of hypothesis tests for different subtomograms matching, which will consequently generate a large number of p-values. Multiple comparison is also suggested to be adjusted to control false discovery rate [3] to provide a stringent statistical criteria.

3. Results

3.1. Examples of pseudo-macromolecular complexes

Randomly selected pseudo-complexes from the 3D-WGAN are shown in Figure 2. Figure 3A shows the nearest neighbors of pseudo-complexes from ${𝒮}_{known}$. Most pseudo-complexes exhibit similar structure as their nearest neighbor in ${𝒮}_{known}$ and resemble the same macromolecular complex. Figure 3B shows the nearest neighbors of known complexes (labeled by their Protein Data Bank (PDB) ID²) from a set of 10,000 generated pseudo-complexes. We defined the metric between complexes as the L₂ norm on the fully-connected layer of the discriminator, which is a high-level feature representation of each complex [20]. Even as nearest neighbors of known complexes, the pseudo-complexes in Figure 3 are visually not identical to the known complexes. Non-rigid differences between the pseudo-complexes and the known complexes make these pseudo-complexes good candidates for hypothesis testing. This shows that our model can produce meaningful pseudo-complexes that have recognizable structural similarities to the training structures. A practical advantage to using the 3D-WGAN is that the generation of pseudo-complexes can be significantly sped up with a GPU. For example, our GTX 1080 Ti GPU could generate 10,000 structures in about 20s (0.002s/image).

Figure 2:

Random pseudo-macromolecular complexes generated with the 3D-WGAN.

3.2. Statistical assessment of template matching on simulated subtomograms

Template matching is applied to real data to decide whether an unidentified subtomogram contains a macromolecule of identical structure to some known macromolecule. For every $KC\in {𝒮}_{known}$, we simulated³ a subtomogram $P_{KC}$ containing $KC$. The hypothesis test consists of 985 randomly generated pseudo-templates $T\left(C_0\right)\in {𝒮}_{pseudo}$ and 15 templates of known structure $T\left(KC\right)\in {𝒮}_{known}$. We performed template matching on $P_{KC}$ following the three-step process described in Section 1. A simulated test was declared successful if both conditions are satisfied:

(Cond. 1) Highest correlation: $KC$ is chosen to be $C_{Interest}$. In other words, $KC$ satisfies $c\left(P_{KC},T\left(KC\right)\right)\ge c(P_{KC},T(C^{\mathrm{\prime }})),\forall C^{\mathrm{\prime }}\in {𝒮}_{known}$

(Cond. 2) Low p-value: The hypothesis test has empirical p-value $\widehat{p}\le 0.01$.

As we perform a test for each $KC\in {𝒮}_{known}$, we performed a total of 15 tests and achieved an average success rate of 12/15 = 80%, indicating high power of our hypothesis testing procedure. Two successful cases are shown in Figure 4. Others are shown in Supplementary Sections 1.5 (two unsuccessful cases) and 1.6 (rest of results).

Figure 4:

Results from successful hypothesis tests on simulated subtomograms containing 1LB3 and 4V7R. The blue histograms model the distribution of $c\left(P_{KC},T\left(C_0\right)\right),C_0\in {𝒮}_{pseudo}$. The vertical dashed lines denote $c(P_{KC},T(C^{\mathrm{\prime }})),C^{\mathrm{\prime }}\in {𝒮}_{known}$, with the ten highest labeled with the PDB ID of $C^{\mathrm{\text{'}}}$ and a rank out of the 1000 (i.e. the empirical p-value). $C_{Interest}$ is highlighted with the red color. The green line marks the p-value threshold of 0.01.

From the smooth distribution of histograms in Figure 4, we can deduce that our generative model samples diverse and realistic pseudo-complexes from $f_{structure}$, instead of reproducing the known complexes, which would result in the histograms clustering only around the vertical lines.

3.3. Statistical assessment of template matching on experimental subtomograms of ribosomes

With the same hypothesis testing procedure, we performed template matching on experimental subtomograms of Yeast 80S ribosome (PDB ID: 4V7R) from the EMPIAR-10045 database [12]. We used the 07 tomogram, which contains 376 subtomograms in total. The experimental subtomograms were originally 200³ voxels with 0.217nm pixel size. For our experiments, they were preprocessed with a 2.17nm Gaussian blur and resized to 64³ voxels with 0.6nm pixel size, the same dimension and pixel size as the density maps of the training structures and the simulated subtomograms. As shown in Figure 5A, these subtomograms are visually much noisier than the simulated ones, which illustrates the fact that CECT template matching is extremely challenging. Without preprocessing, it is extremely difficult to even visually detect the macromolecular structure contained within the subtomograms.

Figure 5:

(A) Slices of raw subtomograms containing ribosome; (B) Results from hypothesis tests on experimental subtomograms of ribosome. 0000001 is successful and 000130 is unsuccessful.

With a p-value cutoff at 0.01, there were 264 (70.21%) successful tests, where Cond. 1 and Cond. 2 were satisfied for a known ribosome complex of interest. 45 subtomograms (11.97%) resulted in Cond. 1 failure. 67 subtomograms (17.82%) resulted in Cond. 2 failure. Figure 5B shows one successful and a case of Cond. 2 failure from our experiments.

We note that due to our rescaling procedure, some subtomograms may be missing part of their ribosome structure, which may cause template matching to fail. Therefore, we further selected a subset of 100 subtomograms with the highest ribosome alignment score, corresponding to high confidence in containing the whole ribosome structure. 92 tests were successful, 1 resulted in Cond. 1 failure, and 7 resulted in Cond. 2 failure.

3.4. Detecting false positives from template matching

When the subtomogram does not contain a macromolecule or when the macromolecule in the subtomogram does not match with any of the known templates, it is desirable for template matching to conclude that no template matches with the given subtomogram. This is not possible if the template with the highest alignment scores is always concluded to be a match. Even when thresholding, it is difficult to choose a single cutoff that works in all possible cases and requires hyperparameter tuning. Using 20 experimental subtomograms that do not contain any macromolecules, we performed template matching along with our hypothesis testing procedure and we were able to prevent 8 (40%) of the false positives that would have occurred if we simply chose the highest alignment score as a match. Our statistical testing method makes more reliable claims on the results of template matching and can significantly reduce the number of false positives.

3.5. Learning the shape manifold of macromolecules

Similar to previous works [17, 20] with GANs, we found that our model was able to capture the shape manifold of macromolecular complexes. We show this by interpolating between the latent vectors of S and D, resulting in a smooth transition from a proteasome (PDB ID: 3DY4) to a ribosome (PDB ID: 4V7R) as shown in Figure 6. Our starting point S was the latent vector of the nearest pseudo-complex for the proteasome, and our ending point D was the latent vector of the nearest pseudo-complex for the ribosome. We generated the pseudo-complex at the i^th step with the input vector S +(D − S)· i. This smooth “deformation” from one structure to another illustrates the shape manifold of macromolecules and results in an effect similar to deformable image registration. Since deformable image registration can be quite computationally expensive (i.e. Large Deformation Diffeomorphic Metric Mapping [15]), with a well-trained model, our 3D-WGAN model could potentially be used as a computationally efficient heuristic for deformable registration of 3D shapes.

Figure 6:

Intermediate shapes generated by interpolating between the latent representation of proteasome (PDB ID: 3DY4) and ribosome (PDB ID: 4V7R). This shows that our generative model can learn the shape manifold of macromolecular structures.

4. Conclusion

Without hypothesis testing, existing template matching approaches are not rigorous and are not statistically credible enough. Physical limitations to CECT remain a major difficulty that can bias the cross-correlation score and may even cause template matching to fail. To reliably conclude the identity of a subtomogram and to reduce false positive rates, we propose an image-derived approach for performing hypothesis testing for template matching by constructing a generative model for macromolecular complexes. We used the 3D-WGAN since it could efficiently produce a diverse population of novel pseudo-macromolecular complexes that are not simply rigid rotations of the original training structures. By sampling from the learned distribution of macromolecules $f_{structure}$, we could successfully conclude, with both simulated and experimental subtomograms, with high statistical credibility that the given subtomogram contains a macromolecule identical to the complex of interest. In addition, the 3D-WGAN generative model has potentially other applications in CECT. We have shown that the 3D-WGAN can learn the shape manifold of macromolecules [20]. By interpolating between latent representations, a 3D-WGAN can be used to visualize smooth transitions between structures. This ability for smooth deformations can be potentially extended to a heuristic for deformable image registration.

In addition, our approach is not limited to the 3D-WGAN and works with any reasonable generative model. A future work would include experimenting other generative approaches (e.g. VAE, shape-space modeling [15]) with the hypothesis testing procedure. The current GAN approach may also be improved with larger training sets or more complex architecture (e.g. [13]). Finally, our procedure of using generative modeling, particularly the GAN, for hypothesis testing can be generalized to be applicable in many other template matching tasks with 2D or 3D images. Some prominent examples include object detection and fingerprint scans, where statistical tests are very important to ensure the quality of the result and to avoid false positives [19].